Spring Boot Tutorial Brian Matthews
Featured Article: Unit and Integration Testing for Microservices Using Spring Boot Test and Test Containers
In association with the Dublin Java User Group and their partners, Brian will be joining us live online Tuesday 25th May to give answers to your questions – register here.
Over to you Brian!
The Demo Project
The demo project is a simple micro-service back-end for a to-do list application. The REST API is implemented using Spring Boot. It accepts HTTP + JSON requests from a client uses a relational database to stores to-do items assigned to users. The relational database used is MariaDB.
This demo project doesn’t have any security and little or no input validation. Let’s pretend that’s to keep things simple and rather than have me admit that it’s because I’m lazy.
The REST API exposes the end-points described in table below using Spring Boot Web MVC:
In the remainder of this article, for the sake of clarity and breviy, I have only included snippets of code. However, the full source is available from GitLab at https://gitlab.com/bmatthews68/testcontainers-article and can be downloaded by running the following
git clone command:
The to-do items are stored in the database table from the schema definition below:
The owner and to-do item identifiers are 128-bit type 1 UUIDs encoded as URL-safe base 64 strings which are globally unique, human-readable and relatively compact.
0 → urgent; 1 → high; 2 → medium; 3 → low.
0 → not started; 1 → in progress; 2 → done.
The data transfer object (DTO) below is used to represent the to-do item:
The data access object (DAO) persists data in a relational database using JDBC using
In the following sections we are going to consider approaches for unit and integration testing the REST API.
Unit testing focuses on testing a single unit of an overall system. However, there is no universally agreed definition of what constitutes a single unit. Is it a single process, a module or an individual class?
In Java applications we typically unit test at the individual class level using frameworks like JUnit, TestNG or Spock. We will use JUnit because it is included in Spring Boot’s curated bill of materials.
Two important considerations to bear in mind when automating unit tests are:
Not all classes are worth unit testing — Writing and maintaining tests for the getter and setter methods of simple POJOs such as TodoItemDTO.java is probably a waste of time.
100% code coverage doesn’t guarantee quality — We often have definitions of done that call for some high percentage (e.g 85%) of the code to be covered by unit tests. On the face of it that is an admirable goal.
The Pareto Principle easily applies to the effort of seeking 100% coverage. 80% of the code is covered by the first 20% of effort, and the remaining 20% requires 80% of the effort.
Even if we get to 100%, we’re still not guaranteed that we’ve tested all possible combinations of execution paths through the system.
Naively trying to game the coverage number by writing low value unit tests for simple accessors methods won’t guarantee that we have a stable system in production either.
The naive approach would be to configure our unit tests to use a shared database server in a test lab somewhere on our network. However, this approach has the following problems:
Interference — Care has to be taken to make sure build agents and test runs do not interfere with each other especially if they are operating on shared seed data or purging data at the end of a test case.
Accumulated state — Tests have to be written to ensure that they clean up after themselves. Otherwise, we will end up with data accumulating in the database wasting space, impacting performance, and potentially interfering with future test runs.
Loss of control — In many organisations, even simple schema changes will require approval from the DBAs and may also have to be executed by the DBA on their schedule.
Coordination overhead — Using a shared resource is going to require an additional coordination overhead when there are multiple teams involved.
Accessibility — It’s not always possible for team members to access shared resources when working remotely or travelling.
One approach to break the dependency on a shared database is to simulate
@JdbcTemplate using a framework such as Mockito, JMock or EasyMock. We will use Mockito because like, JUnit, it is included in Spring Boot’s curated bill of materials.
Mocking involves simulating the behaviour of a real object with a substitute, called a mock. Mocking frameworks allow us to define rules called expectations that define how the mock object should respond to method calls depending on the values of arguments. We can even allow the substitute to call the real method or provide an alternative implementation if some complex processing is required.
In the example below, the mock
JdbcTemplate intercepts all calls and allows the developer to provide alternate implementations or rely on the default do nothing behaviour for
JdbcTemplate methods. The example below is not using any Spring Boot Test capabilities and only relies on JUnit 5, AssertJ and Mockito.
The use of
MockitoExtensionwill cass all attributes annotated with
@Mockto be initialised with mock implementations.
JDBCTodoItemDAOwith a mocked
Create an expectation that will simulate the behaviour of invoking the
PreparedStatementSetterthat binds the update parameters to the
This all a bit tedious to write and painful to maintain on an active codebase. It doesn’t really suit mocking methods that invoke callbacks and becomes totally impractical if you are using persistence frameworks, like Spring Data or Hibernate ORM, that dynamically generate the SQL for you.
Use an embedded database
Instead of mocking the database we can use an embedded database. An embedded database is a lightweight database runs inside the application process rather than as a standalone server.
Spring Boot will automatically configure an embedded database if 1) we do not explicitly configure one ourselves, and 2) it finds the driver for H2, HSQLDB, or Apache Derby on the classpath. We will be using H2 for no other reason than pure habit.
The following snippet from the
pom.xml adds the H2 driver as a test dependency. We don’t need to specify the version number as that is provided by the Spring Boot bill of materials.
Spring Boot Test is able to manage the life cycle of this database. If Spring Boot finds a schema.sql file on the class path, then it will use that to create the database schema. In addition, if there is also a data.sql file on the classpath, it will be used to populate the database with seed data.
We’ve seen the
schema.sql earlier in article and below is the
data.sql is only a test resource since it only contains seed data for our tests.
Spring Boot Test is capable of initialising all or some of an application’s context and allows Spring beans to be autowired into the test cases.
@DataJbcTestannotation tells Spring Boot Test to initialise the embedded database schema using
schema.sql, load the seed data using
data.sql, and create JDBC related beans such as
Spring Boot Test will inject the
JdbcTemplatefrom the application for use in the test cases.
JdbcTemplateinitialised by Spring Boot Test.
By default, our test cases are considered transactional and will be rolled back upon completion. We can use the
@Commitannotation if we want the changes to be persisted. However, its bad practice to rely on test case execution order.
This approach overcomes the concerns that arise with mocking. A real
JdbcTemplate is being used, so we don’t have to worry about mocking the behaviour of methods that rely on callbacks. Its better suited to testing classes that use persistent frameworks that generate SQL statements dynamically.
We would end up having to maintain two copies of the database schema. One for the embedded database and one for the targeted production environment.
We cannot test classes that on proprietary features of the targeted production environment.
Integration testing is about testing some or all of the units of an overall system together. By this definition, it could be argued that the approach described earlier in “Use an embedded database” is a kind of integration test. However, I believe this is not really valid, because we usually wouldn’t use an embedded databases in finished products.
Instead, we are going to use a library called Testcontainers to manage the lifecycle of a MariaDB database. Testcontainers makes it easy to configure, launch and manage containers running any Docker image from JUnit tests. However, Testcontainers has applications specific modules for commonly used systems providing DSLs to make it even easier to configure and manage the applications running in the container. There is module for MariaDB.
DAO & database
First, we’ll integrate the DAO and MariaDB.
The easiest way to include Testcontainers in our project is to use its bill of materials to manage dependency versions as shown below:
Then we can add test dependencies for the individual modules we need for JUnit 5 and MariaDB support:
In the example below, we are going to use a container running MariaDB that we will configure using a DSL to set the database name and user credentials, create the database schema and populate it with test data:
We still use the
@DataJdbcTestannotation to initialise the Spring Data JDBC framework for testing.
We don’t want an embedded database to be configured by
@DataJdbcTest, so we override the default behaviour using
@Testcontainersannotation indicates that we want the Testcontainers to manage the lifecycle of container objects annotated with
@Containerin the test class.
@Containerannotation marks the container objects created by the test case. It is
staticso there will only be one instance of MariaDB created for all test cases.
We specify the image version rather than rely on the
Using the DSL methods to set the database name and user credentials.
The MariaDB test container has support initialising the database using a single script via
withInitScript()DSL method. However, we are using separate scripts for the schema and seed data. So we need to bind them to the directory that the MariaDB container image will scan on start-up. The files have to be renamed since they will be loaded in alphabetical order.
Testcontainers assigns random port numbers for exposed service ports. So we can’t rely on hard-coded values in
application.properties. So we have to create a dynamic property source.
We bind the
getJdbcUrl()DSL method to the
spring.datasource.urlapplication property. This allows auto-configuration for Spring Data JDBC correctly initialise the
The drawback with this approach is that the overhead of starting/stopping containers will increase the execution time of our integration tests.
However, that is more that compensated for by the piece of mind we gain by more closely simulating the production environments in which our application will be expected to run.
API, DAO & database
The demo application is a micro-service with a RESTful API. If we want to have more comprehensive integration testing of the API and DAO layers with the database, then we can use Spring Boot Test’s MockMvc support.
We want Spring Boot Test to run a servlet container with a randomly assigned port for accessing the RESTful API end-points.
@Transactionalannotation means that any database changes will be automatically rolled back after each test case.
@AutoConfigureMockMvcannotation will configure the
MockMvcobject to with the randomly assigned port for the servlet container.
MockMvcobject we will use to test the RESTful API end-points.
GETmethod to be performed injecting path variables and setting headers.
Logs the outgoing request and incoming response.
Verifies the status code.
Using JsonPath to verify the contents of the response body.
In this article we’ve examined approaches for unit and integration testing applications using JUnit 5, Testcontainers and Spring Boot Test.
Hopefully, we can agree that Testcontainers is powerful, yet easy to use, and should be preferred over relying on embedded databases.
If that’s not enough consider that Testcontainers has DSLs for other relational and non-relational databases, Docker Compose, Elasticsearch, GCloud, Kafka, Localstack, Mockserver, Nginx, Apache Pulsar, RabbitMQ, Solr, Toxiproxy, Hashicorp Vault, and web drivers.
If the container you need is not supported directly you can still deploy it as a generic container or even create your own DSL module to share with the world.
If you’d like to be a guest writer on IrishDev.com, the online home of the Irish Software Developers’ Network, check out our Guest Writers section
Brian Matthews is an architect/engineer specialising in the development of cloud native applications for the Java and Spring eco-system.
Living in Russia, Brian regularly visits his homeland Ireland to share his knowledge with the Dublin Java User Group members.
Visit: www.linkedin.com/in/bmatthews68/ / @btmatthews68
Our subscribers are also reading….
Related: Java News, Events, Jobs
Get Instant Irish Tech News Updates on our Social Channels….