Let's talk

Quarkus, Testcontainers, and Storages

Published: 11.12.2020

Being involved in a project where Quarkus is intensively used, we’ve started to work on some integration
tests and system tests where I’ve involved the persistent storages as well.

We’ve said persistent storages, yes, we have two: Microsoft SQL Server and Amazon S3. I will try to show next how both of
them can be integrated for real into the testing part by using Testcontainers,
without creating mock objects, like usually.

I found this article useful because the integration of Quarkus and Testcontainers is a different story(see
this and this),
compared with the integration between Spring Boot and Testcontainers. I’m not trying to say that Spring Boot has better
integration with Testcontainers, but if you are coming from the Spring Boot world, then Quarkus has some minor(major)
differences in certain aspects of how it is working. I’ve worked for a few days until I’ve finally finished integrating
them into the project (thanks mainly to Sergei Egorov) and I want to share the knowledge that I have with others who are
trying to achieve this goal.

Quarkus project setup

First of all, let’s have a look at how my Quarkus project looks like. The project is available on my Github profile and
what is important at this moment is to see what dependencies we need, by having a look at the
pom.xml file.

We will see that along with Quarkus dependencies there are used some Testcontainers specific dependencies:

  • org.testcontainers:junit-jupiter
  • org.testcontainers:testcontainers
  • org.testcontainers:mssqlserver

Also, it’s worth noticing that the software.amazon.awssdk:s3 it’s present in the pom.xml file.

Repository

In the project there is a simple abstraction called
FruitRepository
which is basically a simple Java interface that exposes two methods:

  • fruitById — which is designed for retrieving a Fruit object from the persistent store by using a specified id
  • saveNew — which purpose is to save a new Fruit object into the persistent store by providing the object as a method parameter

This interface has two implementations, one for each persistent store used by the application. Basically, the two implementations are:

  • JpaFruitRepository – which is the implementation used when the application stores data into Microsoft SQL Server. For Java developers, it’s obvious that this implementation uses JPA2.x and Hibernate.
  • S3FruitRepository – which is the implementation used when the Amazon S3 is used as a persistent storage mechanism.

We will not dig more into details about the functional aspects of the application as it is not of our interest.
We will let you discover them by having a look at the project.

Testing the S3FruitRepository

We will have a look at how the setup for testing the S3FruitRepository implementation is made by using Testcontainers and
running an S3 provider (which in our case is MinIO) as a Docker container.

The test class that tests the integration between the application and the S3 provider is the following one:

@QuarkusTest
@Testcontainers
public class FruitsS3IntegrationTest {

    @Inject
    public ObjectMapper objectMapper;

    @Inject
    public S3RequestProvider s3RequestProvider;

    private static final String ACCESS_KEY = "AKIAIOSFODNN7EXAMPLE";
    private static final String SECRET_KEY = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY";
    private static final int port = 9000;

    private static S3Config s3Config;
    private static S3Client s3Client;
    private static S3FruitRepository s3FruitRepository;

    @Container
    public static GenericContainer<?> s3 = new GenericContainer<>(DockerImageName.parse("minio/minio"))
            .withExposedPorts(port)
            .withEnv("MINIO_ACCESS_KEY", ACCESS_KEY)
            .withEnv("MINIO_SECRET_KEY", SECRET_KEY)
            .withCommand("server /data");

    @BeforeAll
    static void setUp() {
        s3.start();
        s3Config = new S3Config(ACCESS_KEY, SECRET_KEY, endpointUrl(s3), "integration-test-region");
        s3Client = s3Config.s3Client();
        s3Client.createBucket(CreateBucketRequest.builder().bucket("test-bucket").build());
    }

    @BeforeEach
    public void oneByOneSetup() {
        s3FruitRepository = new S3FruitRepository(s3Client, objectMapper, s3RequestProvider);
    }

    @AfterAll
    public static void tearDown() {
        s3.stop();
    }

    @Test
    public void test_uploadObject() {
        Fruit fruit = Fruit.apple();
        s3FruitRepository.saveNew(fruit);

        Optional<Fruit> retrievedFruit = s3FruitRepository.fruitById(fruit.id);
        assertTrue(retrievedFruit.isPresent());
    }

    private static String endpointUrl(GenericContainer<?> s3) {
        return "http://" + s3.getHost() + ":" + s3.getMappedPort(port);
    }
}

We should be aware of the following aspects:

  • the test class uses two annotations: one for using all the functionalities provided by Quarkus (eg: CDI) and another one for using
    the functionalities provided by Testcontainers;
  • the container definition is done by using the @Container annotation like it is stated in the
    Testcontainers documentation.
    It is important to note the fact that we are using JUnit5, because the Testcontainers setup is different for this JUnit version;
  • regarding the S3 container, we are using the official MinIO Docker image,
    which is available on DockerHub. The important aspect here is the fact that we are specifying the desired environment variables and
    configuration for the image to be created by using the exposed GenericContainer Java API;
  • even if in the Testcontainers documentation it is stated that the container will be automatically started when running a JUnit
    test method, if we will use the Testcontainers in combination with QuarkusTest then the container will not be started by the
    framework (more details in this Quarkus issue), so we must manually handle
    this as it can be seen in the methods annotated with @BeforeAll and @AfterAll.

Testing the JPAFruitRepository

In the case of the JPAFruitRepository implementation, we will have a different approach for testing this class by using a
Microsoft SQL Server running as a Docker container.

The test class looks like below:

@QuarkusTest
public class FruitsJpaIntegrationTest {

    @Inject
    JpaFruitRepository jpaFruitRepository;

    @Test
    public void test_jpaRepository() {
        Fruit fruit = Fruit.apple();
        jpaFruitRepository.saveNew(fruit);

        Optional<Fruit> retrievedFruit = jpaFruitRepository.fruitById(fruit.id);

        assertTrue(retrievedFruit.isPresent());
    }
}

But… where are all the Testcontainers configurations?! Well, in this case, we are using
an “automagically”-working feature provided by Testcontainers:
JDBC Support.
This feature is added by using the Testcontainers mssqlserver module.

So, first of all, we should have this dependency added to our project, and we have it because it was added at the very beginning.
Now, let’s have a look at the application.properties
file used by Quarkus in order to get the necessary values for configuring different parts of our application. The file is below:

# datasource configuration
quarkus.datasource.db-kind=mssql
quarkus.datasource.username=SA
quarkus.datasource.password=sqlserver2019!
quarkus.datasource.jdbc.url=jdbc:sqlserver://localhost:1433;databaseName=ACME;applicationName=CertProxy;encrypt=false;
quarkus.datasource.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver

# s3 configuration
s3.accessKeyId=AKIAIOSFODNN7EXAMPLE
s3.secretAccessKey=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
s3.endpointUrl=http://localhost:9000
s3.regionName=integration-test-region
s3.bucket=test-bucket

# test datasource configuration
%test.quarkus.datasource.db-kind=mssql
%test.quarkus.datasource.username=SA
%test.quarkus.datasource.password=sqlserver2019!
%test.quarkus.datasource.jdbc.url=jdbc:tc:sqlserver:2017-latest://localhost:1433;databaseName=ACME
%test.quarkus.hibernate-orm.database.generation=drop-and-create

We can observe that in the file there are three main sections, the first of them being the one where the configuration values
for the datasource are provided and the last is responsible for configuring the datasource for running the tests.

In the last configuration section we must observe the way by which the JDBC URL is configured: jdbc:tc:sqlserver:2017 latest://localhost:1433;databaseName=ACME. The strange part here is the “tc” inserted between the protocol part and the database type, but this is not so strange when thinking of the Testcontainers framework. In fact, this type of configuration is part of the Testcontainers JDBC support.

We have all these configurations in place and according to Testcontainers documentation, we will be able to run the FruitsJPAIntegrationTest. Well… we can try running the test by using mvn test -Dtest=FruitsJpaIntegrationTest, but we will face the following error:

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.096 s <<< FAILURE! - in org.acme.resteasy.FruitsJpaIntegrationTest
[ERROR] test_jpaRepository  Time elapsed: 0.023 s  <<< ERROR!
java.lang.RuntimeException: java.lang.RuntimeException: Failed to start quarkus
Caused by: java.lang.RuntimeException: Failed to start quarkus
Caused by: javax.persistence.PersistenceException: [PersistenceUnit: <default>] Unable to build Hibernate SessionFactory
Caused by: org.hibernate.exception.GenericJDBCException: Unable to open JDBC Connection for DDL execution
Caused by: java.sql.SQLException: Driver does not support the provided URL: jdbc:tc:sqlserver:2017-latest://localhost:1433;databaseName=ACME

But … why? Well… Testcontainers uses a specific implementation of the JDBC Driver: org.testcontainers.jdbc.ContainerDatabaseDriver. If we will have a look at the provided test configuration values for the datasource inside the config file we will see that the value for the JDBC Driver is no more present. This is because according to Testcontainers documentation:

"As long as you have Testcontainers and the appropriate JDBC driver on your classpath, you can simply modify regular JDBC connection URLs to get a fresh containerized instance of the database each time your application starts up."

This is true, but not in the case of Quarkus. Quarkus does some optimizations and sets the JDBC Driver value at build-time rather than at runtime, as usual, or like in the case of Spring Boot. More details can be found on this Quarkus issue.

Because of this, the workaround of the issue is to set JDBC Driver with the desired value by using environment variables, as specified by Quarkus [documentation](), and let Quarkus integration tests to choose this at runtime. The environment variable along with the value that must be set is the following one: _TEST_QUARKUS_DATASOURCE_JDBC_DRIVER=org.testcontainers.jdbc.ContainerDatabaseDriver.

NOTE: In the case of the Microsoft SQL Server it must be added a file called: “container-license-acceptance.txt” in the /test/resources. Add the following line to it: “mcr.microsoft.com/mssql/server:2017-latest”. Find more details here.

Now, the setup is complete and we can run the test again by using maven. We will see that this time the test will run with success.

Conclusions

  • We can instantiate containers based on Docker images by using Testcontainers GenericContainer API;
  • Remember that when using Quarkus, specifically @QuarkusTest, the Docker container must be run and stopped explicitly from code;
  • We can leverage the Testcontainers JDBC Support and test the integration between our application and persistent store;
  • Remember the setup which must be done for having the Testcontainers JDBC Support running when using Quarkus.

NOTE: If you want to read more about what we’ve presented in this article, there is a Slack thread between us and
Sergei Egorov which helped a lot in figuring out how Testcontainers and Quarkus can be used together.

Java Microservices Software Development Software Engineering Testing

Other news:

View all Articles