Testing DynamoDB Interactions using LocalStack and Testcontainers

Overview

Testing the data layer and datasource interactions is very crucial when developing a backend application. In this article, we’ll explore on how we can write Integration Tests for our Java Spring-boot application which interacts with AWS DynamoDB for storing and retrieving records.

We'll be creating an example data model and a repository class which performs basic CRUD operations on the provisioned DynamoDB Table. Then we'll be looking at how we can write Integration tests for our repository using LocalStack and Testcontainers where we'll be creating an In-memory DynamoDB Table inside a disposable docker container and making our application connect to it while running the tests.

With this setup, we'll be ensuring a controlled and isolated environment for testing our DynamoDB interactions. The code referenced in the article can be found in this Repository.

Contextualizing Keywords

Spring Boot : is a popular framework for building Java-based, production-ready services with minimal configuration.
AWS DynamoDB : is a fully managed NoSQL database service provided by AWS and we'll be using it as the datasource that our backend application connects to.
LocalStack : is a cloud service emulator that enables development and testing of AWS services without connecting to a remote cloud provider. We'll be provisioning an in-memory DynamoDB table inside this emulator.
Testcontainers : is a library that provides lightweight, throwaway instances of Docker containers for integration testing.

AWS Configuration

In order to facilitate connection between the application and DynamoDB table, we'll need to define beans of AmazonDynamoDB and DynamoDBMapper in a configuration class.

When connecting to an actual provisioned DynamoDB table in AWS, we don't need to specify an endpoint URL since it automatically uses the default endpoint for each service in the configured AWS Region. But when connecting to the Table created inside the LocalStack container, we'll be needing to define an endpoint URL as well.

To solve the above issue, we'll be using Spring Profiles to define seperate beans of class AmazonDynamoDB, that'll get injected into the application context depending on whether the application is running in test or non-test profiles.

Additionaly we'll make use of @ConfigurationProperties to map the values defined in the active .yaml or .properties file to a POJO that'll be referenced by our configuration class.

As mentioned above, The endpoint field is optional and is needed to be defined only for testing and local development when using LocalStack.

Below is a snippet of our application.yaml file where we've defined the required properties which will be automatically mapped to the above defined class, along with the current active profile that we'll be using in our bean configuration class.

Now let's create our AwsConfiguration class that defines the beans required for connecting to DynamoDB depending upon the active profile.

The bean specifying the endpoint URL for connecting to the LocalStack container will be created only if the active profile is test or local. With this, we've successfully created the correct AWS DynamoDB configuration for our application.

Data Model and Repository Layer

We'll be creating an entity class MedicalRecord to represent the data stored in DynamoDB with the below defined attributes. We're leveraging lombok for generating getter and setter methods for the attribute to reduce boilerplate code.

To perform basic save and retrieve operations on the above defined entity, we'll be creating it's corresponding Respository class. The earlier defined bean of DynamoDBMapper will be autowired into this class via Constructor Injection with Lombok.

Testing

Now that the context is established, we'll be moving on to the main agenda of this article: To test the data layer's interaction with DynamoDB.

Dependencies

Let’s start by declaring the required test dependencies in pom.xml:

Prerequisite: Running Docker

The prerequisite for running a LocalStack container via Testcontainers is, as you've guessed it, an up and running Docker instance. We need to ensure this, when running the Test suite either locally or when using a CI/CD pipeline.

When using Github actions, The default Ubuntu runners come pre-equipped with Docker, eliminating the necessity for manual Docker setup procedures.

Creating DynamoDB Table using Init Hooks

Localstack gives us the ability to create resources when the container is started via Initialization Hooks. We'll be creating our init-dynamodb.sh script in the src/test/resources folder which we want to be executed when the LocalStack container is started.

The script creates a DynamoDB table MedicalRecords for which we've already created a model class in our application.

Now in our Integration Test class, we need to do the following before we start to write any test cases:

Activate the test profile using @ActiveProfiles
Copy our init-dynamodb.sh script into the container to ensure DynamoDB table creation.
Configure a Wait strategy to wait for the log Executed init-dynamodb.sh to be printed, as defined in our init script.
Dynamically configure the AWS configuration properties needed by the application in order to create the required connection beans using @DynamicPropertySource.

Now once we're done configuring the Localstack container, we can test the MedicalRecordRepository class with appropriate test cases:

And with the above, we've successfully tested our repository class that interacts with DynamoDB table for storing and retrieving records.

The created LocalStack containers will be destroyed automatically once the Test Suite is completed, hence we do not need to worry about cleanup.

Pro-tip: Reduce Boilerplate Testcontainers Code

In our Integration Testing setup, we encounter repetitive boilerplate code for initializing and configuring the LocalStack container, as well as dynamically adding properties to it. These steps will be repeated in every Test class that tests a functionality related with the data layer.

This duplication clutters our Test classes and we'll be looking at a possible solution for this issue using the lifecycle callback extension available in JUnit 5 called @BeforeEachCallback.

Inside our test folder itself, we'll be creating a class DynamoDBInitializer which implements the BeforeEachCallback interface and performs the boilerplate functionality.

Then, we'll be creating a custom Marker Annotation @InitializeDynamoDB that registers the above created Extension class using @ExtendWith.

Now, we can annotate all of our Integration Test classes which require DynamoDB Table creation to test functionalities with our custom annotation @InitializeDynamoDB.

NOTE: To have a single shared container that is used by all Integration Test classes, we've implemented the BeforeAllCallback interface. This approach will reduce test execution time significantly and eliminate the need to start and stop containers for each test class.
For cases where this behaviour is not needed, BeforeEachCallback interface can be implemented instead.

Conclusion

Through this article, we've explored on how we can create an isolated testing environment with LocalStack and Testcontainers to write Integration Tests for our data layer, specifically for one that interacts with AWS DynamoDB Database.

Using what we've learnt, we can confidently write integration tests without relying on external resources or impacting production data.

Site Terms, Privacy, and more.