One-stop guide: Kotlin + Spring Boot 2 + AWS S3 + Testcontainers

If you are wondering how to mock AWS infrastructure for the integration tests in your Kotlin and Spring Boot based application or you are interested to taste Kotlin as a primary language for the Spring framework, this article is for you.

Let’s implement the small REST API to Upload, Download, List, and Delete the S3 Objects, and cover with an integration test. The main components are listed below:

Kotlin 1.4, JDK 11, Gradle 6, Spring Boot 2.4, and AWS SDK for Java to interact with AWS S3, the Simple Storage Service.

To test it locally without connection to the AWS S3 we are going to use Testcontainers and Localstack. To document the API we have Swagger 2.

To create the new project based on Spring and Kotlin you either have to start a new project in IntelliJ IDEA through the wizard or using the constructor at https://start.spring.io/ choosing the next options:

If it builds with ./gradlew build then we ready to move forward.

Before we start I would like to mention a few things:

• I cut off some info e.g package name, imports, etc. to follow the KISS
• The feature-based package structure cause I think it’s easier for the enter
• Clone the repository to read and explore the code at the same time

WEB APPLICATION

S3Service interface contains an API contract for the business logic: download, delete, list the files, and bulk upload operations. For this stage, it doesn’t really matter what kind of data we plan to return in each of these methods, you likely will change it later.

If you never worked with S3 it’s important to understand what does mean bucket and object in terms of AWS S3. In this project, we will call Object a file to keep it simple for understanding.

S3Controller provides the endpoints of the business code. We use the S3Service interface to inject service through the constructor into the controller.

Infinite discussions around do we need to inject abstractions instead of concrete classes until we don’t have multiple implementations, but in real life, we are never ready for the time when we need to deal with the next API version of our vendor, do we?

S3ServiceImpl - here we need to write our logic using a contract of the S3Service interface. We will agree that we work with unversioned objects.

You could notice that Kotlin comes with a lot of cool features such as Scope functions, Elvis operator, Named arguments. For example, where javaClass comes from? In Kotlin we have the Extension Properties so javaClass property just returns the runtime Java class of the object. All these gears make life easier.

S3BulkResponseEntity. In case you were wondering what is the List<S3BulkResponseEntity>which we return in the uploadFiles()method. As we deal with multiple files within one query, we want to provide a detailed response about each of the uploaded files.

Thus we need to create a data class. Even though Lombok won’t help here, Kotlin takes care of all boilerplates the same way as Lombok does, so we don’t need to worry about getters and setter for mutable variables, hashCode() and equals() contract, and many more.

The final step of the main part is to configure S3Client which is basically SDK for AWS. If you look at the S3ServiceImpl you can see that we inject the S3Client there.

REST API DOCUMENTATION

To provide a convenient way to test our REST API we use Swagger which explores our controllers and generates WEB UI without any front-end programming.

If everything is right and the SwaggerConf class placed to the classpath within the scope of Spring scanning it will find the component and configure the Swagger. The interface will be here: http://localhost:8080/swagger-ui/index.html

TESTS

At the very beginning, we planned to write an integration test. First of all, we need it because the main function of our application is to interact with AWS S3. The plan to invoke tests using the AWS environment for every build doesn’t sound good. On the other hand, to implement the mock of S3 REST API sounds pretty complex. The Localstack library here comes to play.

Anyway, we will start with the interface S3TestService that declares auxiliary methods. We don’t want to mess up at the main application, right?

S3TestServiceImpl is the next step and where we write the concrete implementation. To inherit the main code and keep the contract we extend S3ServiceImpl and implement our new S3TestService interface.

s3cleanup() runs through the existing buckets, removes files inside, and buckets themselves. Our container is a singleton so the cleanup will have a sense as soon as we add the second test.

LocalStackContainerConf - the configuration that defines two beans: one for the container with the LocalStack instance and the second one for the list of services that LocalStack will need to be initialized with.

As we already have the preconfigured container it is time to create the S3Client for the test environment, it will interact with the Localstack instance. Let’s call this bean S3TestClient. Besides, we need to configure test S3Service to inject it into the S3Controller later.

Both of the beans we will place into the new S3TestConfig. We need to mark our beans @Primary to give them more priority comparing to the main ones in terms of autowiring.

Finally, we have all the required components to create the integration test. S3IntegrationTest along with @SpringBootTest have a lot of things to do: it starts the web-server on a random port, prepares and runs the Docker container provided by Testcontainers, injects the S3TestService to the main S3Controller, and eventually sends a request to upload multiple files to the Localstack’s S3 mock-instance with the checking of the file-list size after that.

This is the end. Thank you for the reading and hopefully, you got a good time!

SOURCE CODE & DOCS

For the source code welcome to https://github.com/antukhov/s3warehouse

CONCLUSION

⛔️ Please keep in mind, this is a demo implementation without the coverage of many edge-cases, don’t integrate it “as is” into production code.

FYI Swift has a lot of common things with Kotlin so I would recommend taking a look at it too if you are a fan of cutting-edge trends in languages.

Great if after the reading you learned something new and I appreciate any feedback whatever it is 😉