Shift-Left Workload, leveraging AI for Test Creation
How Amazon Q can automate test creation for faster, higher quality delivery
Stuart Clark
Amazon Employee
Published May 8, 2024
The shift-left testing approach has become widely adopted as it helps catch defects earlier in the software development lifecycle when they are less costly to fix. However, one key disadvantage i found when I was building code everyday was the increased workload on our team and myself to write more comprehensive tests earlier on.
With shift-left, developers need to invest some significant time and effort into creating thorough unit tests and integration tests before even getting to higher-level testing activities. This front-loaded testing workload can however slow development velocity and impact productivity, especially for teams already stretched thin. Teams and developers must carefully weigh the short-term costs of this upfront investment against the long-term benefits of catching defects sooner and delivering higher quality software. But what if there was a way to ease this workload burden?
Recently I had to refactor some old code to include some testing as this would be part of an upcoming demo I was doing. (It is funny how we can look back at our old code with such judgement!) I thought this would be a good test for Amazon Q to generate test cases based on my code and to see if this could inspect my code and identify important code paths that should be tested, thus allowing me to adopt shift-left practices quickly and by automatically supplementing the tests.
This was my starting code, a Python script that uses the boto3 library to create a VPC, an internet gateway, a route table, a subnet, and associates the route table with the subnet in my AWS environment.
I asked Amazon Q was to build a suite of tests, including unit tests and integration tests based on my demo example code. If you are new to this, check out this great blog HERE on Getting started with Amazon Q Developer in Visual Studio Code.
I was really surprised how this came back in less than two seconds.
The tests are divided into two classes:
TestVPCCreation: This class contains unit tests for individual functions that create VPC resources, such as VPC, Internet Gateway, Route Table, and Subnet. Each test method mocks theboto3resource and verifies the correct method calls and return values.TestVPCIntegration: This class contains an integration test that creates and verifies the correct creation of all VPC resources in sequence.
We can now see new code example aligns perfectly with the concept of "shift left testing" for several reasons:
- Early Testing: It focuses on unit testing the individual functions responsible for VPC creation (
create_vpc,create_internet_gateway, etc.). This allows catching errors early in the development process, before they become part of a larger integrated system. - Isolation: By mocking the
boto3.resourcefunction, the unit tests isolate the functionality of the code and ensure it works as expected regardless of actual AWS interaction. This simplifies debugging and reduces reliance on complex AWS environments for testing. - Focus on Functionality: The tests verify if the functions call the appropriate boto3 methods with expected arguments. This ensures the code interacts with AWS services correctly without directly creating resources or requiring access keys.
- Repeatability and Maintainability: Unit tests allow for repeatable and automated testing, making it easier to identify regressions during future development. This improves the overall maintainability of the code.
The Future of Automated Test Creation As generative AI continues to advance, we will likely see it play an increasingly vital role in automating tedious and labor-intensive tasks like test creation. This will allow us as developers to focus more on strategic, creative work while AI handles the heavy lifting of building quality in from the start. The shift towards pervasive automation of all testing activities is sure to accelerate adoption of critical quality practices like shift-left. Solutions like Amazon Q and Bedrock are paving the way by making it easier than ever to create and maintain a comprehensive test suite spanning all levels.
Shift-left is a proven strategy, but its benefits are amplified when you have AI doing the heavy lifting of comprehensive test creation. Get started today by experimenting with this and let me know in the comments if GenAI is helping your shift-left.
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.