Mastering Performance and Chaos Engineering for Modern Cloud Applications on AWS
Deloitte's approach to Load Testing and Chaos Engineering in AWS offers organizations the opportunity to leverage its inherent advantages, including on-demand provisioning and efficient resource management.
Hemant Ahire
Amazon Employee
Published Apr 1, 2025
Mastering Performance and Chaos Engineering for Modern Cloud Applications on AWS
Load Testing and Chaos Engineering help ensure reliability and resilience of modern applications. For example, an e-commerce site uses Load Testing to handle traffic surges and Chaos Engineering to disrupt payment processing during peak loads, preventing financial losses and facilitating a smooth shopping experience. Together, they build robust, dependable systems.
Many organizations now use cloud-based systems like AWS for Load Testing, unlike traditional on-premises setups. Cloud Load Testing leverages auto-scaling for on-demand resource provisioning, simulating realistic workloads. It also assesses performance across regions or availability zones, helping to ensure a consistent user experience for global audiences. In this blog, you will learn about Deloitte’s approach to implementing Load Testing & Observability along with Chaos Engineering in an AWS Cloud environment.
Deloitte's Approach to Load Testing in AWS
Figure 1 – Load Testing using AWS & JMeter for application under test.
Traditional on-premises Load Testing faces infrastructure and cost challenges. Deloitte overcomes these by shifting Load Testing to the cloud. Deloitte uses JMeter to create load test scripts for front-end, backend, and microservices on AWS containers/Amazon EC2 instances. This approach replicates diverse user demands, leveraging AWS's scalability to simulate realistic scenarios globally. It addresses scalability concerns and provides insights into application resilience, ensuring robust and adaptable Load Testing. Apart from JMeter, tools like LoadRunner, Gatling, Locust, etc. can also be leveraged to perform Load Testing on AWS.
Deloitte's Approach to Chaos Engineering in AWS
Chaos Engineering implementation using AWS Fault Injection Simulator (AWS FIS) is based on the basic building blocks which may serve as a blueprint across organizations: -
Figure 2 - Chaos Engineering using AWS Fault Injection Simulator (AWS FIS).
Understanding client requirements is the initial step, involving a deep dive into the customer’s business objectives, assessing the existing AWS infrastructure, and identifying specific pain points and concerns related to system reliability and resilience. This is followed by a thorough assessment and analysis of the client's AWS environment to identify weaknesses, vulnerabilities, or failure points. Collaborating with the client, a chaos engineering strategy is crafted, determining suitable AWS resource targets, establishing success criteria, and selecting appropriate chaos experiment scenarios. AWS FIS experiment templates are then developed, specifying target AWS services and regions, including parameters for introducing chaos, such as delays, failures, or resource termination. The chaos experiments are executed using the AWS Fault Injection Simulator (FIS) service, and the results are analyzed to pinpoint shortcomings, vulnerabilities, and areas needing development.
Remediation and recommendations are offered based on the findings to enhance system resilience and reliability. Identified vulnerabilities are addressed iteratively through cross-team collaboration, continuously conducting trials, observing system behavior, and making necessary modifications to the client's AWS infrastructure and applications for ongoing improvement and resilience.
Deloitte utilized AWS Fault Injection Simulator (AWS FIS) to conduct controlled fault injection experiments across various AWS services, enabling teams to build confidence in their application's behavior under different failure scenarios. This uncovered critical vulnerabilities, ensuring a low MTTR (mean time to recover) before go-live. Additionally, it reduced the time required for application migration and modernization by three weeks due to automated templates for AWS Fault Injection Service.
Potential Business Benefits of adopting Deloitte’s Quality Engineering methodology with AWS:
By following the above architecture and approach by Deloitte and leveraging AWS services like Amazon EC2, Amazon RDS, Amazon S3, AWS Lambda, Amazon ECS, AWS Fault Injection Simulator (AWS FIS) for Load Testing and Chaos Engineering, organizations can help ensure the application's performance and reliability, allowing it to operate efficiently. Here are detailed potential benefits:
Proactive Failure Identification and Mitigation: Address weaknesses before they cause real-world issues.
Reduced Downtime and Consistent Performance: Minimize downtime and maintain consistent service levels.
Superior User Satisfaction: Deliver faster, more responsive, and reliable applications.
Avoidance of Outage Costs: Prevent downtime-related revenue losses.
Optimized Resource Utilization: Efficiently allocate resources to reduce infrastructure costs.
Faster Recovery Times: Reduce Mean Time to Recovery (MTTR) through better preparedness.
Enhanced Monitoring and Alerting: Improve issue detection and resolution.
Efficient and Cost-Effective Scaling: Scale applications efficiently to meet demand.
Improved Predictability: Aid in accurate capacity planning and risk management.
Differentiation Through Reliability and Speed: Gain a competitive edge with reliable and fast applications.
Sustainable Business Growth: Support growth with efficient resource use and risk management.
Conclusion
Deloitte's approach to Load Testing and Chaos Engineering in AWS offers organizations the opportunity to leverage its inherent advantages, including on-demand provisioning and efficient resource management. This approach contributes to improving resiliency and minimizing downtime.
This flexible methodology can be readily adopted by organizations, either as it is or with slight customization tailored to their specific requirements. Interested parties are encouraged to reach out to Deloitte to gain further insights into its consulting experience in Load Testing and Chaos Engineering.
For more information on how Deloitte can assist with Load Testing and Chaos Engineering, please contact us to explore the possibilities of enhancing your cloud infrastructure and performance.
Authors:
Pradeep Mirchandani – senior manager, Deloitte Consulting LLP
Yasoda Kumar Balireddy – senior consultant, Deloitte Consulting LLP
Indrajit Bose – consultant, Deloitte Consulting LLP
Hemant Ahire – Principal Solutions Architect, WW Migrations and Modernization, AWS
Vijay Rangaswamy - WW Migration and Modernization Partner GTM, AWS
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.