Situation
Our Fortune 100 client encountered high infrastructure costs when running load tests on multiple applications, due to maintaining the machines with a JMeter image. To address this issue, a shift-left approach was required for performance testing.
Custom dashboards gave insights into performance metrics, helping to identify and address any issues promptly.
Additionally, our client needed a comprehensive dashboard for live monitoring and post-test analysis. Live monitoring offers real-time insights into performance metrics during the tests, allowing for immediate detection and resolution of any issues. Post-test analysis presents a thorough evaluation of the test results, helping to identify trends, bottlenecks, and areas for improvement.
Action
We implemented a serverless infrastructure for performance testing that dynamically adjusted based on the size of user load. This infrastructure built the necessary containers to handle the load efficiently. Performance testing was containerized, with each container executing tests along with the total number of virtual users per container.
We developed a custom Docker image with JMeter, utilized parameters to configure it appropriately. During test execution, data is sent to Elasticsearch in real-time, allowing for continuous monitoring and analysis. For horizontal scaling, containers are launched using AWS Elastic Container Service (ECS) and AWS Fargate with the number of containers specified in AWS Cloud Deploy.
To capture and store load test results, we launched an AWS-managed Elasticsearch instance within an application-specific account. Custom dashboards provided detailed performance testing reports and gave insights into various performance metrics, helping to identify and address any issues promptly.
The execution of load tests was initiated through AWS Cloud Deploy, streamlining the process and ensuring that tests could be run efficiently and effectively.
Results
- Reduced costs using shift-left approach that integrates performance testing earlier in the development lifecycle.
- Immediate detection and resolution of issues with real-time insights from live monitoring.
- Consistency using Terraform for automated deployment process.
Fast detection and resolution of issues with real-time insights from live monitoring.
Data sent to Elasticsearch in real-time, for continuous monitoring and analysis.
Shift-left approach reduced costs by integrating testing early in the dev lifecycle.
-
90%
Reduction in manual maintenance efforts -
-
67%
Reduction in infrastructure cost -
-
8x
Automated test run by bots per day