Situation
Our client is a multi-billion-dollar North American logistics group in the retail sector had significant challenges with the quality of their releases, resulting in numerous production issues and frequent hotfix releases. They sought to accelerate testing, enhance visibility into the application release state, and reduce the defect rate by at least 50%.
Our custom test automation framework supported various testing needs, including API, web, and mobile.
Upon analysis, we discovered that their testing practices were entirely manual, with each test cycle taking several days and lacking evidence of what was tested and its state.
Action
We implemented shift-left testing using behavior-driven development (BDD) practices, which reduced manual testing by over 70% and enabled testing through continuous integration and continuous delivery (CI/CD) pipelines.
A custom test automation framework was developed to support various testing needs, including API, web, and mobile testing. Features such as service virtualization and test data management facilitated early testing in the development phase and supported end-to-end testing. This core testing framework is used as an open-source tool within their environment, allowing the developer community to collaborate and enhance the framework for future needs.
We trained all scrum teams, product owners, developers, and Software Developers in Test (SDETs) on BDD practices and established governance for backlog clearance and in-sprint automation. The central framework team focused on extending features, while SDETs in scrum teams assisted with in-sprint automation and backlog clearance.
We created continuous test pipelines for development, test, and stage environments, running relevant tests frequently to ensure no major breaks in planned releases.
Results
- Established a culture of release testing through automation pipelines and in-sprint automation, significantly reducing release cycles and defects.
- Daily notifications gave updates on the state of the application release and any issues detected.
- Enhanced visibility into the application release state, allowing for more informed decision-making and quicker response times to issues.
- Increased test coverage significantly, ensuring more comprehensive testing and higher quality releases.
Automation pipelines and in-sprint automation reduced release cycles and defects.
Daily notifications provided updates on the state of the application release.
Issue response time was faster with visibility into the application release state.
-
50%
Automated coverage -
-
30%
Reduced deflection rate -
-
40%
Reduction in regression effort