A case for centralised Test/ QA Management in an Agile & DevOps world

Reviewing the data on QA/ Testing budgets from the report surveys, we can see a startling decline in the budget spent on testing since 2015. The figures paint a picture of the trend in today’s QA world with the emergence and steady growth of Agile, DevOps and Continuous Testing. Let’s face it, testing has become more efficient and cost-driven.

Testing is fast becoming an integral part of software development, gradually losing the autonomy of the traditional waterfall model. In an Agile and DevOps world, developers often take up the role of QA/tester making it difficult to track the real budget for testing.

Tooling is another area where testing budgets are reducing at a steady rate. Organisations no longer spend the same amount of money on commercial automation tools and test management tools as they have done previously. Open source tools are gradually replacing commercial ones. In most of the projects I have worked on recently, I often recommend the use of open source tools, thereby reducing the budget spent on testing and saving time without sacrificing quality.

Another area of saving is the test environment set up. Emergence of cloud technology is making it easier and more cost efficient to provision test environment on demand. Leading cloud technology providers such as AWS, Google and AZURE have competitive pricing when providing test environments. It is possible to pay as you use, thereby reducing the cost of building and maintaining test environments.

Automation is a constant feature in most QA reports and according to the WQR, the data shows the new trends in automation are moving from the traditional test automation tools to more intelligent, sophisticated tools embedded in the pipeline. DevOps and Agile arel about removing barriers and releasing software to the market faster and more effectively; this can only be achieved by introducing continuous testing.

What is continuous testing?

Continuous testing is the process of executing automated tests as part of the delivery pipeline in order to obtain feedback on the business risks associated with a software release as fast as possible. The main objective of continuous testing is to extend test automation and address the increased complexity in today’s fast-paced software development and delivery. Continuous testing provides the right feedback to the development team in a time critical manner.

Software testing has traditionally been a slow, costly process that delays releases while delivering business value. Testing is still seen as the bottleneck with automation taking up time and resources. Test data, lack of required skill sets, test scripts maintenance, frequent changes in applications under test, interaction with multiple technologies such as microservices, API, browsers, legacy systems all make it very difficult for automated test tools to keep up. Most automated scripts often become redundant, causing another bottleneck in the software development cycle. The dynamic change in emerging technologies also makes it difficult for automated tools to be the solution to fast and effective testing delivery.

Case for a centralised QA function fully aligned with Agile and DevOps

As a technologist and experienced QA Manager who has had the privilege of working across multiple organisations, implementing multiple technologies (Waterfall, Scrum, Agile , DevOps and Continuous Testing ), I am going to focus on some of the fundamental reasons why I think centralised QA still makes sense, thinking about how it can help to bridge QA, Agile, DevOps and Continuous Testing processes.

In recent years, we have clearly seen the value of Agile in enhancing software development and delivery. From a QA perspective, it has not been as fulfilling as expected. Working faster sometimes means working with less documentation, less planning and subsequently less time to test and reduced test coverage. With my experience of managing a QA team in an Agile environment, I am aware that QA teams can sometimes feel that working in an Agile methodology is causing the earth to move under their feet. The decentralised structure of Agile makes QA difficult to control and implement appropriately.

A centralised QA function will work better in agile environments as there will be a repository of skills and knowledge that can be distributed to development teams. Knowledge can be shared and there will be continuity of skills and knowledge in the QA pool.

For a centralised QA/Test functions to work effectively, I believe the following methods should be adopted:

1. A centralised QA / Testing and automation strategy should be developed and aligned with organisational QA goals and sponsored by C-level executives.
2. The centralised team should work according to a scaled agile framework. Each development unit should have a dedicated QA team with QA lead working closely with the development manager. The focus of the team should be the end to end testing.
3. A QA engineer should be assigned to each Scrum team, attending Scrum meetings, attend stand up meetings and report to the QA lead.
4. All QA engineers from the centralised pool should be empowered and trained. They will not only be testing features, but they must be able to work with product owners to become the authority or ‘go to’ people on the products. The QA lead should also have the authority to decide when a feature can be released from a quality perspective.
5. All QA engineers must also have coding skills. QA engineers must write, develop or run automation test scripts; they must start writing test scripts much earlier with UI-less user stories that require testing using Application Programming Interfaces.
6. Automation is the backbone of continuous testing and hence a robust automation framework should be put in place. In each Scrum team, there should be a QA engineer assigned as the automation enabler. The enabler will maintain the framework, decide on how frequently to run the regression packs and when to shorten the regression test. The automation enabler also makes the writing and running of automation easily repeatable for the rest of the QA team. This is the basis of development in test.
7. Everyone in the centralised QA team should eventually become a ‘Developer in Test’.
8. It is very easy to side-line the QA function in an Agile team as the development manager’s priorities often differ from those of a QA manager. Although quality is important, it is not always the most important thing. When quality issues arise in a version, and defect fixing requires more time than planned, quality may be traded off for a faster release. The feature may be moved to a later sprint. The QA manager's priority is the inverse of the development managers. QA managers prioritise an end-to-end, user-like quality view and see useful features as just one part of the overall user experience and quality story.

To make continuous testing work in a DevOps and Agile world, it is important that QA managers maintain their autonomy. QA teams must have the freedom to perform their task of product quality assurance without considering the development manager’s view. They need to set priority for defects from a QA perspective and act as the quality gateway for all releases. This can best be achieved by having a centralised QA function.

 

Author