When should you stop testing?
In the fast-paced digital age, software development teams are expected to deliver innovative software within shorter delivery lifecycles. To meet these demands, the development teams have adopted agile, DevOps, and automation methodologies to speed up the process.
Continuous Testing (CT) is the process of executing automated tests as part of the software delivery lifecycle in order to obtain timely feedback from businesses in order to save on costs and time. However, an important question that bothers many testers is “When should one stop testing?”
It’s a question that is normally answered in advance as part of the exit criteria for a test plan. Let’s consider the real-life possibilities:
- When we run out of time
- When the testers and/or the test environment are all re-deployed for another test
- When the project budget runs out
- When we have reached an acceptable level of risk
- When all the defects have been found.
The test professional’s view would be d) – when an acceptable level of risk is reached. Too often, project and program demands mean that we are forced to compromise and end up being corralled into “accepting” running out of time, or the test environment being redeployed, or the project budget running out. However, the last option of having all the bugs found sounds impossible – right?
So, in the real world, what happens after we stop testing upon running out of time or having reached an acceptable level of risk? In these cases, testing doesn’t stop. Whether we have done this consciously, or not, “testing” continues but is carried out by the end-user community and any defects or bugs found at this stage are fed back into the IT help desk. But is that optimal? Are we asking the right question? Instead of “When should you stop testing?” shouldn’t we ask “Why should you stop testing?”
The real goal ought to be achieving an acceptable level of risk and testers can do so only if they adopt a policy of continuous testing. We conducted a global survey to outline the challenges and potential approaches to transforming test practices in the age of agile and DevOps. We surveyed 500 senior decision-makers in corporate IT functions, working for companies and public-sector organizations across eight different countries.
Out of the 500 surveyed respondents, 32 percent acknowledged having fully adopted continuous testing. Twenty-five percent of those surveyed stated that a vast majority of their teams practice continuous testing.
Enterprises are now embracing continuous testing by adopting test automation for various quality assurance (QA) tasks along with the move towards agile and DevOps.
While organizations today have adopted agile and DevOps methodologies, testing remains the missing key needed to unlock the complete transformation of the software delivery lifecycle. Our Continuous Testing Report (CTR) 2019 looks at some of the key trends, challenges, or gaps in the areas of testing and the remedial measures required that organizations can undertake to improve.
- Sogeti UKMake an enquiry
0330 588 8200
Sogeti UKMake an enquiry
0330 588 8200
- Phil LuptonSolutions Director, Delivery Director and Account Director, Sogeti UK
Phil LuptonSolutions Director, Delivery Director and Account Director, Sogeti UK
40 years in IT.
15 years specialising in QA and Testing Services.
Statistics degree background.
I have worked on both sides of the fence: Head of Testing for Sainsbury’s as a permanent employee and as a Senior Test Consultant in two large IT services companies delivering to a multitude of large clients across all business sectors.
Currently I am interested to see how the advances in AI and the availability of huge volume of data and information can be corralled to make the QA and Testing function sharper, and more efficient in today’s Digital world.