Unravelling the Risk-Based Ruse

I recently watched an interview between an investigative reporter and a local government official who was representing a Local Authority food standards agency, the exchange went as follows:

Reporter: Following a public information request; it states that, this Local Authority hasn’t carried out a single food-standards check in any of its food outlets in the last year.
Government official: That’s right: we operate a risk-based approach.
Reporter: How can you justify not carrying out any tests in the last year and the potential risk to public health.
Government Official: We judge that risk to be very low.

Is this acceptable? Would the end-users, the public in this case, understand the potential implications of taking a risk-based approach?

How can we tell how good the judgement is that: no tests were necessary?

Do we know what the constraints are? Can we calculate that this is a “low-risk” independently of any prevailing constraints, or has the “low-risk” assessment been driven by other factors such as “not enough, or any, budget” or “not enough time” or “not enough band-width”.

So: how can we provide a more transparent judgement to allow the affected stakeholders to have a clearer line-of-sight and, crucially, the opportunity to flex one or more of the prevailing constraints if they believe that the resultant risk-based decision is, for want of any other term, “too risky”.

Therefore I think we have a duty to add the constraining caveats that underpin the covering statement “we are using a risk-based approach” whenever we use that phrase to better describe and throw-light on the quality assurance approach being taken on the customers’ behalf and to allow our stakeholders an informed view and the opportunity to challenge and potentially adjust the constraining parameters.

How good are we at assessing risk? Remembering that the end-user or the business should have the correct perspective but how does the end-user community perceive the word “risk”? Their judgement is likely to be coloured by their role. The riskiest areas highlighted will be those that support their core roles without being able to consider objectively the competing risks for other roles.

Assuming we can arbitrate and make that assessment then we can assign our resources intelligently to the areas of most risk.

Classically we can test the functions that are most likely to go wrong, and those with the biggest impact of failure if they do go wrong first. This is a well-known technique and a well-trodden path for test strategies and test plans.

Having established a risk-based approach with all of the constraining factors transparently articulated and agreed, we still face two more hurdles:

1. How well can we implement the approach?
2. How should we react to changes of plan so that the underpinning approach is not compromised?

Let’s look at what typically happens in practice for any IT project delivery of a reasonable size and complexity.

There is usually a high-correlation between the most critical functions and the most complex ones, leading to the most complex functions being delivered last, and often “late”, which confounds the neatly arranged risk-based plan where the complex ones have the priority.

As the QA/Test team is in place and keen to be making progress, typically effort then gets spent on testing anything that is ready for test, usually the least complex or the untouched elements. The time for testing the functions labelled “high-risk” gets squeezed and delayed nullifying the correct application of the carefully planned risk-based approach.

It is rare to see a project team, mid-flight, re-assess the risk profile once the test execution train is rolling. Expending effort and making progress concentrate the management attention and become the metrics of choice, instead of measures of residual risk which was the underpinning dynamic of the original planning. In short: the planned risk-based approach is compromised by stealth.

I propose a checklist that needs to work alongside any project that proclaims to be adopting a risk-based approach, and as a sidebar, is there any other approach in the real-world?

At the planning stage:
Do we have the time, effort and cost constraints documented?
Do we understand the risk assessment approach?
Do we agree with the risk assessment? {Have all stakeholders had their input?}
Can we accept the planned level of risk that will remain at project closure? {and if not we should re-visit the constraints}

At the execution stage:
Can we keep track of the level of residual risk?
Can we make plan flexibly so that conscious decisions can be made to vary or not vary the constraints based on the projected level of residual risk at project close?

If we follow this risk-based approach checklist, I believe we will be better informed on whether or not we wish to try one of those food outlets!

...

Now, where did I put that take-away menu...