Implementing Good Issue Management

The recording of testing issues is a crucial part of the testing process, not only because it allows the test team to demonstrate whether the system under test is up to the required quality, but also because it enables the developers to reproduce and solve failures.

An issue can have a variety of causes and is not always caused by a defect in the system. Issues can also be described as test incidents and are not only software defects. They may be classified as: change requests (when functions not fully fit for purpose are found), test script errors, user errors (in the case of UAT), third party issue (when the system interfaces to a third-party application) and defect – almost always the most common classification. Defects themselves can be due to code errors or environmental issues, and the investigating developer will determine this.

The Issue Management Strategy

A raised issue will have a status assigned to it throughout its life, and the status may be defined as describing the action currently pertaining to the issue, e.g., new, assigned, under investigation, fixed, rejected, etc. This is important as it shows how the issue is being handled at any moment in time. An issue should also have a detailed description, with reproducible steps to replicate it and a screenshot of how the issue was captured. Other technical details, such as build version, etc., may also be added if appropriate. This information facilitates the resolver’s investigation.

The Issue Triage Meeting

Issue management is governed by the regular issue meeting. In most cases, the test manager will be the owner of the issue management process and chair the issue meeting. In other cases, usually large programmes, there will be a dedicated issue manager who will fulfil this role. The issue meeting will take place at regular intervals during the test activity; this may be daily, weekly or at other intervals, depending on the intensity of test execution. The main stakeholders attending this meeting will be the test manager, issue manager (if applicable), project/release manager and one or more developers. In the case of UAT, a business stakeholder will also be present. In the meeting, the discussion will focus on the content of newly raised defects and also on their prioritisation for resolution. Once new issues have been addressed, the discussion will move to other open issues and updates will be added to these issues.

Prioritisation Levels

One of the most contentious aspects of the issue meeting is the assignment of priority and severity levels. Priority is simply the urgency with which an issue needs to be fixed. This will depend on the business impact and overall product risk – the higher the risk, the higher the priority. Technical severity is the impact of the defect on the functionality of the system and the test execution process. There may be issues which are high priority, due the business impact, but are low in technical severity. One example could be spelling mistakes on a corporate website, where the company’s reputational risk is at stake. Reporting on priority and severity is critical for assessing the risks to the quality of the product.

Priority levels should have a definition of resolution times. For example, a priority 1 issue would often have a 24-hour resolution expectation, a priority 2 a 72-hour resolution time, etc. The SLAs would be agreed in advance and depend on the criticality of the system under test and the capability of developers to resolve issues. These criteria would normally be defined in the Issue Management Strategy and help with setting expectations, thereby preventing needless arguments during the issue management meetings. Any breaches of the agreed SLAs would require explanation in the meetings.

Reporting

The test manager (or issue manager) is responsible for updating issues and producing issue reports, which would then be incorporated in the general test manager reports, which may be daily or weekly, depending on the project. Issue reports are essential for highlighting areas of risk for the quality of the software product and setting expectations for all key stakeholders. Given that issue reporting will be generated using utilities in the issue management tool, it is of paramount importance that the tool can produce dashboards, graphs and other visual aids that can give a straightforward view of issue status.

Conclusion

It can be seen that good issue management is of critical importance for the quality of the testing process in IT projects. Having a structured issue management process enables the resolution of testing issues to proceed much more smoothly and lessens risk for the outcome of the project. It is essential for all organisations delivering IT projects to ensure that they have an agreed issue management strategy in place.