The Importance of a Requirements Traceability Matrix

Tracking the requirements for each sprint, release and project can be difficult especially in fast paced development environments. Depending on the organization, application and adopted methodology, each release can have a wide range of requirements. Some projects may only have a handful of requirements, while other projects could have requirements numbering in the hundreds.

Several different types of requirements need to be considered for software development. These can include: business requirements, user requirements, UI requirements, functional requirements, non-functional requirements, technical requirements and more. For example, user requirements cover the functions or features related to using the application – e.g. a screen shows the correct information, and that information can then be sent to the printer. An example of a technical requirement could be enabling cross compatibility on multiple systems.

Requirements should be both objective and testable. Some requirements will only need one simple test, other requirements will need several tests to ensure they are functioning correctly. The real challenge involves planning what tests to run on each requirement, how many, and tracking the status of these tests while maintaining a fast-moving release schedule.

Why is tracking requirements important?

News reports are littered with examples of companies that roll out new code only to find out that they have introduced a catastrophic glitch –costing them millions. These include major companies such as Amazon, Microsoft and Google. One such example is Knight Capital Group. After making some changes to their application they rolled out the new code on a Tuesday evening. On Wednesday morning a glitch in the application began automatically buying up billions of dollars in stocks. In just 45 minutes it executed operations that should have taken place over several days. In less than an hour, that glitch cost the company $440 Million dollars and bankrupted the company.

While we will never know the details of which tests were or were not run, we can safely surmise that a crucial requirement for the new code had not been tested. It’s likely that the company was not aware that an important test had been missed.

Introducing the Requirements Traceability Matrix.

The traditional solution to keeping these requirements and tests organized is a Requirements Traceability Matrix (RTM). In a nutshell, this matrix tracks a many-to-many relationship – many requirements to many tests. One test can cover multiple requirements, and one requirement can require multiple tests. Typically, the matrix shows the requirements across the top as columns and the associated tests down the side as rows. In this example there is an “X” to indicate which test cases relate to which requirements.

Historically, these RTMs have most commonly been created using a spreadsheet which can adequately display the many-to-many relationships. However, the problem with spreadsheets is the maintenance they require. They can be time consuming to set up and maintain. They must be constantly updated with new information to reflect the tests performed, test results, defects identified, updated requirements, testing coverage and so on – an additional burden for an already busy team.

There is no question that the old RTM spreadsheet is a very useful resource of information, but to enable teams to keep up with the increased delivery velocity, it desperately needs an update. How can the logging efficiency of the RTM be improved so it’s kept current and useful?

Types of Requirements Traceability Matrix

There are a variety of different types of RTMs. The most common types are:

• Forward Traceability – This type of RTM is used to make sure that the project progresses according to plan. It makes sure that the project has all the appropriate requirements applied to it going forward, and that each requirement is adequately tested.

• Backward Traceability - This type of RTM is used to ensure that the project stays on track. It provides traceability to the original requirements to mitigate scope-creep and that unnecessary additional code, features and tests aren’t added to the project.

• Bi-Directional Traceability – This RTM is a combination of both the Forward and Backward Traceability matrix. It tracks and ensures test coverage of all requirements. It is also used to determine the impact of changes in associated project requirements.

The Matrix Value

Tracking test and requirement association is only part of the RTMs provided benefit. A greater value is provided in showing test coverage for a given release. This enables a manager or stakeholder to quickly determine the level of risk associated with a production run. If you don’t know your test coverage, you can’t really know how much risk is associated with a release. The RTM provides an at-a-glance roadmap of testing progress enabling you to know and meet acceptable levels of risk before a push to production.

RTM for the Enterprise Environment

Tracking test case associations with their requirements counterparts is only part of the solution. When requirements for a large release can number in the hundreds or even thousands, it is essential to also keep track of the test coverage for a given release. At the enterprise level, the spreadsheet method simply cannot keep up.

Whether you use Agile, DevOps or Waterfall or some hybrid methodology, you need to track the requirements associated with every release, test every requirement, and confirm the results of the test. With the spreadsheet method, these means keeping the RTM current becomes an essential but enormous resource burden. The pace of the release environment means that current information becomes outdated almost as soon as it is added to the RTM.

This leaves enterprise development teams vulnerable to outdated information. If you don’t know what your test coverage is, you can’t effectively manage the risks associated with moving a given release to production. A modern requirements traceability matrix must be updated automatically to stay current and to provide an accurate roadmap of testing progress.

The Modern Requirements Traceability Matrix

Plutora has created a highly automated and modernized RTM, tailor-made to the demands of fast paced enterprise release environments. Here’s how it works.

• Integration - Plutora has the ability to integrate with and collect information from your existing development lifecycle tools like HPQC, Selenium, Jira, Remedy and more.

• Centralized Datamart - It leverages these integrations to track requirements automatically from start to finish in a centralized data mart.

• Track Progress & Results -Plutora also automatically tracks and records testing progress, results and defects. This allows the RTM to be automatically updated in nearly real-time and automatically distributed to all appropriate stakeholders.

• End-to-end View- This RTM tracks test cases with their associated test plans, requirements, defects blocked tests, comprehensive release test coverage and state of readiness.

• Improved Efficiency - This automated RTM tracking, reporting and distribution significantly improves the overall performance by allowing your team members to focus on what they were hired to do. It can also be scheduled for automated distribution to stakeholders.

• Improved Quality – It provides the ability to see at a glance the test coverage for every requirement in the release. It allows release managers and other stakeholders to instantly determine if a release is ready for the push to production. It removes the gamble of deployment.

• On the Go Access – The Plutora SaaS based solution enables secure access from any major browser worldwide. It also works on tablets, smartphones and the iWatch. .

• No Equipment Required – The beauty of a SaaS based solution means that you and your team have full access wherever you are without the need of local support or additional hardware.

• Configure Your View – The Plutora RTM also provides the ability to filter by project, release, test plans and test runs. It also provides the ability to configure the RTM by adding cross-tabs and charts to the top to show information that is most important to you.

• Methodology Agnostic – It works with any methodology including Agile, DevOps, Waterfall and Hybrids to draft, discuss, approve, build, test, retest and deliver on your terms.
  
  

Even with so much information packed into the Plutora RTM, reading it at a glance is easy and far more informative than any legacy spreadsheet.

• Test Plan – Selecting a test plan from the dropdown will instantly load all of the associated requirements, test cases and their predetermined associations.

• Requirements – Just like other RTM structures, the requirements for your selected release are listed across the top as columns in a spreadsheet. It tracks all types of requirements and helps you stay organized and efficient.

• Test Cases – The test cases listed down the left side as rows indicate all the test cases for a given test plan. This includes unit tests, integration tests, regression tests, user tests, performance tests and so on.

• Defect Counts – In addition to the “X” as displayed in the basic RTM above, Plutora has incorporated colored boxes to show “Pass”, “Fail”, and “Blocked” status of each tests.

• Progress and Results – The green squares indicate successful tests. Red squares indicate unsuccessful tests with a number in the box showing identified defects. Gray squares indicate tests that haven’t started yet.

• Drill Down – Click on a square with a number to drill down and see a list of the identified defects for that requirement and their current status.

The benefit of the Plutora RTM is that all test results are automatically tracked and stored in a centralized data mart creating a single source of truth. It eliminates the need for someone to collect, compile data in order to roll-up and distribute reports. With little more than a glance, you can quickly identify detailed test coverage for even the most complex releases, you can track testing progress, results and discovered defects.

In the same few seconds, you, your team and your stakeholders can determine if a release is really ready for deployment, or if it needs more attention – and where. Most importantly, everyone is perfectly aligned with the same visibility and the same working objectives. The benefits are clear, which leaves only the question, “What is this visibility and team alignment worth to our team?”

The requirements traceability matrix is powerful. It’s one of those tools that once you’ve used it, you can’t imagine how you ever got along without it.