Most Common Enterprise Release Risk: Last Minute QA Testing

It’s a “given” in software development - release schedules always slip. Requirements shift, developers underestimate timelines, and quality assurance (QA) finds unexpected defects. Another law of software development is that final release dates are often inflexible. If a market or a holiday shopping season defines your release date you understand how important it is to meet a fixed timeline.

Once you’ve promised a release to the business at the end of the quarter, you are under pressure to deliver software on time and under budget no matter how much your internal timelines have slipped. As a release manager, it is your job to make the best of a challenging situation and to find creative ways to bring a project to an on-time delivery.

As release deadlines near, IT management is often faced with a series of difficult decisions:

• Do you sacrifice features to meet an inflexible deadline?

• Do you release code with known, but acceptable defects?

• Do you compromise quality and ask QA to rush the testing process?

Usually, it’s the last question that is answered affirmatively. QA timelines are compressed and QA managers are left with the task of regression testing software releases in a rush to make up for time lost during the development cycle. This “last-minute” approach to QA testing and verification is an industry-wide practice, and you can recognize it by the number of late-night, last-minute testing sessions that you see at your company.

To make matters worse, QA is frequently asked to test on shared environments that do not have the correct data to uncover problems in production. QA has to wait for environment setup and teardown that, in turn, leads to even more tightly compressed testing timelines. Compressed QA timelines? Testing on inadequate environments? All of these are recipes for greater production downtime and higher risk to the business.

How can you avoid these last-minute, late-night approaches to QA? Start by planning your test environments and release schedules in advance.   QA timelines are often compressed because QA occupies the “transition point” between development and production. A development-driven IT department tends to underestimate the amount of effort required to qualify a release for production.

To protect your QA department from late-night, last minute QA - defend your schedule, organize your test environments, and keep track of your team’s capacity.

1. Defend your schedule - Create a plan and stick to it: QA shouldn’t just be a passive actor in a release schedule. A QA department should proactively communicate how much time it needs to verify software. If QA needs two or three days to verify a release put this timeline in the schedule and make sure that your release managers understand that this is not flexible. When time is lost during development, don’t let planners try to compensate by stealing it from QA.

2. Organize your test environments - Reserve QA environments ahead of time: QA and performance testing environments are typically in a state of constant transition with multiple teams competing for resources and databases in various states of readiness. If your project has a fixed timeline plan your environment requirements in advance to make sure that avoidable environment delays do not affect your QA team’s timeline. QA teams often have to wait for databases to be refreshed or environments to be fixed. Such delays can be avoided by giving your environment managers a solution to plan and assign environments alongside release schedules.

3. Keep track of your QA team’s capacity - Avoid QA burnout: A fact that IT management often forgets is that IT is about people - people who need to sleep, eat, and sometimes take weekends off to relax. If your QA department is being asked to stay up late nights to verify software every single day of the week you are going to suffer from attrition and burnout. You need a solution that can help manage the capacity of your QA staff. One that shows you if there are windows of downtime that present opportunities to shift people between projects rather than asking your team to work at 150% for three week’s straight. Managing your team’s availability becomes more important when software releases become more frequent.

Plutora provides solutions to manage test environments, organize your environments, and to keep track of your team’s capacity. Our software has been designed to support the entire IT department and to organize the various teams associated with a software release.

In our experience, with the largest, most-challenging software releases, the QA department is asked to perform super-human tasks at the tail end of the development cycle. The team is asked to stay up all night to make up for time lost elsewhere, and they are often asked to wait multiple days while databases are refreshed. These problems can be avoided with better management, coordination, and planning and these capabilities are exactly what Plutora provides release managers, environment managers, and QA managers.