Control Costs: Keep Track of Test Environments

Test environments cost money; this is the inescapable fact of test environment management. It can be difficult to justify the expense in your conversations with finance and management, especially if they lack a software development background. Businesses are often fine with dropping millions on production infrastructure because there's a direct line between keeping production running and revenue. On the other hand, QA environments are often viewed by management as unnecessary overhead. "Why do we need all these QA environments?"I've even spoken with an executive who viewed increased spending on test environments as a negative. His mistaken assumption was that his company was forced to spend money on test environments because his developers wrote buggy code. It was the first time I had heard such an incorrect assumption, but I wasn't surprised. "IT" is still a mystery to most senior executives and the idea that testing is only necessary because programmers write bad code merely reflects the inexperience this company had with software development.For test environments, your best arguments for an increased test environment spend are quality and agility. Test environment numbers are directly related to the quality of customer experience in production. That being said, it is still an uphill battle. Test environments are overhead, and CFOs can be skeptical of the return on investment. In the next few blogs, I'm going to focus on tools you can use to reduce test environment cost. Here's the first suggestion.

Keep Close Track of Test Environment Allocation

A poorly executed test environment management effort is a "fire-and-forget" operation. Managers allocate environments to projects, projects are left on their own to manage infrastructure, and environments are rarely reclaimed. In these organizations the test environment management team doesn't manage environments, they simply hand out hardware to development teams without measuring any metrics. A group that just hands out hardware in an IT department is akin to a free candy store; everyone will show up asking for an unlimited number of "test environments."This model leads to inefficiency. Organizations not keeping a close eye on test environment allocation fail to understand which environments are being used. When a TEM just hands out capacity and infrastructure to development groups, there is no economic or market pressure to use environments efficiently. Teams just ask for environments and sit on them, even if they aren't being used. "Have you used this test environment in the last year? I'm not sure."

That's not a Test Environment, That's a Secret Project

So high-profile projects might end up with twelve QA environments, all requested under the pretense that this project needed to support twelve simultaneous development streams. This is usually not the case. Even the largest, most complex projects in the industry can only successfully build on three parallel development tracks for version N, N+1, and (maybe) N+2. But, this sort of parallel development is full of risks and coordination challenges. If a team has twelve QA environments, in reality, this team likely uses QA environments to support personal development branches and other systems unrelated to QA testing. For example, one of our clients handed out thirteen QA environments to a single project, only to realize that the project had repurposed some of this infrastructure to another internal project that was being developed "off the books." What was intended to be used as QA infrastructure was repurposed as CI/CD infrastructure for a secret project they were going to propose for next year's budget. In this way, the test environment management budget was being used to support another department's secret infrastructure.If your organization is large enough, and you start using Plutora to keep track of test environments, there's a good chance you'll find entire racks full of hardware being used to support "testing." But in reality, they are being used to support non-standard build pipelines, custom developer environments, and other uses that aren't properly tracked. (And, some of these systems may be necessary, but they shouldn't be tracked against your test environment budget.)

Fail to Keep Track of Test Environments: Pay a Penalty

Organizations that don't use Plutora to keep track of test environment use are often paying many times what they should. They experience a lack of transparency that encourages teams to hoard test infrastructure, and they create the possibility of a "dark budget" that departments can drive entire projects through without being accountable.If you are attempting to reduce your overall test environment management costs, one of the first steps should be auditing your test environments and finding out which ones aren't used. You may be surprised to find that your organization is holding onto test environments used to qualify long-retired software systems. Or, you may be surprised to find testing environments that have been repurposed by teams for uses far outside the scope of software testing.Put simply, one of the most effective ways to keep test environment costs down is to identify wasteful environments and reclaim them using Plutora.