Plutora Blog - Test Case Management, Test Environment Management
Reclaiming Underutilized EnvironmentsReading time 5 minutes
What is an Environment?
When we discuss Environment Management, we’re not thinking about simple systems. An environment isn’t just a web server and a database. Instead, we’re thinking about complex systems with a lot of moving parts. For example, a large e-commerce website that generates billions in revenue may have several different environments: one for development, another for QA, a staging environment, and a production environment.
Environments are Complex and Time-consuming
Each of these environments may consist of several separate systems for inventory management, payment processing, catalog management, and website rendering. These aren’t environments that can be spun up and down in a matter of hours. No, these environments often have more than 30-40 VMs each playing an important part in a complex system. These environments also have databases, caches, and other dependencies which are often managed by several separate groups.
In a complex environment,”setting up the QA” environment may take days or weeks of coordination between multiple teams. This is the reality of large enterprise software systems today. It isn’t good enough to set up a development environment and a production environment. You need an environment to capture different stages of a given release: development, QA, staging, production, and if you need a testing environment for production deployments you will likely need a separate environment for emergency fixes to production.
Baseline Environment Requirements:
4-5 environments (DEV, QA, Staging, Production, ProdFix)
Running Releases in Parallel Increases Environment Demands
At a minimum a large software project requires four separate environments, and if your organization runs multiple releases for the same system in parallel the number of environments can multiply even more so.
For example, if a major website is launching on a new architecture they might run a release in phases. Phase A features might launch in January while Phase B features might be scheduled for March and Phase C features are scheduled for June. Over the course of these six months, you will likely have release timelines that overlap with each other, the level of overlap will drive the requirements for the number of environments you need.
- Phase A might be in active development from July to November entering into QA in December and going into Production in January.
- Phase B might be in active development from August to January, entering into QA in February and going into Production in March.
- Phase C might be in development from August to March, entering into QA in April, and going into Production in June.
To support these timelines, you’ll need three development environments, two QA environments, two staging environments, and potentially two production environments.
Wait. Why two production environments? The best practice for complex website deployments is to launch new versions of a website or application in parallel with the old version and run an experiment between the two. If Google updates search functionality, they launch the new version of the search for a small population of users 1-5% and they run an A/B test before deciding to proceed with a full release.
With parallel release schedules, your environment requirements just jumped up from 4-5 environments to 9-11 environments. This is also assuming that only two of the three phases will ever overlap in QA and Staging.
Parallel Release Environment Requirements:
9-11 environments (3x DEV, 2x QA, 2x Staging, 2x Production, 2x ProdFix)
Reminder: Environments are Expensive
Think about the environment requirements for a large e-commerce site or a large bank. Each of these environments often requires separate hardware allocation, database licenses, software installations, monitoring systems, logging systems – these environments are not cheap.
While you can likely save money by using some shared resources here and there, doing so introduces inefficiency. If your QA and Staging environments hit the same service backends or databases you will be constantly managing these conflicts. For example, if you want to run a performance test against a staging environment you will need to make sure that it won’t interrupt QA due to shared resource conflicts.
Your development environments may only cost a few tens or hundreds of thousands of dollars to stand up. This is cheap compared to your production environment. For applications that need to scale to meet global demand, a production environment with the necessary levels of redundancy might cost upwards of $10-100 million to support traffic.
Hardware and software cost is only one factor. Often much more costly is the time and effort required to setup and configure these environments. If you don’t have a handle on your release environment requirements you will be constantly asking people to build and rebuild environments to support development.
- “Can you configure the QA system to hit the Phase B database?”
- “What version of Phase A code is deployed to Staging and can you configure the services to connect to the right cache?”
- “Can we do a deployment to Production today for the Phase B release than can you deploy Phase C to the Staging servers?”
If you’ve ever been an environment manager on a large software project will understand the challenge. You understand that environment management isn’t a part-time responsibility. It is an all-consuming effort that requires constant attention, and you’ll also understand that you’ll get this question all the time:
“Is anyone using that environment?”
The reality of environment provisioning in a resource constrained organization is that our labels for environments (Development, QA, Staging, and Production) they are just that: labels. In most organizations teams are struggling to find the appropriate resources needed to deploy and test complex software systems.
Without proper planning for environment management and without a tool like Plutora showing people how releases drive environment requirements you end up juggling whatever hardware you have available to meet the current need.
Next time someone asks you “Is anyone using that environment?” You should tell them to go look at the Plutora Environment Manager dashboard.