What is Test Environment Churn?
Jan 4, 2016
Test Environment Churn and Test Environment Pollution are two things that happen when you have environment managers creating hundreds of environments to support hundreds of projects across several departments. In this post I’m going to focus on Test Environment Churn: what is it? And, what does it represent in the context of Test Environment Management.
Churn rate is a common measurement for businesses tracking employee or subscriber retention, and it is a metric that is measured over a period of time. For example, if you are tracking the churn rate for subscribers for a business over a year, a 33% churn rate means that one third of new customers signing up in a given year will cancel a service. You can also apply the same metric to retaining new employees. Businesses often keep track of the churn rate for employees trying to drive that number down to increase employee retention.
Churn is a measure of stability for test environments, and it is a metric that can change depending on the type of application development or service management your organization is practicing. In the context of test environment management:
Low environment churn over the course of a year or a quarter means that you are supporting a stable set of environments for a stable set of teams.
High environment churn signals a high likelihood that the environments you are creating today will only be used for a short period of time only to be discarded for a new set of environments to support a new collection of projects.
High Environment Churn
High environment churn happens when you have a rapidly evolving set of environments and teams working on highly dynamic technologies. At a startup or in an organization using the latest technologies you often encounter teams running proof-of-concept projects or teams developing front-end systems that may only be required for a brief period of time.
In an organization moving toward a microservice architecture you’ll often have immediate requests for new development environments as software developers start to disassemble large, monolithic systems into smaller, more independent modules. This is one of the main factors leading to a rapid increase in the number of environments required in most large organizations.
With microservices and new approaches to deployment automation it is common for environments to feel like “Gold Rush” towns. One day you might have a request for several environments to support a particular set of microservices and the next day the attention will move to another part of the system architecture. In a high churn environment your test environments might be easy to create, but they are just as easy to abandon. When you support a high churn environment your SLAs are measured in hours or days and the lifetime of any given test environment can range from hours to months.Pros: In a high churn environment you’ll have practice creating and maintaining new environments because you’ll be asked to create environments every day. It is also very likely that the high churn rate for test environments will encourage more environment automation and self-service management of infrastructure. If you have created self-service environment creation tools for your projects you have achieved one of the goals of DevOps. By putting environment creation in the hands of developers you are enabling a more immediate approach to environment management, and your job as an environment manager has made a transition from day-to-day implementation to week-to-week governance.Cons: It can be exhausting to support a high churn environment because your environment managers have to maintain a constant state of readiness. When you have a high rate of environment churn you are forced to create self-service systems that allow developers to create and manage environments without being blocked by central support function. This has some advantages, but it also creates some governance problems. Your environment management teams will have to focus more on quota issues and with high churn you have an increase risk of developing test environment pollution over time as environments become abandoned.
Low Environment Churn
Low environment churn happens in an IT department that is supporting less internal application development and more service management. When your IT department is supporting legacy systems or system developed by third-party vendors there is usually less demand for immediate, full-stack testing environments.
An example is a web site that runs Oracle applications to support e-Commerce or a bank running a third-party system for account management. In these scenarios it is unlikely that a group of developers are going to show up over night with requirements for novel, cloud-based architectures. When you support a low churn environment your SLAs are measured in weeks or months and the lifetime of any given test environment can range from months to decades.Pros: When you have low churn you can plan ahead multiple quarters or years and have a very predictable budget for test environment requirements. You are rarely surprised by multiple, simultaneous requests for environments and your teams aren’t going to be pointing at you as a constant source of delay. Low test environment churn is also more compatible with centralized environment management. When you have low churn you don’t have to emphasize self-service as much as you have to emphasize stability. Your job is to keep a relatively static set of environments available and to respond to requests for new environments within an SLA measured in weeks.Cons: When you have low test environment churn it is easy to lose track of the systems you support. When a legacy system runs for multiple years without interruption this can create situations where no one remembers how to perform a critical maintenance or upgrade task. When environment are so stable they don’t have to be touched for years you run the risk of forgetting how to support them.
For example, if an Oracle application that sits at the heart of your business is running for several years uninterrupted and a team suddenly requires a test environment manager to perform an upgrade it may take weeks to find an individual with the correct skillset to perform this upgrade.
Using Plutora to Measure and Manage Test Environment Churn
Plutora is the first tool that let’s you track multiple approaches to test environment management across all of your projects and departments at once. You can track the stability of your test environments using Plutora’s comprehensive overview across the organization and identify different groups of projects that cause different levels of test environment churn.
For example, with Plutora you can identify a group of less dynamic, more stable projects that form a solid foundation of test environments that may need less attention than projects that require constant environment updates and changes as software evolves. When you look for different levels of churn this helps you assign resources and tailor your test environment management support activities to match the expectations of different teams.
Download our free eBook
Mastering Software Delivery with Value Stream Management
Discover how to optimize your software delivery with our comprehensive eBook on Value Stream Management (VSM). Learn how top organizations streamline pipelines, enhance quality, and accelerate delivery.