Managing Test Environments

Test Environments (also called QA or Pre-Production Environments) are a vital cog in delivering fully tested code and instilling confidence there will be a successful project or release. Any weakness with the robustness of the test phase will likely have cost implications for the business further down the track so it is important your test environment suffers for minimal issues. Below I will discuss some common issues which arise from poor processes in managing Test Environments:

• No source of truth for the Environment Schedule

• Environment does not have the Connectivity to execute all test cases

• Test cases executed against incorrect code or config settings

• Uncertainty over Responsibilities and Ownership of the Environment

• Unstructured Change and Defect Management Processes

• Competition for the Same Environment

No source of truth for the Environment Schedule

With schedules being emailed around for weeks and months prior to the commencement of the testing phase, some stakeholders may be looking at different versions and dates. A single source of truth is imperative in ensuring that no time in the test phase is wasted as well as ensuring that an environment is not double booked and does not suffer from excessive periods of down time.

Environment does not have the required connectivity to execute all test cases

There can be many reasons why a test environment may be lacking the required connectivity, such as cost, complexity, resourcing or lack of vendor support. Where lack of connectivity leaves some test cases being unable to be executed, it is important this is highlighted as early as possible to relevant stakeholders so that a decision can be made in regards to whether this situation is acceptable and what steps can be taken to remedy this and ensure the integrity of a release is not compromised.

Test cases executed against incorrect code or config settings

Some test phases commence with all the code deployed into the test environment while in other instances there may be numerous drops occurring over a few weeks or months. Planning the drop dates as well as knowing and communicating when the code has been deployed is important to ensure there is no duplication of testing efforts.

Another reason testing for this occurring could be because that there may be no record of the version of all components in an environment. This may cause some or all of the executed test cases being redundant and needing to be re-run.

Uncertainty over responsibilities and ownership of the environment

Some organizations may have large dedicated test environment management teams while in others the test environment management may be managed by a hybrid of test and development resources. If a test environment does not have a single point of responsibility then the environment may be poorly setup or poorly maintained due to the lack of accountability and competing tasks for relevant resources. A lacklustre test environment may limit the scope of test cases that can be executed and reduce confidence in the testing process.

Unstructured Change and Defect Management Processes

Environments are regularly shared by projects when there is no record keeping and an un-structured approach to creating and deploying Environment CR’s in a test environment it is likely that working code will at some point be broken. This will create extra overhead for developers and testers in investigation, resolution and deployment of the fix into the test environment.

Competition for the Same Environment

In the face of changing business priorities or the need to extend your testing window (possibly due to some issues above) then it is important that an informed decision can be made in regards to which project should be given precedence and what are the downstream impacts to other projects and their environment bookings.

How Plutora can help manage these issues?

While issues always come up to create complexities, having a centralized tool like Plutora can control and mange the above issues by providing the following functionality:

Test Environment Source of Truth

Plutora provides users with a way to setup, request and confirm environment bookings. A coordinated view of all Environment Bookings can be found in the Environment Booking Screen.

Tracking System Impacts

At both Project level and at Enterprise Release level Plutora tracks the systems (applications) being impacted. By having this information Release Managers & Test Managers should be able to know well before the commencement of a testing cycle commences what the connectivity needs of an environment are.

Version tracking of code, stack, layer and components

Plutora provides users the ability to store version details of components in a test environment. Also when raising Environment CR’s the new version details can be included so the Environment details are updated in real time once the CR has been executed.

Uncertainty over responsibilities and ownership of the environment

Plutora has a strong emphasis on the RACI (Responsible, Accountable, Consulted, Informed) Matrix across all areas. By assigning roles to users or a team for an Environment, it should be clear who is responsible for particular tasks so that an Environment is well maintained.

Structured Environment Change Request Processes

Plutora provides a structured menu for raising Environment Change Requests and listing details of the CR as well as whether they are scheduled or completed.

Clear Understanding of the Impacts of Adjusting Environment Bookings

All Environment data within Plutora can be seen on the Environment Booking screen. By selecting Project View and running an impact analysis of a delay to a project, users are able to quickly see booking conflicts and impacts should and an environment booking need to.