Admin weighing down releases
With the natural caution of an insurance provider, QBE was no stranger to a well-planned, well-organized release process. However, as release cycles went from quarterly to 6 weeks and then to monthly, the volume of governance managed by the team of just 2 increased significantly as overhead for each release was overwhelming. It included managing status, coordinating release schedules and scopes, ensuring compliance & governance, coordinating IT resources and most importantly, making sure that business value was being delivered. Every release needed to coordinate with up to 10 major projects and was intermixed with dozens of product enhancements and bug fixes. Lengthy deployments plans were managed using MS Project requiring email confirmations of the various activities. The operations team was also engaged when production infrastructure changes were needed. Tracking this information was done mostly in spreadsheets, meetings and by pulling information out of individual systems.
Planning and scoping out a release involved information from 5 different tools (Clarity, ServiceNow, Jira, ExecView and ClearQuest). Release managers had to meet frequently with teams to validate information pulled and reconcile that against the latest plans and updates. It was nearly impossible to get real-time updates. Once gathered, the information quickly became stale.
Governing the release itself was an endless back and forth between release managers and the individual dev & test teams, along with the other departments. The challenge was to ensure quality releases to production. This involved progress of features, user acceptance testing and defects in order to ensure new issues aren’t introduced into production.
Managing the cutover process involved lengthy deployment plans managed in MS Project. These went through approvals during the planning. During the execution of deploying the release into production, activities were managed via email exchanges with the same release managers spending time sifting through the communications to ensure high quality software delivery.
Vulnerability to human error
Amalgamating all this data into consolidated plans left releases especially vulnerable to human error - each team completing their plan template could put in the wrong information, or it could be copied across wrong to the main plan. Sometimes information was merely out-of-date – a spreadsheet hadn’t been updated – but that still made it wrong.
“The release process was so manual, it was taking time away from focusing on the scope and quality of releases. You couldn’t be a release manager because you were spending all day updating spreadsheets.”Sophia Pepe, Enterprise Application Release Manager, ANZO
The release managers were so tied up with the administrative burden, it was giving them no time to do what they did best – put forward their subject matter expertise and guide the release to success. The QBE release team needed a solution to eliminate these spreadsheets. The key factor was that it must integrate seamlessly with their existing release tool environment, consolidating information rather than just being another tool added to the mix.
Test environment collisions
The DevOps team had a strong reliance on spreadsheets to track the preproduction environments that were utilized during the various development and testing phases. As crucial resources, these environments were in high demand. Without centralized ownership or scheduling, scheduling collisions and misconfigured environments caused project delays and invalidated testing efforts. The lack of configuration management exacerbated the problem that when an environment was scheduled, the required configuration wasn’t actually set properly. During the subsequent validation stage, tests would fail, with defects sent back to the development team where they would discover the misconfiguration. These challenges wasted precious time from dev and test.
QBE was understandably cautious about taking a leap with a new tool – it was imperative that no errors entered their existing process. Work started on the Plutora implementation in February. Two months later, one of the release managers carried out a Plutora shadow run. Plutora ran alongside the usual flurry of spreadsheet updates. By May, the spreadsheets were burned, and QBE fully moved to Plutora managed releases.
“The way everyone came on board and embraced it, it's clear it's made people’s lives much easier.”Sophia Pepe, Enterprise Application Release Manager, ANZO
It’s often the case that introducing a new tool or way of working experiences friction from those on the frontline. From day 1, the release team experienced no pushback, not a bad word about Plutora from any of the developers, testers and operations – even the change-averse members of the team were onboard. With just a few hours training, they were ready for the first deployment night.
Shifting task ownership for faster deployments
The first and clearest benefit is that the mass of spreadsheets has been reduced to a far more manageable one. Each team can now update their tasks in Plutora, and the information is updated in real-time. Release managers can see the status of every task from one clear central source.
“I didn’t want to be updating spreadsheets for the rest of my life. With Plutora, it’s like looking at your old spreadsheets without having to do the work of building them”Sophia Pepe, Enterprise Application Release Manager, ANZO
Deployment planning meetings have changed from battling through unwieldy 300 line Microsoft Project Plans, struggling to keep track of who is responsible for what, to everyone having their assigned tasks clearly visible. The right questions get asked, answered immediately and the plan in Plutora updated on the spot.
Plutora also manages the compliance and sign-offs for scope items and projects during code deployments. Instead of managing approval of release notes over email, teams create a Test Environment Change Request (TECR) to go through the approval flow process, which automatically generates the status that the release notes request. The result? Hand-off times eliminated, and deployment times dropping from 9 hours to 6.
One clear view of the entire value stream
Plutora hasn’t only made things easier for the developers and release managers – upper management also benefit from the tool’s visibility. Managers need their information to be up-to-date, easy to understand, and instantly accessible.
Before Plutora, getting a report to management would be yet another burden upon the release manager. Pulling out relevant data from the many spreadsheets and formulating it into a report would take hours. In a fast-paced release, this could be out-of-date almost immediately. Now, managers can access the Plutora Insights dashboard and check release status in real-time for themselves.
“Managers don’t have to check in with me how the release is going, they can see it for themselves from the Insights dashboard.”Sophia Pepe, Enterprise Application Release Manager, ANZO