Continuous Integration vs. Continuous Delivery vs. Continuous Deployment

Nov 13, 2018

The world of software development seems to create new buzzwords, processes, and methodologies almost as fast as it develops new software tools. The reason is that the industry is constantly evolving and becoming more efficient. However, in the software development world, there’s no such thing as one-size-fits-all solutions. Some methodologies will work better in certain scenarios than they do in others. The big question everyone is asking is, what will work for us? Or rather, what will work for us in this or that particular scenario? Recognizing that each of our organizations may have a variety of different methodologies from one product stream or dev team to another.In this article, we will discuss the differences between “Continuous Integration,” “Continuous Delivery,” and “Continuous Deployment,” and how they may or may not work for your organization. We will also discuss how they compare to an Agile or DevOps environment.

To better understand how they each fit in and are applied, we need to develop a baseline development lifecycle to ensure we’re all on the same page. While there are a variety of nuances of the lifecycle from one organization to the next, in general, the key phases remain more or less the same. The core phases of the development lifecycle typically include a planning stage, followed by coding, building, integration, testing, release, and then deployment to the production environment.

Software Development Stages

To better establish our foundation, we need to understand the basics of the Agile and DevOps methodologies and their roles in the value stream. To do that, we’ve provided a short explanation of each to get us started.

The Agile methodology

Agile Software Delivery Stages

This methodology takes the iterative approach to software development, by adding layer upon layer, building, or rather evolving, the application one sprint at a time. Regarding the lifecycle, the Agile methodology addresses the Plan, Code and Build stages. When the developer checks the code back in, automated processes then build and integrate these new iterations multiple times a day. After the code is integrated by automated processes, the new code is then passed through the remaining stages of the lifecycle manually, with little or no further automation.


DevOps Software Delivery Stages

DevOps is described in Wikipedia as “a software development methodology that combines software development (Dev) with information technology operations (Ops). The goal of DevOps is to shorten the development life cycle while also delivering features, fixes, and updates frequently in close alignment with business objectives. The DevOps approach is to include automation and event monitoring at all steps of the software build.” …Development + Operations = DevOps. Typical DevOps methodology can overlap with Agile, but it typically picks up once the code is checked in and then addresses the rest of the stages in the value stream.

Iterative Methodologies vs Waterfall

Iterative vs Waterfall Software Delivery

Both Agile and DevOps take the iterative approach to software development. These methodologies are highly beneficial when something in production is better than something being developed. The image below provides an example of how even when the project is still being developed, there is something available and fully functional for the customers to use.

This image illustrates the value of the iterative approach. With the iterative approach, each floor is developed, built and tested before the workers move on to work on the next floor. This way the building tenants can move into that floor while the construction then begins and moves forward on the floor above, and so on. In contrast, the waterfall methodology requires the entire building to be fully developed before it can be built. And only when the entire product is built can it begin to be tested to determine if it’s acceptable for clients.

One of the key challenges with waterfall is in troubleshooting. Using the above example, if there is an electrical issue found on the 8th floor of the building, the technician needs to test each floor to narrow down and identify where the issue is originating. Only then can the issue be addressed.

However, with the iterative approach, if the same issue occurs, there is only the 8th floor to test. Each of the previous floors already passed the tests, so we can already rule out that the issue isn’t located there. If by chance the issue was located on the 4th floor, the tests would have identified it, enabling the developers to resolve the issue before they started work on the 5th floor.

Don’t get me wrong. The iterative approach isn’t the grand solution to every problem. We will discuss those challenges in detail later on in this article.

Why do I care?

Competitive differentiation is based upon a company’s ability to deliver their products and services faster than their competitors. This means that the software solutions that collect, track, manage and deliver those solutions has to also be delivered faster, cheaper, and with better quality than their competitors. Each of these methodologies discussed below delivers on all three of those accounts, though in slightly different ways, varying degrees and with different strengths. They can each be custom tailored to your specific needs, including products, velocity needs, project scale, project quantity, and workflow.As companies strive to reduce operational costs, development teams are doing all they can to increase delivery velocity. With the focus on increasing speed, development teams are also under pressure to do so without jeopardizing quality. The classic mindset is that you can’t have both speed and quality. However, with these methodologies, the teams that use them prove this saying wrong daily, at least in regards to software development.

What is Continuous Integration (CI)?

Wikipedia describes Continuous Integration (CI) as “the practice of merging all developer working copies to a shared mainline several times a day.” The CI Model takes Agile a step further by automating the integration and Testing.The thought of integrating new code with the mainline of the application code several times a day can be a scary thought for those more familiar with the waterfall methodology. At a glance, it seems that this would be massively problematic, resulting in the introduction of numerous bugs into the mainline code. However, as the majority of development teams have already discovered, the result is quite the opposite. The code segments being worked on at any given time are much smaller and are therefore easier to both code and debug if there is an issue. So, unlike the waterfall methodology where you create an entire version of the application and then submit it for testing, with continuous integration, the developer submits only small segments of the application at a time. Each of these code segments is then saved and automatically sent to the build server either immediately, or at regular intervals throughout the day. The build is then automatically passed to a test server that executes a series of automated unit tests to ensure that the new code segments work as planned. For additional information on the CI framework, Gartner has published a great article called, “A Guidance Framework for Continuous Integration: The Continuous Delivery 'Heartbeat,” by Bill Holz and Sean Kenefick.

By doing this, the new code segments are built, integrated and tested in mere minutes, providing the developer with immediate feedback on the status of the code and any related issues.

How it works

With CI, the hand-off from one stage to the next is fully automated up to the completion of the testing stage. This is starkly different than that of the waterfall methodology, where the hand-off between stages is typically fully manual throughout the entire lifecycle and deals primarily with completed applications vs small code segments. This automation is achieved through the use of a combination of tools that perform the code check-in, build process, integration to the mainline application and testing. The remaining hand-offs from one stage to another in the lifecycle are all manual.

Continuous Integration (CI)

Continuous Integration (CI) Software Delivery Stages

Key Strengths of Continuous Integration (CI)

  1. TestingIt’s no secret that testing the same test scripts every day can be a highly monotonous duty. Most development teams have found that the automation of test scripts wherever possible goes a long way. Not to mention, it also provides immediate feedback to the developer in mere minutes.

  2. Issue ResolutionWhen the automated test scripts do find an issue in the code, because the code segments are small, the developers have been able to make quick work of issue identification and resolution.

  3. Phased ChangeoverWith small iterative changes, studies have confirmed that the migration from an old version of the software to a newer version is much easier because the users are eased into the changes, instead of having to learn a new version of the software all at once.

  4. Improved Team MoraleSurprisingly, submitting these iterative changes has also improved dev team morale, especially when tasked on larger projects. The reason is that it’s part of our human nature that we like to see accomplishments regularly. Unlike the waterfall methodology, where a large project can drag on for weeks or even months without completing any key milestones, the iterative approach allows developers to achieve multiple milestones per day. This also provides the immediate gratification of seeing their code passing the automated tests several times a day.

  5. Increased VelocityTeams have found that with the iterative changes to the code, automated builds, automated testing, and the rapid identification of issues, the overall velocity of the project delivery is significantly faster in comparison to waterfall methodology.

  6. Improved QualityBecause issues are found and resolved with each sprint, those larger, more comprehensive development issues are substantially mitigated. This is because once a sprint has been deployed, subsequent code iterations are then added to the code that has already been proven to be functionally correct. This eliminates the issue of needing to fix multiple incorrectly developed modules because of a coding issue in a single module.

  7. Improved BudgetWith the increase in delivery velocity and shifting the identification and resolution of coding issues left, teams are saving a lot of time and resources, which all add up to a healthier bottom line.

What is Continuous Delivery (CD)?

As the next evolutionary step of CI, continuous delivery (CD or CDE) takes the development lifecycle automation a bit further. With CDE, the code is compiled, tested and dropped into the staging environment. Though not required for a continuous delivery model, some teams will also automate and execute system tests once the code is in the staging environment. Some highly recommended resources to learn more about continuous delivery include the book “Continuous Delivery” by Jez Humble and David Farley, and also Jez Humble’s website’s important to note that many credible authors do not differentiate between Continuous Delivery and Continuous Deployment. So, in researching these, it can sometimes be confusing to determine which one they are talking about.

Continuous Delivery (CD)

Continuous Delivery (CD) Software Delivery Stages

The key benefits of the CDE model include all the same benefits of the CI model, plus a few more

  1. Improved VelocityThe delivery speeds are notably faster due to the addition of the automated release into the staging/test environment. If your team also automates the system tests, it makes the overall delivery that much faster.

  2. Phased ProgressionLike Agile and CI, small segments of code are iteratively added to the mainline throughout the day. As the new code is added throughout the day, the mainline application evolves step by step to the target state.

  3. Always Production ReadyWith this methodology, the product version that sits in the staging or simulated production environment is always in a fully production-ready deployable state. This allows it to double checked, or to be held until other features are added and enabled. The advantage is that it is always ready and can be released into production at the push of a button. This methodology gives your leadership total control over product delivery.

  4. Release ControlWith a product that’s always production ready, the deployment to production becomes a business decision, providing total control over the user experience.

  5. Additional TestingThis model allows the production-ready product to undergo additional, more comprehensive testing in various test environments, including integration testing, system integration testing (SIT), User Acceptance Tests (UAT), load testing, performance testing, and any others that your organization may require.

What is Continuous Deployment (CD)?

As the current evolutionary pinnacle of development lifecycle efficiency, continuous deployment (CD) enables a development team to integrate code segments to the production environment several times per day. This methodology is ideal for development teams that work on large websites or online tools.Application of CD can be found at a number of large online organizations such as Amazon, Google, WordPress, and many others. It allows them to assign developers a specific set of functions to which they will continually work on and improve upon based on user feedback. Because of the small iterative changes that are constantly being added and updated to the production product, the users don’t experience any big changes that can sometimes cause consumer anxiety. Rather, a series of small simple changes add up over time, becoming big changes. This feeling of continual evolution has, in many cases, become expected by consumers for many of these high-traffic websites.

The level of automation achieved throughout the CD lifecycle is made possible by a combination of several different tools that work in tandem to receive-execute-and-pass to the next tool/stage in the lifecycle. Once the developer saves the new code, it is automatically built, integrated, tested, and deployed directly to the production environment. Because of this direct deployment into the production environment, it may not necessarily be the best solution for everyone. Some perceive it as taking control of when to move code to production from the leadership and giving it to the developers.

Continuous Deployment (CD)

Continuous Deployment (CD) Software Delivery Stages

The key benefits of this are similar to those of CDE. However, there are also others that should be mentioned.

  1. Morphed MigrationThe product gets built iteratively in production and more or less morphs from one version to the next. This approach substantially reduces the anxiety that customers have when they are given a whole new version of the software that they now have to learn.

  2. Enhanced Delivery VelocityGone are the days of creating code and lobbing it over the fence to builders, who then bounce it to the testers and so on. With continuous deployment, the developer can check code and get a pass/fail notification within minutes. If an issue is detected, they can resolve it and have the new version submitted and into production within a few minutes. If no issues are detected, the new code is moved automatically into production.

  3. Shift LeftWhen a developer checks in code, the automated processes take the code and move it through the entire lifecycle and if it passes each gate, it gets deployed directly to production. This provides the developer with nearly immediate feedback regarding if a code segment has a defect or not. If an issue is detected in the automated tests, the developer is notified and can resolve the issue immediately.

  4. Expedited TestsTesting has traditionally been a bottleneck in the development lifecycle. However, with CD, automated tests can be set up throughout the value stream to be as thorough as needed. This means the testing can be set up at the appropriate stages to address the progression of the code throughout the lifecycle. This can include automated unit tests, integration tests, regression tests, performance tests, and so forth. These can be set up and executed at their respective stations in the lifecycle providing a thorough testing framework to ensure quality. However, because the gate management and handoff from one state to the next are also fully automated, the progression of the code through the lifecycle occurs at light speed when compared to the legacy waterfall approach.

  5. Real World ExperimentationThis model also allows for real-world experimentation, such as A/B testing, and allows for immediate consumer feedback. For example, if you need to change the location of a certain feature on a website, but you’re not sure of the best new location, you could make the change, monitor real-time customer usage, and make adjustments accordingly.

What are the challenges?

Over the years, the evolution and efficiency of the development lifecycles have certainly caused a lot of discussion among developers and leadership. However, while it has certainly benefited development velocity, quality, and consumer confidence, there are also some downsides that need to be considered.


Issues associated with a lack of visibility are always at the top of the list, causing challenges in different areas. A few of them are:

  • Managing ProgressTracking requirements as they get pulled and worked on by developers may sound easy enough. However, the reality is that the current progress of multiple autonomous development teams, the current state of each requirement, and its current status as it gets carried along the chain of automated processes can be very dynamic and fluid. This level of fluidity can make reporting by legacy tools, such as spreadsheets, completely obsolete before the data can be fully collected. This is especially true when each stage of the pipeline is managed by a separate tool that doesn’t share information in the same way or format as the other tools, creating a series of information silos that need to somehow be consolidated and deciphered. So, unless you have a central repository of information to dynamically track and report in near real-time the current state of every detail of every requirement, sprint, and scrum, your visibility is insufficient. The question then becomes, how can you manage what you can’t see?

  • Managing Dependencies & QualityIn addition to tracking requirements in a highly dynamic development environment, there are other things that also need to be tracked. For example, dependencies need to be tracked so as the code is updated, the dependent functions can also get the appropriate attention and testing they need to ensure quality. Quality and release readiness also need to be tracked and managed. Capacity tracking is essential to efficiently manage the available resources, which can include everything from man hours, people resources, and server availability.

  • Strategic ManagementBecause the reporting process is typically viewed as a drain on resources, reporting is typically limited only to those critical KPI’s that are most essential. That leaves higher level tracking and reporting on things like feature direction and progress on initiatives, which are spread out across a collection of epics and user stories, virtually non-existent.

  • Product DeliveryAs described above, if you can’t effectively track the progress of each requirement with its associated status and defects, and you have no visibility of where things are at in the pipeline, how do you know if your project is going to be delivered on time? How do you know if the small iterative progress that you’re seeing is sufficient to reach the project deadlines? And how can you determine if the product quality will meet specified standards?

  • Product PerformanceWhen you have a large complex product with multiple development teams that are each responsible for a small segment of the product, how do you know if you’re moving the performance needle in the right direction? Sure, this one feature change might help with the immediate performance, but does it help or hinder the other areas of the product and user experience?

The Culture of Control

IT Management feels completely in the dark with the application of these methodologies, requiring development management to find ways to provide sufficient project reporting and transparency to allow for effective managing. The issue of having autonomous and independent development teams makes tracking project progress a huge challenge. There is no single approval process for what or when things will be delivered. The iterative approach results in IT Management feeling like development is more of a black-box, leaving them with the only available response of “it’s done when it’s done.”

Contrast this with the command and control culture of the waterfall methodology, where every project comes with a scope study, project plan, project specifications, and Gant charts resulting in a well-defined linear progression and total control of the process. But, with these newer methodologies, nearly all of that control is gone… or at least it can feel like it.

Toolset Disparity

Having a product stream workflow that is nearly fully automated can be an amazing accomplishment. It takes the careful coordination of a variety of tools such as Remedy, ServiceNow, Jira, Pivotal, Selenium and many more. And while these tools may sufficiently integrate with each other to support the workflow, they don’t do well-sharing information with each other. This means that even with an end-to-end solution, there are still numerous information silos creating a lack of visibility at the portfolio level. This limits the ability to track the aggregate of the work in progress. IT Leadership and other stakeholders have only fragmented visibility along the pipeline. Dev teams get static information that is collected by spreadsheets or other legacy limiting their ability to dynamically react.

The disparity issue is further compounded with the inclusion of multiple product streams and an even larger number of autonomous and independent development teams creating multiple parallel workstreams. Each product and team, with their own set of development tools, further complicates the issue of coalescing a unified and current view of artifacts within the value stream. The end result is that with so many sources of isolated data, leaders and their teams are floundering in data overload from a series of different tools throughout the development pipeline that don’t necessarily tell the story in the same way.

Addressing the Needs

Plutora offers the most complete set of value stream management platform capabilities. It integrates with these disparate development chain tools, to manage both their respective functionalities and their information buckets, as though they were all one big unified solution. By collecting information from each tool across every product stream and dev team, Plutora is able to provide complete real-time visibility throughout the various workflows, providing the necessary information to effectively manage and know your actual current product health and progress at any given point in time.

Value Stream Management

Plutora addresses each of the challenges listed above by providing a centralized data hub, collecting information from each of the tools and coalescing the data into a single storyline. Here’s a sampling of how these benefits are realized by Plutora customers.“A combination of people, processes, and technology that maps, optimizes, visualizes, and governs business value flow (including epics, stories, and work items) through heterogeneous enterprise software delivery pipelines.”- Forrester

Release Planning & Orchestration

Plutora combines the release information from each of the different sprints, scrums, and product streams, providing a single source of truth. This unified source of information allows stakeholders to intelligently manage and make strategic decisions based on accurate and current data.

Plutora also enables teams to more effectively collaborate across geographic borders and time zones to ensure complete alignment on every aspect of every release. This enhanced collaboration capability simplifies discussion chains, making them more meaningful while minimizing the volume of email that typically inundates team members.

Deployment Coordination

The deployment workflow involves the planning, collaboration, and coordination of a variety of team members. Plutora simplifies this process, drastically reducing the time spent at each step. This means you and your team members will spend less time in conference rooms trying to coordinate and reporting status updates, and more time doing the work you were all hired to do. This increased efficiency also means that deployments go smoother, with fewer hiccups and faster reactions to issues. This all results in faster and easier deployments with a huge reduction in stress and the unknown.

Test Environment Management

Plutora organizes, manages, tracks and automates test environments, their components, scheduling, access, approvals and of their usage. Test Environment teams and management are able to plan, build, coordinate, schedule, and even automate the spin-up and spin-down of test environments to meet the needs of the product delivery teams. This solution also provides unprecedented transparency and visibility to all team members and stakeholders.

Analytics & Visualization

With a centralized data mart, Plutora tracks every workflow detail automatically from all in-house and integrated development pipeline toolsets, allowing for full and robust analytics and reporting on any aspect of the value stream. This provides end-to-end views ranging from the strategic high level to the individual progress and current state of a specific requirement. Visualization and pipeline transparency have never been this good.

Value Stream Mapping & Metrics

Because Plutora collects information and tracks artifacts throughout each of the in-house and integrated tools in the value stream, it provides a simple vision of clarity, showing the full value stream that has never been possible before. What used to require days and weeks to identify bottlenecks, now takes seconds. Simply pull up the value stream map and all the data and values are instantly updated showing your current state value stream map at any given time. This information also enables you to project future state value stream maps, as well as historical points in time.

Gone are the days of having to settle for KPI data simply because it was all that was available. Plutora aligns the data throughout the entire value stream, providing you with the metrics and KPI’s your team needs to ensure impactful and healthy team drivers are always available.

Governance & Compliance

Because Plutora automatically tracks and logs all value stream activities, showing who, what, where, when and how, historical logs are always accurate and bullet-proof. No more spreadsheets or other legacy solutions that require team members to record information that may or may be accurate. With Plutora, you know your historical data is accurate.

Workflows are also protected with Plutora workflow management that can be easily configured differently for each team and product stream to ensure that the appropriate checks, approvals, and gates are met. Nothing is left to chance, or allowed to fall through the cracks.

Integration & Dependency Management

Plutora has the ability to integrate with a wide variety of software development tools including Remedy, Selenium, Jira, GitHub, Chef, ExebiaLabs, ServiceNow and hundreds of others. This integration provides a single point login, allowing you to access the entire toolset functionality from within Plutora, providing you with a highly optimized workflow. This integration also allows you to automatically track and record activities throughout the toolsets minimizing the stop-and-record aspects that drain workflow efficiency.

Dependencies are also tracked by Plutora at multiple levels, ensuring that product code, testing, and test environment dependencies are recognized and addressed, and also that those inter-dependencies throughout the value stream toolsets are effectively managed.

Enterprise Agile Planning & Management

The planning and management of Agile development teams are drastically simplified with the increased visibility and manageability that Plutora delivers. Sprints and scrums can be planned and coordinated with associated dependencies, and then tracked throughout the lifecycle to ensure that every user story is addressed, tested and delivered just as it is supposed to. Plutora also simplifies the coordination of multiple development teams and their various product streams ensuring they all come together in a cohesive unified solution.


Whether your team uses one of these methodologies or something else, the key to improving both delivery velocity and product quality, with reliable repeatability, is through the automation of as much of the lifecycle as possible. The trick is to achieve this level of automation without giving up control and visibility. The best way to maintain control of every aspect of your projects with complete pipeline visibility is the Plutora value stream management solution.

Streamline your software delivery with Plutora!

Streamline your software delivery with Plutora!

Imagine a single dashboard managing all enterprise software delivery, boosting visibility, efficiency, and cutting costs. Experience Plutora's solutions today!

Imagine a single dashboard managing all enterprise software delivery, boosting visibility, efficiency, and cutting costs. Experience Plutora's solutions today!

Download our free eBook

Mastering Software Delivery with Value Stream Management

Discover the key to optimizing your software delivery process with our comprehensive eBook on Value Stream Management (VSM). Learn how leading organizations are streamlining their software pipelines, enhancing quality, and accelerating delivery.

Deliver Better Software Faster with Plutora

Deliver Better Software Faster with Plutora

Deliver Better Software Faster with Plutora