DevOps Zone is brought to you in partnership with:

Leaving university, I thought I'd be a developer happily knocking out code but was always drawn to tech support. How do we help people use our tools better? Now, I mostly specialize in consulting and introducing new customers to our tools. I am a Tech Evangelist and Lead Consultant at Urbancode. Eric is a DZone MVB and is not an employee of DZone and has posted 83 posts at DZone. You can read more from them at their website. View Full User Profile

Continuous Delivery’s Trouble with Web Services

  • submit to reddit

Typical Continuous Delivery, where builds are promoted along a pipeline or lifecycle is broken for complex architectures – especially web service based applications. I love these build promotion approaches and have cheered them on for six years, but they use a very imperfect model.

In the diagram below, we look at a typical build deployment pipeline including a build, a variety of testing environments, approvals, and eventual release to the live production environments. The three components (Services 1,2,3) are all set to use this model independently.

Three deployment pipelines running in parallel

There are several problems when using this model:

  1. The web services are interdependent. A change in an API for #1 may break #2 unless an update for it is also released.
  2. Because things are released together, the “sign-off” rarely applies to the release of a single version of a single component. Instead, a collection of items is approved.
  3. The pipeline always starts with a build. While appropriate for most web services, releases often include components that are not built: incremental static content moves, database updates or report migrations.
  4. The build pipeline is too rigidly ordered (a complaint I’ve made before).
Continuous Delivery book by Humble and Farley provides a few examples of this cleverness. We are instructed to version our database in such a way that updates to it can be applied repeatedly (like builds). This is sensible advice we fully agree with and even provide utilities for in our tooling.

The book also suggests an “Integration Pipeline” in Chapter 13. In this approach each major component has a short testing pipeline and is marked as good. A new build containing all the latest good components is created and moves through its own pipeline towards production release. The authors bend over backwards encouraging you to avoid this practice if you can.

I understand why all too well. The “build” that contains the latest good versions of each of the components is hard to work with. Sometimes we only want to release half the components. Sometimes, the “latest good” of one component is incompatible with the “latest good” of another and we would rather release a slightly older version of that component. Only with exceptional discipline can we  eliminate the numerous failure modes of assembling all the “latest good” and people are not good at discipline.

Using a build pipeline to model release orchestration and planning activities is simply a bad fit. It feels like wearing shoes a size too small. You can squeeze into them and they protect the soles of your feet, but expect pain.

What we want to do instead is to decouple our deployment pipeline or delivery chain from the concept of “a build”. Instead, we should focus on the idea of a Release Set or Snapshot which is a lightweight deployable unit for these larger systems.

Snapshots are similar to Integration Pipeline builds in that they reference versions of related components. But rather than retrieve and assemble those related components into a final application in as part of a super build, we instead, use a release set to logically collect the component versions. The Release Set or Snapshot is not “built” so much as “declared” and is analogous to the classic Release Manager’s spreadsheet that details the versions of each component that should be tested and released together. The Release Set also provides a natural granularity for managing approvals.

We also need to reject the notion that a Release Set will contain exactly one version of each component. For instance, in the Report migration example above, there might be a number of different new reports that have been independently developed and tested. If the component is naturally promoted in an incremental fashion, we include multiple, ordered versions in the Release Set.

Last week I worked with a team that was trying to figure out how to better update a complex CRM application. As an example, the team shared a recent release plan detailing updates to each of several dozen components. Modeling that in a build pipeline would be a nightmare. With release snapshots, we have a promising way forward.

I’ll be talking a lot more about the limitations of traditional CD, approaches for accomadating those limitations, and alternate models like Release Sets in our upcoming webcast, “Next Gen Continuous Delivery.” Please join us for that event.

Published at DZone with permission of Eric Minick, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)


Timo Lihtinen replied on Wed, 2012/03/14 - 11:35am

This is not the end of the story. What if you are in situation when say Service 1 is external consumer you have no control of? You end up supporting multiple versions in Prod with some roadmap for retiring old versions and rolling out new ones. I’d be happy to implement something like this with uDeploy. Extend uDeploy to SOA scenario.

Byron Ryder replied on Thu, 2013/07/25 - 3:51am

I prefer using Snapshot because it is easier to use for lightweight systems, when my cousin presented it to me I understood this was exactly what I was searching for. Even the guys from Contact One Call Center can approve that its easier to use release snapshots instead of build pipeline, it takes too much time and is not effective anymore.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.