DevOps Zone is brought to you in partnership with:

Dror Helper is an experienced software developer has written and designed software in various fields including video streaming, eCommerce, performance optimization and unit testing tools. He is passionate about programming best practices and all things software development, and has been a guest presenter at several user group meetings and ALT.NET events. Dror's blog can be found at http://blog.drorhelper.com where he writes about unit testing, agile methodologies, development tools, programming languages and anything else he finds interesting. Dror is a DZone MVB and is not an employee of DZone and has posted 53 posts at DZone. You can read more from them at their website. View Full User Profile

Top DZone Article of 2011: The 7 Levels of Continuous Integration

01.29.2012
| 21497 views |
  • submit to reddit

I’ve noticed that when other developers talk about “continuous integration” they do not always mean the same thing.

The following is an outline of the seven stages of continuous integration based solely on my own experience:

  1. No Server, no automation - no problems
  2. Have nightly build that run each day – at least we know that the project compiles
  3. Started writing “unit” tests – now running as part of the nightly build
  4. Created a build that runs on each commits – who broke the build?!
  5. Added static code analysis and style checking to the build
  6. Server Build, deploy & run the tests on several environments also used to deploy on testing (and production) environments\
  7. Everything is automated – Even Gui tests runs automatically on the latest code, you even got as far as testing the installer. Code is analyzed and results are sent by email. Nothing is done manually unless it is done only once…

These are mine stages as I experienced them in several jobs – what are yours?

 

Happy coding…

References
Published at DZone with permission of Dror Helper, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Neil Shannon replied on Fri, 2011/05/20 - 7:21am

We're at 5 right now.

Jordan Lee replied on Fri, 2011/05/20 - 8:40am

Sounds about right. Have never fully reached the 7th stage yet though.

Jon Davis replied on Fri, 2011/05/20 - 2:54pm

I'm typically a Level 1 SUPASTAH!! *jazz hands* Not that it's best, just sayin. I am. :-/

Balázs Bessenyei replied on Fri, 2011/05/20 - 4:33pm


Between 6 and 7. (The test FW and the environments is too unstable, so human verification is needed for the failed GUI tests.)

However we have a few additions. 

Level 5: newly introduced serious static code analysis errors or failed unit tests also causes build failure. Email based notification for the code comiters (and SCRUM Masters, just in case the responsible has left the building before successful build, thus violating the check in process). These checks are also extended to JS and configuration files as well, not just Java.

After successful Level 5 version control Labels are updated to point to the last successful build.
This is true for Level 6 as well, after the deploy have been verified to be successful and basic functionality is working.  

Patroklos Papapetrou replied on Sat, 2011/05/21 - 11:51am

We have adopted Continuous Integration a year ago. Now we are at stage 5.
We
have full automatied builds (trunk for every commit, nightly,
integration tests etc. ) , unit tests, database integration tests (
dbunit ),  static analysis ( quite easy that part with Sonar ) and now
we are trying to automate some gui tests using Selenium.
 You can add maybe between stage two and stage three the adoption of a developers guidelines to ensure tha continuous integration is part of the every-day development lifecycle

Edwin Quita replied on Sun, 2011/05/22 - 10:10pm

we are at level 6 where build, deploy, and tests are done on different environments (dev, uat, prod). we've combined a set of maven(checkstyle, findbugs, cobertura) plugins and jenkins(Publish Over SSH, Hudson Setenv Plugin, etc) plugins to do this.

David Updike replied on Thu, 2011/05/26 - 1:01pm

Dror, excellent topic. I was very energized by your list to come up with my own based on a lot of Agile projects over the past 7 years. My list has 9 stages. I have been part of teams that have floated between steps 2-8. I'm still looking for that 9!

9 Stages of Continuous Integration with each step building on the prior step.
1. No Server, no automation - no problems (ignorance is bliss)
2. Daily or Nightly builds – know that the project compiles
3. Builds include Automated Unit Tests (AUTs)
(AUTs run against only local/mock data)
4. Multiple Builds per Day with AUTs.
Stop-the-Line approach for failed builds.
5. Builds with AUTs on each commit.
6. Build on each Commit with AUTs, static code analysis, style checking, etc.
7. Build on each Commit with AUTs, static code analysis, style checking, etc. and select important Functional Tests (Functional Tests run against real external data)
8. Builds with AUTs, static code analysis, style checking, etc. and multiple Functional Unit Tests Suites to run at different times/environments on Commit, Nightly, Weekend, etc.
9. Everything is automated – GUI, Unit Tests, Acceptance Tests, Functional Tests, all deploys, etc. Nothing is done manually unless it is done only once.

Marko Taipale replied on Sun, 2011/05/29 - 1:47am

I think the list is missing the continuous deployment and testing the properties:

 

10: 9 + security, performance, scalability and failovers tested automatically (in different cadence)

11: 10 + continuous deployment (on commit / on feature push / daily).. optionally to the immune cluster

12: 11 + automated alerts and real time monitoring

13: 12 + automated rollback mechanism

 

Often the database and automated data migrations are not part of the CI, so that could also been taken into account as a variable in all the levels.

 

Eric Minick replied on Mon, 2011/05/30 - 7:31pm in response to: Marko Taipale

Back at the 2009 CITCON, we looked at the maturity as spanning several categories (build, deploy, test, report)- you could be more advanced on the build side than the deployment side - or deploying to production but be doing lots of manual testing.

 Jeffery Fredrick and I wrote that up and converted it into a wall-chart. There's a matching white paper too (reg required for the wp). 

 You might be starting too advanced even. Since the first challenge is integrating the code (before testing that integration and releasing it), I always include a nod to reasonable use of source control. It's amazing, but there are still a decent number of shops whose SCM practices and tools are pretty much non-existant. 

Emma Watson replied on Fri, 2012/03/30 - 6:12am

Shouldn't there be an eight level which takes the continuous delivery aspect into a account. This isn't possible for all applications, but still relevant where you have so much trust in your automation that you actually ships it automatically if it completes all steps in the build.

Swing

Carla Brian replied on Sun, 2012/05/27 - 9:41am

All of these articles are really helpful. Keep up the good work. - Dr Paul Perito

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.