Agile Zone is brought to you in partnership with:

Julian writes by night as The Build Doctor, a poorly disguised non de plume. By day he is a freelance build engineer, systems administrator and all round useful guy. Julian lives near London with his fiance and children. He enjoys balancing a love of real ale and Indian food with cycling. Julian has posted 16 posts at DZone. View Full User Profile

A way to cool dependency Hell?

02.19.2010
| 7073 views |
  • submit to reddit

How to break a deploy: Take one codebase. Sieve in a new class. Mix in the dry ingredients and a new runtime dependency. Place another dependency on a pre-warmed Hudson, bake for 10 minutes (on a medium heat) and then deploy. Oh dear. It didn’t deploy.

We’re a bit crap about managing the external dependencies of our code. I’m not talking about libraries but more basic dependencies that your application might have, like native code libraries, or commands. There’s two ways you can do this:

  • You can make people responsible for the care and feeding of your testing and production environments. This is easy to implement, but stupid. I think it would only work in an environment with exceptional communication.
  • Or you can insist that any application must declare what it depends on.

Keeping environments up to date keeps lots of people in a job. It’s a really dumb job. At my day job we’ve taken the latter route, using Puppet.

Puppet is a tool for systems administration. But you can use it even if you don’t know fsck from fmt. The way we’re using it is to be an executable specification of the dependencies that your application needs. For example: A test just failed on a new server – with this output:


Validation failed: Avatar /tmp/stream20091208-22414-y3anvf-0 is not recognized by the 'identify' command.

I realised that it needed thelibmagic1 package and possibly libmagic-dev. I could have installed them then and there onto the machine. I’d have forgotten about it in the excitement. So I added them to a file on that project called dependencies.rb. This file is run by Puppet before we deploy. It gives our developers enough control over the target operating systems so that they can make small changes to the deployment environments. We’ve been running this via a Capistrano task on our project; we typically run it as the deployment user; that way we can easily make changes to crontabs. Puppet won’t exit cleanly if it can’t install all the dependencies, so it’s a good way to test.

Here’s an abridged version of our dependencies file:

   1. class dependencies {   
2.
3. include $operatingsystem
4.
5. class ubuntu {
6. package {
7. 'libcurl3': ensure => present;
8. 'libcurl3-gnutls': ensure => present;
9. 'libcurl4-openssl-dev': ensure => present;
10. 'g++': ensure => present;
11. 'build-essential': ensure => present;
12. 'libmysqlclient15-dev': ensure => present;
13. 'libxml2-dev': ensure => present;
14. 'libxslt1-dev': ensure => present;
15. 'libmagic1': ensure => present;
16. 'libmagic-dev': ensure => present;
17.
18. }
19. }
20.
21. class gentoo {
22. cron {
23. 'some cron job':
24. command => '/engineyard/bin/command blah blah',
25. user => 'admin',
26. hour => ['*/2'],
27. minute => ['12'],
28. ensure => 'present';
29. }
30. }
31.
32. class darwin {
33. notice("Nothing to do in this OS.")
34. }
35. }
36. node default {
37. include dependencies
38. }

In this file we define a class (dependencies), which doesn’t do much but look for a corresponding inner class to match your operating system. Right now we have a very simple arrangement: The dependencies::gentoo class contains crontabs and the like for EngineYard. The dependencies::ubuntu class names all the native dependencies of our rubygems. We have an empty class for Darwin to stop the Mac OS X machines from complaining. That’s it. Here’s the Capistrano task:

  desc "Run Puppet to make sure that dependencies are met"
task :dependencies, roles => :app, :except => {:no_release => true} do
run "cd #{release_path} && rake dependencies"
end

Image courtesy of eflon.

References
Published at DZone with permission of its author, Julian Simpson. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)