DevOps Zone is brought to you in partnership with:

Ranjib is a system administrator at Google. Prior to Google, Ranjib was a senior consultant with ThoughtWorks. He works on private cloud implementation strategies, cloud adoption, system automation etc. He has worked on both application development as well as system administration, for past 6 years. Prior to ThoughtWorks, Ranjib was working with Persistent Systems . Ranjib has done his gradation in lifescience and masters in Bioinformatics. Ranjib is a staunch FOSS supporter. Ranjib is a DZone MVB and is not an employee of DZone and has posted 13 posts at DZone. You can read more from them at their website. View Full User Profile

Evolving Enterprise Infrastructure Using Chef

08.26.2012
| 7777 views |
  • submit to reddit

People consider Chef as a configuration management tool. You specify the state using the infrastructure DSL that Chef provides. You just apply yoru configurations, and you get reproducible infrastructure. Here are the main topics:

  1. Chef is a modular infrastructure management toolkit. There are several way to look at chef: At its core chef is an API layer over day to day system administration. All other Chef components leverage this API.
  2. chef-client and chef-solo are two binaries that execute Chef scripts in client/server or serverless chef setups.
  3. Knife is a command line interface to chef-managed infrastructure.

All these things together give you traditional configuration management style workflows via a Chef managed infrastructure. 

Now, there are ample one-off tasks that happen only once in a server's life time. Incident management, audits are another example. There are configurations related to third party hosted solutions.  Enterprise's own service deployments.  Most of these workflows don't model well in conventional configuration management style infrastructure management.

For such workflows chef can be easily extended. Chef provides multiple avenues to extend its capabilities:

  •  Chef internally distributed as a set of different libraries.  


    At its core chef has Mixlib::ShellOut , which is a separate rubygem. This is the component that provides portable command execution. Since 90% of the time configuration management systems dispatch command, this is the component that can be called the heart of Chef. If you have lots of shell-script based tools you can easily port them with mixlib-shellout. I have rarely seen bash scripts that check the return value of every command it invokes. It's more difficult to maintain long shell scripts. With mixlin-shellout its much easy to express safe guard measures.

  • Chef uses mixlib-json to convert JSON data. This is also a separate gem, and you can independently use it. You might be wondering why this is important. JSON is the principle marshalling solution in chef and most other modern systems. mixlib-json  facilitates the inclusion of a json conversion library from the available ones. You can use mixlib-json independently to grab raw data and serialize them and feed into other programs in a relatively portable way.

  • mixlib-cli and mixlib-config . Mixlib::CLI lets you easily parse command line arguments. While mixlib-config lets you read, access override configuration values. Again both of them are distributed as separate gems. You can use these two and build your custom command line tool chains rapidly.

  • knife-plugins. Knife can be extended using plugins. It's a more formal way of extending knife via sub-commands. The best part of Chef is that it's pure ruby everywhere, so you can pretty much use any rubygem (fog, aws-sdk, openvz) to have any cloud based infrastructure automation tool. There are many cases where your configurations are not applied against a server (linux or windows etc). They can be hosted solutions. Like IAM entries inside AWS, or ultraDNS based resource records, pagerduty escalation policies, salesforce settings, campfire notifications etc.  The knife-plugin ecosystem along with the other libraries provide an elegant way to develop such tooling.



You can use an organization that can start automating smaller chunks of work and then refactor them into portable recipes so that a conventions-configuration management style infrastrcuture is possible. Chef's philosophy emphasizes resources as the primary building block and idempotency is a desired criteria. It is desired because there is an implicit assumption of repetetive invocation of chef runs within a system. Once you have your infrastructure to this level, there is lot magic that can be leveraged from chef, and also a lot other tools that work on top of chef can be adopted.


  • Chef is also available as a single monolithic operating system package. Its called omnibus installers. They are present for most of the popular operating systems. For instance, the linux installer bundles everything on top of glibc. So you have an isolated, battle-tested working Chef binary that you can introduce to your system without inflicting major drifts.

This is crucial, as its lets you minimize iterative migration of manual or semi-automated infrastructure under full configuration management. Or, if not traditional configuration management, still reproducible and automatic infrastructure.  Start with only the omnibus installer and then iteratively automate the most repetitive and time consuming tasks. Whatever it is.

  • Once you have adopted chef significantly you'll realize the volume of chef-scripts is large. This is a by-product of treating  large infrastructure as code. Code needs maintenance. And the maintenance can be easy if appropriate software development practices are applied. Having tests, keeping your chef scipts under a CI adds tremendous value. But the entire paradigm depends upon how easy it is to write tests against chef script. Last year has seen tremendous efforts on this aspect. Now we have decently effective unit testing, functional testing, and integration testing frameworks (chefspec, cucumber-chef, minitest-handler are examples respectively). Chef allows you to treat infrastructure as any other functional domain, like mobile or banking or health care. Any developers can work with domain experts to develop neat chef scripts (an operations guy is an example of a domain expert).

Together these things makes it much easier for an enterprise to iteratively improve and evolve its infrastructure to accommodate newer innovations (like cloud adoption, automatic failovers, disposable servers) with greater agility using Chef....

Published at DZone with permission of Ranjib Dey, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)