Scalarium – Manage Your Amazon EC2 Clouds With Chef

The guys from peritor, who are the creators of webistrano, created an opscode chef based cloud management solution: Scalarium. Jonathan Weiss walked me through their solution, which helps to solve the issue of installing and dynamically configuring applications on a cluster of Amazon EC2 instances. In this post, I want to show you how they use chef cookbooks in a very special way to manage the lifecycle of your web application clusters.

Registering New Clients

When you start a new Amazon EC2 instance from within your Scalarium management console, you tell it in which role that new node should act. Scalrium has its own roles concept, similar to the native chef roles. The new node registers with the Scalarium platform as soon as it is booted and retrieves its chef node configuration. Scalarium uses chef-solo to apply all relevant recipes to install and configure the node. You may either use the predefined cookbook e.g. for setting up a Ruby on Rails stack or use your custom written chef cookbooks. When it is done, all other nodes in your Scalarium mangaged Amazon EC2 cluster receive a livecycle event so they can update their configurations to deal with the new node.

Scalarium Livecycle Events

Scalarium defines four livecycle events:

1. setup
2. configure
3. deploy
4. undeploy

Every event can define different chef run_lists. This breaks the somewhat monolitic approach of chef (do everything, everytime chef-client runs) into more granular parts. The heavy lifiting of installing an application stack is usually done on setup. If you add a new app node, the run_list of the configure event will just change your load balancers configuration to include it into the cluster. And deploying or undeploying your app can trigger even other things to be changed in the cluster. This seems to be a pretty clean way to apply chef recipes.

Scalarium Uses Chef-Solo

Scalarium uses only chef-solo without a chef server. Every node receives a unique node config file depending on its roles and the lifecycle event from the Scalarium platform and runs chef based on that. Additionally, it creates a JSON file containing the current node configuration living at /var/lib/scalarium/cluster_state.json. Scalarium dynamically configures your cluster whenever new nodes are added or old ones disappear by re-triggering the configure lifecycle event and sending down an updated cluster state.

I’ve not used Scalarium myself yet, but it seems to be a pretty decent cloud management platform comparable to RightScale and other competitors. If you already a chef wizard, you get a feature rich base which you can tailor with custom chef cookbooks. Have you already tried it out or are you using it on a daily basis? Let us know your experiences with it in the comments!