Forget Chef or Puppet – Automate with Sprinkle

Miso has a relatively standard server architecture for a medium-level traffic Rails application. We use Linode to host our application and we provision a number of VPS instances that make up our infrastructure. We have a load balancer equipped with Nginx and Varnish, we have an application server that runs our Rails application to serve dynamic requests, we have a master-slave database setup, and a cache server that has memcached and redis.

Early on, we manually setup these servers by hand by installing packages, compiling libraries, tweaking configurations and installing gems. We then manually documented these steps to a company wiki which would allow us to mechanically follow the steps and setup a new instance of any of our servers.

Automating our Setup

We decided recently that we wanted a more robust way of provisioning and configuring our different types of servers. Mechanically following steps from a wiki is inefficient, error-prone and likely to become inaccurate. What we needed was a system that would allow us to provision a new application or database server with just a single command. Here were additional requirements for our desired setup:

  • Simple to setup and configure
  • Lightweight system using familiar syntax
  • Preferably utilizing ssh and commands in the vein of Capistrano
  • Modular components that can be assembled to setup each server
  • Locally executable such that I can provision a server from my local machine
  • Doesn’t require the target machine to have anything installed prior

The set-up and provisioning tools which are most popular for open-source seem to be Puppet and Chef. Both are great tools that have a lot of features and flexibility, and are well-suited when you have a large infrastructure with tens or hundreds of servers for a project. However, Chef and Puppet recipes are not trivial to create or manage and by default they are intended to be managed from a remote server. We felt that there was unnecessary complexity and overhead for our simple provisioning purposes. Thanks for the corrections in the comments regarding the remote server requirement for Puppet/Chef.

For smaller infrastructures like ours, we felt a better tool would be easier to manage and setup. Ideally, something that is akin to deploying our code with Capistrano. A tool that can be managed and run locally and that can be maintained easily. After some exploration, we stumbled upon a ruby tool called Sprinkle, probably best known by one example automated script called Passenger Stack.

There are several aspects of Sprinkle that made this our tool of choice. For one, it is locally managed and the setup is done leveraging the rock-solid Capistrano deployment system. Also, even though Sprinkle is written in Ruby, the tool does not require Ruby or anything else to be installed on the target servers since the automated setup is executed on your development machine and communicates with the remote server using only SSH. The best part is that there are only a few concepts required to understand and use this system.

Understanding Sprinkle

The rest of this article is intended to be a long and comprehensive overview of Sprinkle. While I have read many blog posts about Sprinkle written across several different years, I hadn’t seen a post that covered each aspect of Sprinkle in full detail. The goal is that by the end of this post, you should be able to understand and write Sprinkle scripts as well as execute them. Let’s start off by exploring the four major concepts that make up Sprinkle and that will be used to build out your server recipes: Packages, Policies, Installers, and Verifiers.

Packages

A package defines one or more things to provision onto the server. There is a lot of flexibility in a way a package is defined but fundamentally this represents a “component” can be installed. Packages are sets of installations, options, and verifications, grouped under a meaningful name. The basic structure of a package is like this:

# packages/some_name.rb
package :some_name do
  description 'Some text'
  version '1.2.3'
  requires :another_package

  # ...installers...

  verify { ...verifiers... }
end

Note that defining a package does not install the package by default. A package is only installed when explicitly mentioned in a policy. You can also specify recommendations and/or optional package dependencies in a package as well:

# packages/foo_name.rb
package :foo_name do
  requires :another_package
  recommends :some_package
  optional :other_package

  # ...installers...
  verify { ...verifiers... }
end

You can also create virtual aliased packages that are various alternatives for the same component type. For instance, if I wanted to give people a choice over the database to use when provisioning:

# packages/database.rb
package :sqlite3, :provides => :database do
  # ...installers and verifiers...
end

package :postgresql, :provides => :database do
  # ...installers and verifiers...
end

package :mysql, :provides => :database do
  # ...installers and verifiers...
end

You can now reference that you want to install a :database and the script will ask you which provision you want to install. For more information on packages, the best place is looking at the source file itself.

Policies

A policy defines a set of particular “packages” that are required for a certain server (app, database, etc). All policies defined will be run and all packages required by the policy will be installed. So whereas defining a “package” merely defines it, defining a “policy” actually causes those packages to be installed. A policy is very simple to define:

# Define target machine used for the "app" role
role :app, "208.28.38.44"
# Installs the packages specified with a 'requires' when the script executes
policy :myapp, :roles => :app do
  requires :some_package
  requires :nginx
  requires :postgresql
  requires :rails
end

A role merely defines what server the commands are run on. This way, a single Sprinkle script can provision an entire group of servers specifying different roles similar to using Capistrano directly. You may specify as many policies as you’d like. If the packages you’re requiring are properly defined with verification blocks, then no software will be installed twice, so you may require a webserver on multiple packages within the same role without having to wait for that package to install repeatedly.

For more information on policies, the best place is looking at the source file itself.

Installers

Installers are mechanisms for getting software onto the target machine. There are different scripts that allow you to download/compile software from a given source, use packaging systems (such as apt-get), copy files, install a gem, or even just inject content into existing files. Common examples of installers are detailed below.

To run an arbitrary command on the server:

package :foo do
  # Can be any line executed in the shell
  runner "run_some_command --now"
end

To install using the aptitude package manager in Ubuntu:

package :foo do
  apt 'foo-package'
  # Supports only installing dependencies
  apt('bar-package') { dependencies_only true }
end

That would install the ‘foo-package’ when you run the ‘foo’ package. To install a ruby gem:

package :foo do
  gem 'foo-gem'
  # Supports specifying version, source, repository, build_docs and build_flags
  gem 'bar-gem' do
    version "1.2.3"
    source 'http://gems.github.com'
    build_docs false
  end
end

To upload arbitrary text into a file on the target:

package :foo do
  push_text 'some random text', '/etc/foo/bar.conf'
  # Supports sudo access for a file
  push_text 'some random text', '/etc/foo/bar.conf', :sudo => true
end

You can also replace text in a target file:

# packages/foo.rb
package :foo do
  replace_text 'original foo', 'replacement bar', '/etc/foo/bar.conf'
  # Supports sudo access for a file
  replace_text 'original foo', 'replacement bar', '/etc/foo/bar.conf', :sudo => true
end

To transfer a file to a remote target file:

package :foo do
  # Transfers are recursive by default so whole directories can be moved
  transfer 'file/some_folder', '/etc/some_folder'
  # Supports sudo access for a file
  transfer 'file/foo.file', '/etc/foo.file', :sudo => true
  # Also a file can have "render" passed which runs the template through erb 
  # You can access variables to output the file dynamically, or pass explicit locals
  foo_port = 8080
  transfer 'file/foo.file', '/etc/foo.file', :render => true, 
            :locals => { :bar_port => 80 }
end

To run a rake task as part of a package on target:

package :foo, :rakefile => "/path/to/Rakefile" do
  rake 'foo-task'
end

To install a library from a given source path:

package :foo do
  source 'http://foo.com/latest-1.2.3.tar.gz'
  # Supports prefix, builds, archives, enable, with, and more
  source 'http://magicbeansland.com/latest-1.1.1.tar.gz' do
    prefix    '/usr/local'
    archives  '/tmp'
    builds    '/tmp/builds'
    with      'pgsql'
  end
end

Installers also support installation hooks at various points during the install process which vary depending on the installer used. An example of how to use hooks is as follows:

package :foo do
  apt 'foo-package' do
    pre :install, 'echo "Beginning install..."'
    post :install, 'echo "Completing install!"'
  end
end

Multiple hooks can be specified for any given step and each installer has multiple steps for which hooks can be configured. For more information on installers, the best place is looking at the source folder itself.

Verifiers

Verifiers are convenient helpers which you can use to check if something was installed correctly. You can use this helper within a verify block for a package. Sprinkle runs the verify block to find out whether or not something is already installed on the target machine. This way things never get done twice and if a package is already installed, then the task will be skipped.

Adding a verify block for every package is extremely important, be diligent to have an appropriate verifier for every installer used in a package. This will make the automated scripts much more robust and reusable on any number of servers. This also ensures that an installer works as expected and tests the server after installation as well.

There are many different types of verifications, for each one there are installers for which they are particularly useful. For instance, if I wanted to see if an aptitude package was installed correctly:

package :foo do
  apt 'foo-package'
  apt 'bar-package'
  verify do
    has_apt 'foo-package'
    has_apt 'bar-package'
  end
end

This will only install the package on a target if not already installed and verifies the installation after the package runs. If we wanted to check that a gem exists:

package :foo do
  gem 'foo-gem', :version => "1.2.3"
  gem 'bar-gem'
  verify do
    has_gem 'foo-gem', '1.2.3'
    # or verify that ruby can require a gem
    ruby_can_load 'bar-gem'
  end
end

If you want to check if a directory, file or executable exists:

package :foo do
  mkdir '/var/some/dir'
  touch 'var/some/file'
  runner 'touch /usr/bin/abinary' do
    post :install, "chmod +x /usr/bin/abinary"
  end

  verify do
    has_directory '/var/some/dir'
    has_file      '/etc/apache2/apache2.conf'
    has_executable 'abinary'
  end
end

You can also check if a process is running:

package :foo do
  apt 'memcached'

  verify do
    has_process 'memcached'
  end
end

For more information on verifiers, the best place is looking at the source folder itself.

Putting Everything Together

Once you understand the aforementioned concepts, building automated recipes for provisioning becomes quite straightforward. Simply define packages (with installers and verifiers) and then group them into ‘policies’ that run on target machines. Generally, you can have a deploy.rb file and an install.rb file that are defined as follows:

# deploy.rb
# SSH in as 'root'. Probably not the best idea.
set :user, 'root'
set :password, 'secret'

# Just run the commands since we are 'root'.
set :run_method, :run

# Be sure to fill in your server host name or IP.
role :app, '83.434.34.234'

default_run_options[:pty] = true

The install file tends to define the various policies for this sprinkle script:

# install.rb
require 'packages/essential'
require 'packages/git'
require 'packages/nginx'
require 'packages/rails'
require 'packages/mongodb'

policy :myapp, :roles => :app do
  requires :essential
  requires :git
  requires :nginx
  requires :rails
  requires :mongodb
end

deployment do
  delivery :capistrano

  source do
    prefix   '/usr/local'
    archives '/usr/local/sources'
    builds   '/usr/local/build'
  end
end

Then you should store your packages in a subfolder aptly named ‘packages’:

# packages/git.rb
package :git, :provides => :scm do
  description 'Git version control client'
  apt 'git-core'

  verify do
    has_executable 'git'
  end
end

You can store your assets (configuration files, etc) in “assets” folder and access them from your packages to upload. Once all the packages and policies have been defined appropriately you can execute the sprinkle script on the command line with:

sprinkle -c -s install.rb

And sprinkle is off to the races, setting up all the policies on the target machines.

Further Reading

There are several other good posts about Sprinkle:

In addition, there are a lot of good examples of sprinkle recipes:

Let me know what you think of all this in the comments and if you have any related comments or questions. What do you use to automate and manage your servers?

This entry was posted in All, Engineering. Follow any comments here with the RSS feed for this post. Post a comment or leave a trackback: Trackback URL.

Add a Comment

Your email is never published nor shared.

*
*

20 Comments

    1. nesquena says:

      Hey, that does look pretty simple and easy to use too! There were a ton of existing recipes for sprinkle and I like that it leverages capistrano directly but DO looks like a great alternative. I will have to try it for my next project.

  1. Allan Wind says:

    Chef does not require a server .

    1. nesquena says:

      Are you referring to chef-solo? http://wiki.opscode.com/display/chef/Chef+Solo Your right that you can get that working without a server. Thanks for the correction. We took a look at that but still opted to go with Sprinkle. Perhaps I should correct the post to reflect that the reason for Chef was more associated to the unnecessary complexity when dealing with simple automation needs.

  2. [...] Forget Chef or Puppet – Automate with Sprinkle | Miso Engineering – good walkthough of sprinkle and list of resources at the end of the article [...]

    1. baumpflege says:

      I have recently started a website, the info you provide on this site has helped me tremendously.
      Thank you for all of your time & work.

  3. Puppet doesn’t require a server either.

    1. nesquena says:

      Haha clearly a case of “Research Fail” on my part. The sad thing is I have used both Chef and Puppet in the past (both using a server to manage deploys) and I wasn’t aware they had a solo mode built in. I had heard of “chef-solo”. Anyhow, thanks for the correction.

    2. stahnma says:

      Puppet has always worked that way. Some very LARGE deployments use puppet without centralized servers, as do many small ones. There’s not really a ‘Puppet Solo’ mode, because it was designed that way from the start. Puppet will run locally, or in client/server mode.

      Seems like it would take less time to learn the current toolsets than to throw them away and write new ones.

      1. nesquena says:

        Thanks for the clarification. When I used puppet, we did it from a centralized server and I was under the (mistaken) impression that this was how most people were using it from talking with others who had been using it. Puppet is a great tool but as I explained in my article, there are cases for simple server provisioning that the lightweight capistrano+sprinkle approach is a better fit.

  4. samnang says:

    Good explanation post! I like it.

  5. I’d be remiss if I did not point out https://github.com/devstructure/blueprint, a decidedly different, simpler take on configuration management. It figures out what you did, freeing you to use standard `apt-get install`, `gem install`, vim/emacs, etc. to configure things.

    1. nesquena says:

      Very interesting gem, thanks for sharing. Impressive that it introspects the server to build the recipes automatically.

  6. Also, for completeness, Puppet, too, can run manifests in a standalone manner: `puppet apply foo.pp`

  7. [...] tornar a vida de desenvolvedores de software e administradores de sistemas mais fácil através de ideias e ferramentas realmente [...]

  8. cristianV says:

    Man thanks for this post, I have been searching a lot of documentation in this post is by far the clearest … thanks.

  9. Tammy Cravit says:

    Thanks for a great introduction to sprinkle! I’ve been setting up a lot of Linode VPS slices lately for a particular client, so I’m really appreciating the ability to automate them. I’ve published my sprinkle configuration at https://github.com/tammycravit/tayloredsw_vps in case it’s useful to anyone.

  10. Kraig Amott says:

    In regards to the time we are able to result in the payments, somebody moves the ends.
    An economist’s guess is liable to be as effective as anybody else’s.

  11. alcanal hth says:

    Hello just wanted to give you a quick heads up. The text in your article seem to be running off the screen in Opera. I’m not sure if this is a format issue or something to do with internet browser compatibility but I thought I’d post to let you know. The style and design look great though! Hope you get the problem fixed soon. Kudos

  12. Nice and informative overview. But why keep reinventing wheels?

    The stuff you describe with packages and verifiers solves a problem that has been solved by DEB and RPM a long time ago. With the extra benefit, that you don’t have to explicitly define the verifiers (except in special cases).

    Also, how would you manage the *entire* system with sprinkle? See my “Puppet and Chef do only half the job” article on http://theagileadmin.com/2012/07/03/puppet-and-chef-do-only-half-the-job/ for more thoughts about this topic.

  13. hello!,I love your writing so so much! proportion we

    keep up a correspondence extra about your post

    on AOL? I require an expert in this house to solve my

    problem. May be that’s you! Having a look forward to peer you.

  14. Greetings! Very helpful advice in this particular post! It is the little changes that will make the greatest changes. Many thanks for sharing!

2 Trackbacks

  1. [...] Forget Chef or Puppet – Automate with Sprinkle | Miso Engineering – good walkthough of sprinkle and list of resources at the end of the article [...]

  2. By O que é DevOps para mim | Felipe Munhoz on August 28, 2011 at 6:00 pm

    [...] tornar a vida de desenvolvedores de software e administradores de sistemas mais fácil através de ideias e ferramentas realmente [...]