Chef Rubyisms and Structure

Russell Bateman
14 August 2013
last update:

Notes on Ruby constructs employed by Chef recipes

The following are Ruby keywords or constructs and examples of them in Chef recipes I've seen.


Notes on Ruby-isms and the structure of Chef recipes

If you're new to Chef, but also to Ruby, and Ruby doesn't just flow from your fingertips, you're screwed. You won't find this stuff explained anyway, especially not in one place.

Our example is setting up so a recipe knows the environment it's being used in. We're going to use hostname rather than muck with the Chef web user interface which I find a bit squirrelly and, anyway, I want all this stuff to be done using knife and not the web UI.


Attributes

In the attributes subdirectory is, e.g.:, default.rb, a Ruby file with definitions such as

# Production environment:
default[ :platform ] = "ubuntu"

# Production environment:
default[ :environment ][ :hostname ] = ""

# Staging (QA) environment:
default[ :environment ][ :hostname ] = ""

# Developer sandbox environment:
default[ :environment ][ :hostname ] = "16.86.192.111"

This can be used by the recipe in recipes/default.rb. Chef appears to make all attributes from all attribute files in the attributes subdirectory available (i.e.: in effect) during processing of the recipe file. Well, at very least it makes recipes/default.rb available.

Inside the recipe file, default.rb, this would be used thus (:platform):

# Currently only tested with Debian...
if not node[ :platform ] == "ubuntu" then
  raise RuntimeError, "Unsupported platform: #{ node[ :platform ] }"
end

It's possible to add to or override attribute definitions from Chef role and node code. Imagine that you're setting up a MongoDB replica node and want to specify its port number, which will be the lone difference between this node and any other recipe nodes to be installed. I would have the choice (possibly) of doing this in one of several places, i.e.: normal, override and default. Here, I use normal.

    {
        "normal": { "mongodb" : { "replicanode" : { "port" : 37020 } } },
        "name": "uas-dev-db04",
        "override": { },
        "default": { },
        "json_class": "Chef::Node",
        "automatic": { },
        "run_list":
        [
            "recipe[apt]",
            "recipe[mongodb::replica]",
            "role[install_database_node]",
            "role[install_replica_node]"
        ],
        "chef_type": "node"
    }

Important note: In the node file, which is expressed in JSON, Ruby symbols (such as :replicanode and :port in the example above) cannot be expressed in Ruby syntax. They are done, as always in JSON, as strings, "replicanode" and "port". Never fear, however. By the time Ruby gets them, they've been automagically converted such that, later on during an .erb transformation in the MongoDB configuration file, the MongoDB replica node's port is specified thus:

    port=<%= node[ :mongodb ][ :replicanode ][ :port ] %>

Here'a slightly more complex normal attribute definition to bolster the previous example, which might not be enough to show the setting of three port attributes. This is a node file definition for a special, multipurpose database node that will furnish a) an arbiter, b) a single MongoDB configuration server and c) a sharding router (mongos).

    {
        "normal":
        {
            "mongodb" :
            {
                "arbiter" :   { "port" : 37016 },
                "configsvr" : { "port" : 47021 },
                "sharding" :  { "port" : 27017 }
            }
        },
        "name": "uas-dev-db05",
        "override": { },
        "default": { },
        "json_class": "Chef::Node",
        "automatic": { },
        "run_list":
        [
            "recipe[apt]",
            "recipe[mongodb]",
            "recipe[mongodb::arbiter]",
            "recipe[mongodb::configsvr]",
            "recipe[mongodb::sharding]",
            "role[install_database_node]",
            "role[install_arbiter]",
            "role[install_configsvr]",
            "role[install_sharding_router]"
        ],
        "chef_type": "node"
    }

With respect to attributes, there is a pecking order. What's defined in attribute files is easily overridden by a node or recipe file, which are overridden by the Chef environment, which is overridden by any role file. Attribute precedence is touched upon in the Opscode document on About Roles: Attribute Precedence. The table strength increases from left to right and top to bottom with automatic and Role as the strongest.


Templates

These are used to create all sorts of files such as configuration files, e.g.: Tomcat's server.xml, with specific settings. If a file is to be left as installed, usually, its contents would not be monkeyed with via a Chef template. Templates are typically done in (using) .erb files.

In the following file fragments, please notice what's in bold face which has relevance between each of the three files being shown.

  1. The recipe file that modifies the already installed server.xml. This is less than 10% of the file, but it's the part that serves as our example. It's the bit that invokes the actual modification of /usr/share/tomcat6/server.xml.
    tomcat6/recipes/default.rb:
        .
        .
        .
        # configure server.xml
        template "#{node[:installPath]}/apache-tomcat-6.0.37/conf/server.xml" do
          source "server.xml.erb"
          mode "0644"
        end
    
  2. The attributes file that lays down some definitions. The bits that are important to this example are in bold face. Look for these definitions in the template file coming up.
    tomcat6/attributes/default.rb:
        node.default[:installPath] = "/usr/share/tomcat6/"
        node.default[:tomcatPath] = "/var/lib/tomcat6/"
        node.default[:serverPort] = 8005
        node.default[:connectorPort] = 8080
        node.default[:ajpPort] = 8009
        node.default[:hostIP] = "16.86.192.114"
        node.default[:multicastIP] = "228.0.0.4"
        node.default[:multicastPort] = 45564
        node.default[:nioAddress] = "16.86.192.114"
        node.default[:nioPort] = 4000
        node.default[:tempDir] = "/var/lib/tomcat6/webapps/"
        node.default[:deployDir] = "/var/lib/tomcat6/webapps/"
        node.default[:watchDir] = "/tmp/war-listen/"
    
  3. Here is the .erb template file. It's basically a copy of what the Tomcat installer will lay down in the appropriate place, but with points of modification. These are in bold. Again, this isn't the whole enchilada which would easily lose the point being made.
    tomcat6/templates/default/server.xml.erb:
        .
        .
        .
        <Server port="<%= node[:serverPort] %>" shutdown="SHUTDOWN">
        .
        .
        .
            <Connector port="<%= node[:connectorPort] %>" protocol="HTTP/1.1"
                       connectionTimeout="20000"
                       redirectPort="8443" />
            .
            .
            .
            <!-- Define an AJP 1.3 Connector on port 8009 -->
            <Connector port="<%= node[:ajpPort] %>" protocol="AJP/1.3" redirectPort="8443" />
            .
            .
            .
            <!-- You should set jvmRoute to support load-balancing via AJP ie:
            <Engine name="Catalina" defaultHost="<%= node[:hostIP] %>">
            .
            .
            .
                <Host name="<%= node[:hostIP] %>"  appBase="webapps"
                      unpackWARs="true" autoDeploy="true">
                .
                .
                .
                <Channel className="org.apache.catalina.tribes.group.GroupChannel">
                    <Membership className="org.apache.catalina.tribes.membership.McastService"
                                address="<%= node[:multicastIP] %>"
                                port="<%= node[:multicastPort] %>"
                                frequency="500"
                                dropTime="3000"/>
                  <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                              address="<%= node[:nioAddress] %>"
                              port="<%= node[:nioPort] %>"
                              autoBind="100"
                              selectorTimeout="5000"
                              maxThreads="6"/>
                  .
                  .
                  .
                  <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                            tempDir="<%= node[:tempDir] %>"
                            deployDir="<%= node[:deployDir] %>"
                            watchDir="<%= node[:watchDir] %>"
                            watchEnabled="true"/>
    

Copying remote files

Here's a random topic. I couldn't figure out how to use Chef recipe constructs file, remote_file, etc. to copy a file from one part of a (server) node's filesystem to another. Ultimately, I resorted to the following, whose comment explains everything.

    # Create a script on the fly to copy the mongod daemon for our configuration
    # server to run off of. (The latter cannot share /usr/bin/mongod because the
    # arbiter is going to be using that one.)
    bash "copy-mongod" do
      user "root"
      cwd "/data/mongodb"
      code <<-EOS
      cp /usr/bin/mongod ./bin
      chown mongodb:mongodb ./bin/mongod
      chmod a+x ./bin/mongod
      EOS
    end