Transcription of MongoDB recipe particulars

This is my own approach to writing a Chef recipe for the installation and maintenance of MongoDB in a data center.

Node definitions

These are on the path nodes.

db01.json:

The replica nodes are idential except for name and normal::port values.

{
    "normal": { "port" : 27017, "rollup" : "replicaset-1" },
    "name": "db01",
    "override": { },
    "default": { },
    "json_class": "Chef::Node",
    "automatic": { },
    "run_list":
    [
        "recipe[apt]",
        "recipe[mongodb]",
        "recipe[mongodb::replica]",
        "role[install-database-node]",
        "role[install-replica-node]"
    ],
    "chef_type": "node"
}
db03.json:

This is a normal, replica node, but it also, arbitrarily rolls up the whole replica set by occasioning configuration done in the MongoDB shell in replicaset.rb, role config-replicaset.

{
    "normal": { "port" : 27019, "rollup" : "replicaset-1" },
    "name": "db03",
    "override": { },
    "default": { },
    "json_class": "Chef::Node",
    "automatic": { },
    "run_list":
    [
        "recipe[apt]",
        "recipe[mongodb]",
        "recipe[mongodb::replica]",
        "recipe[mongodb::replicaset]",
        "role[install-database-node]",
        "role[install-replica-node]",
        "role[config-replicaset]"
    ],
    "chef_type": "node"
}
db04.json:

This is the (single) configuration server.

{
    "normal": { "rollup" : "configsvr", "which" : "configsvr_1" },
    "name": "db04",
    "override": { },
    "default": { },
    "json_class": "Chef::Node",
    "automatic": { },
    "run_list":
    [
        "recipe[apt]",
        "recipe[mongodb]",
        "recipe[mongodb::configsvr]",
        "role[install-database-node]",
        "role[install-configsvr]"
    ],
    "chef_type": "node"
}
db05.json:

And here is the sharding router.

{
    "normal": { "rollup" : "shard-1" },
    "name": "db05",
    "override": { },
    "default": { },
    "json_class": "Chef::Node",
    "automatic": { },
    "run_list":
    [
        "recipe[apt]",
        "recipe[mongodb]",
        "recipe[mongodb::sharding]",
        "recipe[mongodb::add-shard]",
        "role[install-database-node]",
        "role[install-sharding-router]"
    ],
    "chef_type": "node"
}
db06.json:

Tentative: I haven't tested this yet nor the corresponding code in add-shard.rb.

Let's say we were going to have a second sharding router. Let's specify how to enable sharing and add a shard key.

{
    "normal":
    {
        "rollup" : "shard-2",
        "sharding" :
        {
            { "database" : <database-name> },
            { "collection" : <collection-name> },
            { "field1" : <field-name> },
            { "field2" : <field-name> },
            { "fieldn" : <field-name> )
        }
    },
    "name": "db05",
    "override": { },
    "default": { },
    "json_class": "Chef::Node",
    "automatic": { },
    "run_list":
    [
        "recipe[apt]",
        "recipe[mongodb]",
        "recipe[mongodb::sharding]",
        "recipe[mongodb::add-shard]",
        "role[install-database-node]",
        "role[install-sharding-router]"
    ],
    "chef_type": "node"
}

Role definitions

These are on the path roles.

install-database-node.rb:

This is underneath all MongoDB node types.

name "install-database-node"
description "Role for installing basic MongoDB"

for_database_servers = %w{
	recipe[apt]
	recipe[mongodb]
}

run_list for_database_servers
install-replica-node.rb:
name "install-replica-node"
description "Role for installing a MongoDB replica node"

for_database_servers = %w{
	recipe[apt]
	recipe[mongodb]
	recipe[mongodb::replica]
}

run_list for_database_servers
config-replicaset.rb:
name "config-replicaset"
description "Role for rolling up MongoDB replica sets"

for_database_servers = %w{
	recipe[apt]
	recipe[mongodb]
	recipe[mongodb::replica]
	recipe[mongodb::replicaset]
}

run_list for_database_servers
install-configsvr.rb:
name "install-configsvr"
description "Role for erecting a MongoDB configuration server"

for_database_servers = %w{
	recipe[apt]
	recipe[mongodb]
	recipe[mongodb::configsvr]
}

run_list for_database_servers
install-sharding-router.rb:
name "install-sharding-router"
description "Role for erecting a MongoDB sharding router"

for_database_servers = %w{
	recipe[apt]
	recipe[mongodb]
	recipe[mongodb::sharding]
	recipe[mongodb::add-shard]
}

run_list for_database_servers

Data bags

These are on the path data_bags/rollups.

replicaset-1.json:

Yeah, for additional replica sets, there are other data bags...

{
    "id"          : "replicaset-1",
    "name"        : "replica",
    "description" : "Shard 1 replica set",
    "replica_1"   : { "hostname" : "16.86.192.117", "port" : 27017, "node" : "db01" },
    "replica_2"   : { "hostname" : "16.86.192.118", "port" : 27018, "node" : "db02" },
    "replica_3"   : { "hostname" : "16.86.192.119", "port" : 27019, "node" : "db03", "primary" : true }
}
configsvr-1.json:

There are two other data bags for configuration server configuration.

{
    "id"          : "configsvr",
    "description" : "Configuration server",
    "configsvr_1" : { "hostname" : "16.86.192.120", "port" : 27017 }
}
shard-1.json:

For additional shards, there are more bags...

{
    "id"             : "shard-1",
    "description"    : "Shard 1",
    "port"           : 27017,
    "replicaset_bag" : "replicaset-1",
    "configsvr_bag"  : "configsvr"
}

Recipes

These are on the path cookbooks/mongodb.

Attributes

attributes/defaults.rb:
node.default[ :mongodb ][ :package ] = "mongodb-10gen"

Files

files/defaults/mongodb-upstart.conf:
# Ubuntu default upstart configuration calqued on /etc/init/mongodb.conf
description "Keeps MongoDB running between boots"

limit nofile 20000 20000
kill timeout 300         # wait 300s between SIGTERM and SIGKILL.

pre-start script
  exec start-stop-daemon --start --quiet --chuid mongodb --exec /usr/bin/mongod -- --config /data/mongodb/mongodb.conf
end script

script
  sleepWhileDaemonIsUp()
  {
    while pidof $1 > /dev/null; do
	  sleep 1
	done
  }

  sleepWhileDaemonIsUp /usr/bin/mongod
end script

post-stop script
  if [ pidof /usr/bin/mongod ]; then
    kill `pidof /usr/bin/mongod`
  fi
end script
files/defaults/sharding-upstart.conf:
# Ubuntu upstart configuration for sharding router file calqued on /etc/init/mongodb.conf
description "Keeps sharding router (mongos) running between boots"

limit nofile 20000 20000
kill timeout 300         # wait 300s between SIGTERM and SIGKILL.

pre-start script
  exec start-stop-daemon --start --quiet --chuid mongodb --exec /usr/bin/mongos -- --config /data/mongodb/mongodb.conf
end script

script
  sleepWhileDaemonIsUp()
  {
    while pidof $1 > /dev/null; do
	  sleep 1
	done
  }

  sleepWhileDaemonIsUp /usr/bin/mongos
end script

post-stop script
  if [ pidof /usr/bin/mongos ]; then
    kill `pidof /usr/bin/mongos`
  fi
end script

Templates

templates/defaults/replica.conf.erb:
port=<%= @port %>
replSet=<%= @replSet %>
dbpath=/data/mongodb
logpath=/data/mongodb/mongodb.log
fork=true
logappend=true
templates/defaults/configsvr.conf.erb:
port=<%= @port %>
configsvr=true
fork=true
dbpath=/data/mongodb
logpath=/data/mongodb/mongodb.log
logappend=true
templates/defaults/sharding.conf.erb:
port=<%= @port %>
fork=true
configdb=<%= @configsvr_list %>
logpath=/data/mongodb/mongodb.log
logappend=true

Recipes

recipes/replica.rb:
port         = node[ :port ]                 # e.g.: "replicaset-1"
bagname      = "id:" + node[ :rollup ]
bag          = search( :rollups, bagname ).first
replSet_name = bag[ :name ]

cookbook_file "/etc/init/mongodb.conf" do
  source "mongodb-upstart.conf"
  owner "root"
  group "root"
  mode 00644           # -rw-r--r--
end

service "mongodb" do
  supports :start => true, :stop => true, :restart => true, :status => true
  action :nothing
end

template "/data/mongodb/mongodb.conf" do
  source "replica.conf.erb"
  owner "mongodb"
  group "mongodb"
  mode 00644           # -rw-r--r--
  variables( {
    :port => port,
    :replSet => replSet_name
    }
  )
  notifies :restart, resources( :service => "mongodb" ), :immediately
end

service "mongodb" do
  provider Chef::Provider::Service::Upstart
  action [ :enable, :start ]
end
recipes/replicaset.rb:
bagname = "id:" + node[ :rollup ]            # e.g.: "replicaset-1"
bag     = search( :rollups, bagname ).first

id           = 0
found        = false
primary_port = nil
replica_id   = '_id : "%s"' % bag[ :name ]

replica_1 = bag[ :replica_1 ]
replica_2 = bag[ :replica_2 ]
replica_3 = bag[ :replica_3 ]
replica_4 = bag[ :replica_4 ]

replica_members = ""

if !replica_1.nil?
  found = true
  hostname = replica_1[ :hostname ]
  port     = replica_1[ :port ]
  replica_members += '{ _id : %d, host : "%s:%s" }' % [ id, hostname, port ]
  id += 1
  if primary_port.nil? and replica_1[ :primary ] == true
    primary_port = port
  end
end
if !replica_2.nil?
  if found
    replica_members += ", "
  end
  found = true
  hostname = replica_2[ :hostname ]
  port     = replica_2[ :port ]
  replica_members += '{ _id : %d, host : "%s:%s" }' % [ id, hostname, port ]
  id += 1
  if primary_port.nil? and replica_2[ :primary ] == true
    primary_port = port
  end
end
if !replica_3.nil?
  if found
    replica_members += ", "
  end
  found = true
  hostname = replica_3[ :hostname ]
  port     = replica_3[ :port ]
  replica_members += '{ _id : %d, host : "%s:%s" }' % [ id, hostname, port ]
  id += 1
  if primary_port.nil? and replica_3[ :primary ] == true
    primary_port = port
  end
end
if !replica_4.nil?
  if found
    replica_members += ", "
  end
  hostname = replica_4[ :hostname ]
  port     = replica_4[ :port ]
  replica_members += '{ _id : %d, host : "%s:%s" }' % [ id, hostname, port ]
  id += 1
  if primary_port.nil? and replica_4[ :primary ] == true
    primary_port = port
  end
end

configuration = "config = { " + replica_id + ", members: [" + replica_members + "] }"

mongo_command  = "mongo --port #{primary_port} --eval '#{configuration} ; rs.initiate( config )'"
execute "compose-replicaset-configuration" do
  command "#{mongo_command}"
  retries 6
  retry_delay 10
end
recipes/configsvr.rb:
bagname = "id:" + node[ :rollup ]            # e.g.: "configsvr"
bag     = search( :rollups, bagname ).first

which_server = node[ :which ]

configsvr = bag[ which_server.to_sym ]

hostname = configsvr[ :hostname ]
port     = configsvr[ :port ]

cookbook_file "/etc/init/mongodb.conf" do
  source "mongodb-upstart.conf"
  owner "root"
  group "root"
  mode 00644           # -rw-r--r--
end

template "/data/mongodb/mongodb.conf" do
  source "configsvr.conf.erb"
  owner "mongodb"
  group "mongodb"
  mode 00644           # -rw-r--r--
  variables( {
    :port => port,
    }
  )
end

service "mongodb" do
  provider Chef::Provider::Service::Upstart
  action [ :enable, :start ]
end
recipes/sharding.rb:
bagname  = "id:" + node[ :rollup ]           # e.g.: "shard-1"
shardbag = search( :rollups, bagname ).first

listening_port = shardbag[ :port ]

print "port="
pp listening_port

bagname = "id:" + shardbag[ :configsvr_bag ]
bag     = search( :rollups, bagname ).first

configsvr_list = nil
count          = 0
configsvr_1 = bag[ :configsvr_1 ]
configsvr_2 = bag[ :configsvr_2 ]
configsvr_3 = bag[ :configsvr_3 ]

if !configsvr_1.nil?
   count += 1;
   hostname = configsvr_1[ :hostname ]
   port     = configsvr_1[ :port ]
  if !hostname.empty? and !port.nil?
    configsvr_list = hostname + ":" + port.to_s
  end
end
if !configsvr_2.nil?
   count += 1;
   configsvr_2_hostname = configsvr_2[ :hostname ]
   configsvr_2_port     = configsvr_2[ :port ]
  if !hostname.empty? and !port.nil?
    if !configsvr_list.empty?
      configsvr_list += ", "
    end
    configsvr_list = hostname + ":" + port.to_s
  end
end
if !configsvr_3.nil?
   count += 1;
   configsvr_3_hostname = configsvr_3[ :hostname ]
   configsvr_3_port     = configsvr_3[ :port ]
  if !hostname.empty? and !port.nil?
    if !configsvr_list.empty?
      configsvr_list += ", "
    end
    configsvr_list = hostname + ":" + port.to_s
  end
end

if count != 1 or count != 3
  puts "Configuration server count is %d; we need 1 or 3" % count
end

service "mongodb" do
  supports :start => true, :stop => true, :restart => true, :status => true
  action :nothing
end

cookbook_file "/etc/init/mongodb.conf" do
  source "sharding-upstart.conf"
  owner "root"
  group "root"
  mode 00644           # -rw-r--r--
end

template "/data/mongodb/mongodb.conf" do
  source "sharding.conf.erb"
  owner "mongodb"
  group "mongodb"
  mode 00644           # -rw-r--r--
  variables( {
    :port => listening_port,
    :configsvr_list => configsvr_list
    }
  )
  notifies :restart, resources( :service => "mongodb" ), :immediately
end

service "mongodb" do
  provider Chef::Provider::Service::Upstart
  action [ :enable, :start ]
end
recipes/add-shard.rb:
bagname  = "id:" + node[ :rollup ]           # e.g.: "shard-1"
shardbag = search( :rollups, bagname ).first

bagname = "id:" + shardbag[ :replicaset_bag ]
bag     = search( :rollups, bagname ).first

replSet_name = bag[ :name ]
replica_1 = bag[ :replica_1 ]
replica_2 = bag[ :replica_2 ]
replica_3 = bag[ :replica_3 ]
replica_4 = bag[ :replica_4 ]

if !replica_1.nil? and replica_1[ :primary ] == true
  hostname = replica_1[ :hostname ]
  port     = replica_1[ :port ]
  puts "replica_1 is primary"
elsif !replica_2.nil? and replica_2[ :primary ] == true
  hostname = replica_2[ :hostname ]
  port     = replica_2[ :port ]
  puts "replica_2 is primary"
elsif !replica_3.nil? and replica_3[ :primary ] == true
  hostname = replica_3[ :hostname ]
  port     = replica_3[ :port ]
  puts "replica_3 is primary"
elsif !replica_4.nil? and replica_4[ :primary ] == true
  hostname = replica_4[ :hostname ]
  port     = replica_4[ :port ]
  puts "replica_4 is primary"
end

if hostname.nil? or hostname.empty? or port.nil?
  puts "Need hostname:port of one of the replica nodes to complete set-up"
end

mongo_command = "mongo --eval 'rs.addShard( \"#{replSet_name}/#{hostname}:#{port}\" )'"

execute "add-shard" do
  command "#{mongo_command}"
  retries 6
  retry_delay 10
end

# --------------------------------------------------------------------
# TODO: Unless there's only one shard, we need to enable sharding and
# set up the shard key(s) using the MongoDB shell. These commands are:
# $ mongo --port <number> (port number out of shard bag)
#
# sh.enableSharding( "database-name" )
# sh.shardOn( "database-name.collection", { "field-name" : 1, "_id" : 1 } )
#
# The particulars should be governed by tuples in the node definition.
# "normal" :
# {
#     "sharding" :
#     {
#         { "database" : <database-name> },
#         { "collection" : <collection-name> },
#         { "field1" : <field-name> },
#         { "field2" : <field-name> },
#         { "fieldn" : <field-name> )
#     }
# }
#
# --or something like this. The following code is untested:
# --------------------------------------------------------------------
if !shardbag[ :sharding ].nil?
  database_name  = shardbag[ :sharding ][ :database ]
  collection_name = shardbag[ :sharding ][ :collection ]

  enable_command = 'sh.enableSharding( "#{database_name}" )'
  mongo_command  = "#{launch_shell} '#{enable_command}'"
  execute "enable-shard" do
    command "#{mongo_command}"
    retries 6
    retry_delay 10
  end

  # Now compose the shardOn() command. We'll consider up to 4 fields.
  shardon_command = 'sh.shardOn( "#{database_name}.#{collection_name}", {'
  field1 = shardbag[ :sharding ][ :field1 ]
  field2 = shardbag[ :sharding ][ :field2 ]
  field3 = shardbag[ :sharding ][ :field3 ]
  field4 = shardbag[ :sharding ][ :field4 ]
  if !field1.nil?
    shardon_command += ' "#{field1}" : 1'
  end
  if !field2.nil?
    shardon_command += ' "#{field2}" : 1'
  end
  if !field3.nil?
    shardon_command += ' "#{field3}" : 1'
  end
  if !field4.nil?
    shardon_command += ' "#{field4}" : 1'
  end

  shardon_command += ' "_id" : 1 }'
  mongo_command  = "#{launch_shell} '#{shardon_command}'"
  execute "shardon" do
    command "#{mongo_command}"
    retries 6
    retry_delay 10
  end
end
recipes/add-arbiter.rb:
bagname  = "id:" + node[ :rollup ]           # e.g.: "replicaset-1"
bag      = search( :rollups, bagname ).first

hostname = bag[ :hostname ]
port     = bag[ :port ]

execute "add-arbiter" do
  command "mongo --eval 'rs.addArb( "#{hostname}":#{port}" )'"
  retries 6
  retry_delay 10
end

Attribute files

default.rb:
node.default[:mongodb][:package] = "mongodb-10gen"          # this says we install 10-gen's package

# -----------------------------------------------------------------
# We'll be using the host IP address to make node names for MongoDB
# replica set specification.
# -----------------------------------------------------------------
require 'socket'
iphash = Socket.ip_address_list.find {|a| a.ipv4? ? !(a.ipv4_private? || a.ipv4_loopback?) : !(a.ipv6_sitelocal? || a.ipv6_linklocal? || a.ipv6_loopback?) }
$ip_address = iphash.ip_address

node.default[:nodeaddress] = $ip_address
puts $ip_address
node.default[:mongodb][:package] = "mongodb-10gen"          # this says we install 10-gen's package

# -----------------------------------------------------------------
# We'll be using the host IP address to make node names for MongoDB
# replica set specification.
# -----------------------------------------------------------------
require 'socket'
iphash = Socket.ip_address_list.find {|a| a.ipv4? ? !(a.ipv4_private? || a.ipv4_loopback?) : !(a.ipv6_sitelocal? || a.ipv6_linklocal? || a.ipv6_loopback?) }
$ip_address = iphash.ip_address

node.default[:nodeaddress] = $ip_address
puts $ip_address

Auxiliary files

Gemfile:
source :rubygems
gem 'test-kitchen'
metadata.json:
{
  "name": "mongodb",
  "description": "Installs/Configures mongodb-10gen my style",
  "long_description": "",
  "maintainer": "",
  "maintainer_email": "",
  "license": "To kill",
  "platforms": { "ubuntu": ">= 12.04" },
  "dependencies":
  {
    "apt": ">= 0.0.0",
    "mongodb-10gen": ">= 2.4.5"
  },
  "recommendations": { },
  "suggestions": { },
  "conflicting": { },
  "providing": { },
  "replacing": { },
  "attributes": { },
  "groupings": { },
  "recipes": { },
  "version": "1.0.3"
}
metadata.rb:
name             "mongodb"
maintainer       ""
maintainer_email ""
license          "To kill"
description      "Installs/Configures mongodb-10gen"
long_description IO.read( File.join( File.dirname( __FILE__ ), 'README.md' ) )
version          "1.0.3"
depends          "apt"
depends          "mongodb-10gen"
supports         "ubuntu"
README.md:
Description
===========

Add apt repository and install mongodb-10gen.

## Platform

* ubuntu

### Tested on

* ubuntu 12.04(precise)

Requirements
============

- OpscodeCookbook[apt]


Attributes
==========

### group ['mongodb']

update your mongodb.conf values

### group ['mongodb']['config']

update your mongodb_config.conf values

### group ['mongodb']['router']

update your mongos.conf values

Usage
=====

### Available recipes

#### default

- Add 10gen official repository and install newer stable mongodb.
- **disable** autostart when install or serverboot.

#### replica

- set up /data/mongodb/mongodb.conf for any replica node

#### arbiter

- set up mongodb as arbiter for replica set.

#### configsvr

- set up mongodb configuration server.

#### sharding

- set up sharding router(mongos) node.


Additions
=========

Author
======

Author:: Russell Bateman (<[email protected]>)
Chef recipe filesystem layout
.
+-- attributes
|   `-- default.rb
+-- files
|   `-- default
|       +-- mongodb
|       |   +-- arbiter
|       |   |   `-- log
|       |   +-- bin
|       |   +-- configsvr
|       |   |   `-- log
|       |   `-- sharding
|       |       `-- log
|       +-- mongodb-upstart.conf
|       +-- multi-arbiter-upstart.conf
|       +-- multi-configsvr-upstart.conf
|       +-- multi-sharding-upstart.conf
|       +-- README.txt
|       `-- sharding-upstart.conf
+-- Gemfile
+-- metadata.json
+-- metadata.rb
+-- README.md
+-- recipes
|   +-- add-arbiter.rb
|   +-- add-shard.rb
|   +-- arbiter.rb
|   +-- configsvr.rb
|   +-- default.rb
|   +-- README.txt
|   +-- replica.rb
|   +-- replicaset.rb
|   +-- sharding.rb
|   +-- test-add-shard.rb
|   +-- test-configsvr.rb
|   +-- test-replicaset.rb
|   +-- test-run.rb
|   `-- test-sharding.rb
`-- templates
    `-- default
        +-- arbiter.conf.erb
        +-- configsvr.conf.erb
        +-- multi-arbiter.conf.erb
        +-- multi-configsvr.conf.erb
        +-- multi-sharding.conf.erb
        +-- replica.conf.erb
        `-- sharding.conf.erb

14 directories, 32 files





Node definitions
      db01.json
      db03.json
      db04.json
      db05.json
      db06.json
Roles definitions
      install-database-node.rb
      install-replica-node.rb
      config-replicaset.rb
      install-configsvr.rb
      install-sharding-router.rb
Data bags
      replicaset.json
      configsvr.json
      shard.json
Recipes
    attributes/
      defaults.rb
    files/default
      mongodb-upstart.conf
      sharding-upstart.conf
    templates/default
      replicaset.erb
      configsvr.conf.erb
      sharding.conf.erb
    recipes
      replica.rb
      replicaset.rb
      configsvr.rb
      sharding.rb
      add-shard.rb
      add-arbiter.rb
Attribute files
      default.rb
Auxiliary files
      Gemfile
      metadata-json
      metadata-rb
      README.md
      filesystem tree