This is my own approach to writing a Chef recipe for the installation and maintenance of MongoDB in a data center.
These are on the path nodes.
The replica nodes are idential except for name and normal::port values.
{
"normal": { "port" : 27017, "rollup" : "replicaset-1" },
"name": "db01",
"override": { },
"default": { },
"json_class": "Chef::Node",
"automatic": { },
"run_list":
[
"recipe[apt]",
"recipe[mongodb]",
"recipe[mongodb::replica]",
"role[install-database-node]",
"role[install-replica-node]"
],
"chef_type": "node"
}
This is a normal, replica node, but it also, arbitrarily rolls up the whole replica set by occasioning configuration done in the MongoDB shell in replicaset.rb, role config-replicaset.
{
"normal": { "port" : 27019, "rollup" : "replicaset-1" },
"name": "db03",
"override": { },
"default": { },
"json_class": "Chef::Node",
"automatic": { },
"run_list":
[
"recipe[apt]",
"recipe[mongodb]",
"recipe[mongodb::replica]",
"recipe[mongodb::replicaset]",
"role[install-database-node]",
"role[install-replica-node]",
"role[config-replicaset]"
],
"chef_type": "node"
}
This is the (single) configuration server.
{
"normal": { "rollup" : "configsvr", "which" : "configsvr_1" },
"name": "db04",
"override": { },
"default": { },
"json_class": "Chef::Node",
"automatic": { },
"run_list":
[
"recipe[apt]",
"recipe[mongodb]",
"recipe[mongodb::configsvr]",
"role[install-database-node]",
"role[install-configsvr]"
],
"chef_type": "node"
}
And here is the sharding router.
{
"normal": { "rollup" : "shard-1" },
"name": "db05",
"override": { },
"default": { },
"json_class": "Chef::Node",
"automatic": { },
"run_list":
[
"recipe[apt]",
"recipe[mongodb]",
"recipe[mongodb::sharding]",
"recipe[mongodb::add-shard]",
"role[install-database-node]",
"role[install-sharding-router]"
],
"chef_type": "node"
}
Tentative: I haven't tested this yet nor the corresponding code in add-shard.rb.
Let's say we were going to have a second sharding router. Let's specify how to enable sharing and add a shard key.
{
"normal":
{
"rollup" : "shard-2",
"sharding" :
{
{ "database" : <database-name> },
{ "collection" : <collection-name> },
{ "field1" : <field-name> },
{ "field2" : <field-name> },
{ "fieldn" : <field-name> )
}
},
"name": "db05",
"override": { },
"default": { },
"json_class": "Chef::Node",
"automatic": { },
"run_list":
[
"recipe[apt]",
"recipe[mongodb]",
"recipe[mongodb::sharding]",
"recipe[mongodb::add-shard]",
"role[install-database-node]",
"role[install-sharding-router]"
],
"chef_type": "node"
}
These are on the path roles.
This is underneath all MongoDB node types.
name "install-database-node"
description "Role for installing basic MongoDB"
for_database_servers = %w{
recipe[apt]
recipe[mongodb]
}
run_list for_database_servers
name "install-replica-node"
description "Role for installing a MongoDB replica node"
for_database_servers = %w{
recipe[apt]
recipe[mongodb]
recipe[mongodb::replica]
}
run_list for_database_servers
name "config-replicaset"
description "Role for rolling up MongoDB replica sets"
for_database_servers = %w{
recipe[apt]
recipe[mongodb]
recipe[mongodb::replica]
recipe[mongodb::replicaset]
}
run_list for_database_servers
name "install-configsvr"
description "Role for erecting a MongoDB configuration server"
for_database_servers = %w{
recipe[apt]
recipe[mongodb]
recipe[mongodb::configsvr]
}
run_list for_database_servers
name "install-sharding-router"
description "Role for erecting a MongoDB sharding router"
for_database_servers = %w{
recipe[apt]
recipe[mongodb]
recipe[mongodb::sharding]
recipe[mongodb::add-shard]
}
run_list for_database_servers
These are on the path data_bags/rollups.
Yeah, for additional replica sets, there are other data bags...
{
"id" : "replicaset-1",
"name" : "replica",
"description" : "Shard 1 replica set",
"replica_1" : { "hostname" : "16.86.192.117", "port" : 27017, "node" : "db01" },
"replica_2" : { "hostname" : "16.86.192.118", "port" : 27018, "node" : "db02" },
"replica_3" : { "hostname" : "16.86.192.119", "port" : 27019, "node" : "db03", "primary" : true }
}
There are two other data bags for configuration server configuration.
{
"id" : "configsvr",
"description" : "Configuration server",
"configsvr_1" : { "hostname" : "16.86.192.120", "port" : 27017 }
}
For additional shards, there are more bags...
{
"id" : "shard-1",
"description" : "Shard 1",
"port" : 27017,
"replicaset_bag" : "replicaset-1",
"configsvr_bag" : "configsvr"
}
These are on the path cookbooks/mongodb.
node.default[ :mongodb ][ :package ] = "mongodb-10gen"
# Ubuntu default upstart configuration calqued on /etc/init/mongodb.conf
description "Keeps MongoDB running between boots"
limit nofile 20000 20000
kill timeout 300 # wait 300s between SIGTERM and SIGKILL.
pre-start script
exec start-stop-daemon --start --quiet --chuid mongodb --exec /usr/bin/mongod -- --config /data/mongodb/mongodb.conf
end script
script
sleepWhileDaemonIsUp()
{
while pidof $1 > /dev/null; do
sleep 1
done
}
sleepWhileDaemonIsUp /usr/bin/mongod
end script
post-stop script
if [ pidof /usr/bin/mongod ]; then
kill `pidof /usr/bin/mongod`
fi
end script
# Ubuntu upstart configuration for sharding router file calqued on /etc/init/mongodb.conf
description "Keeps sharding router (mongos) running between boots"
limit nofile 20000 20000
kill timeout 300 # wait 300s between SIGTERM and SIGKILL.
pre-start script
exec start-stop-daemon --start --quiet --chuid mongodb --exec /usr/bin/mongos -- --config /data/mongodb/mongodb.conf
end script
script
sleepWhileDaemonIsUp()
{
while pidof $1 > /dev/null; do
sleep 1
done
}
sleepWhileDaemonIsUp /usr/bin/mongos
end script
post-stop script
if [ pidof /usr/bin/mongos ]; then
kill `pidof /usr/bin/mongos`
fi
end script
port=<%= @port %> replSet=<%= @replSet %> dbpath=/data/mongodb logpath=/data/mongodb/mongodb.log fork=true logappend=true
port=<%= @port %> configsvr=true fork=true dbpath=/data/mongodb logpath=/data/mongodb/mongodb.log logappend=true
port=<%= @port %> fork=true configdb=<%= @configsvr_list %> logpath=/data/mongodb/mongodb.log logappend=true
port = node[ :port ] # e.g.: "replicaset-1"
bagname = "id:" + node[ :rollup ]
bag = search( :rollups, bagname ).first
replSet_name = bag[ :name ]
cookbook_file "/etc/init/mongodb.conf" do
source "mongodb-upstart.conf"
owner "root"
group "root"
mode 00644 # -rw-r--r--
end
service "mongodb" do
supports :start => true, :stop => true, :restart => true, :status => true
action :nothing
end
template "/data/mongodb/mongodb.conf" do
source "replica.conf.erb"
owner "mongodb"
group "mongodb"
mode 00644 # -rw-r--r--
variables( {
:port => port,
:replSet => replSet_name
}
)
notifies :restart, resources( :service => "mongodb" ), :immediately
end
service "mongodb" do
provider Chef::Provider::Service::Upstart
action [ :enable, :start ]
end
bagname = "id:" + node[ :rollup ] # e.g.: "replicaset-1"
bag = search( :rollups, bagname ).first
id = 0
found = false
primary_port = nil
replica_id = '_id : "%s"' % bag[ :name ]
replica_1 = bag[ :replica_1 ]
replica_2 = bag[ :replica_2 ]
replica_3 = bag[ :replica_3 ]
replica_4 = bag[ :replica_4 ]
replica_members = ""
if !replica_1.nil?
found = true
hostname = replica_1[ :hostname ]
port = replica_1[ :port ]
replica_members += '{ _id : %d, host : "%s:%s" }' % [ id, hostname, port ]
id += 1
if primary_port.nil? and replica_1[ :primary ] == true
primary_port = port
end
end
if !replica_2.nil?
if found
replica_members += ", "
end
found = true
hostname = replica_2[ :hostname ]
port = replica_2[ :port ]
replica_members += '{ _id : %d, host : "%s:%s" }' % [ id, hostname, port ]
id += 1
if primary_port.nil? and replica_2[ :primary ] == true
primary_port = port
end
end
if !replica_3.nil?
if found
replica_members += ", "
end
found = true
hostname = replica_3[ :hostname ]
port = replica_3[ :port ]
replica_members += '{ _id : %d, host : "%s:%s" }' % [ id, hostname, port ]
id += 1
if primary_port.nil? and replica_3[ :primary ] == true
primary_port = port
end
end
if !replica_4.nil?
if found
replica_members += ", "
end
hostname = replica_4[ :hostname ]
port = replica_4[ :port ]
replica_members += '{ _id : %d, host : "%s:%s" }' % [ id, hostname, port ]
id += 1
if primary_port.nil? and replica_4[ :primary ] == true
primary_port = port
end
end
configuration = "config = { " + replica_id + ", members: [" + replica_members + "] }"
mongo_command = "mongo --port #{primary_port} --eval '#{configuration} ; rs.initiate( config )'"
execute "compose-replicaset-configuration" do
command "#{mongo_command}"
retries 6
retry_delay 10
end
bagname = "id:" + node[ :rollup ] # e.g.: "configsvr"
bag = search( :rollups, bagname ).first
which_server = node[ :which ]
configsvr = bag[ which_server.to_sym ]
hostname = configsvr[ :hostname ]
port = configsvr[ :port ]
cookbook_file "/etc/init/mongodb.conf" do
source "mongodb-upstart.conf"
owner "root"
group "root"
mode 00644 # -rw-r--r--
end
template "/data/mongodb/mongodb.conf" do
source "configsvr.conf.erb"
owner "mongodb"
group "mongodb"
mode 00644 # -rw-r--r--
variables( {
:port => port,
}
)
end
service "mongodb" do
provider Chef::Provider::Service::Upstart
action [ :enable, :start ]
end
bagname = "id:" + node[ :rollup ] # e.g.: "shard-1"
shardbag = search( :rollups, bagname ).first
listening_port = shardbag[ :port ]
print "port="
pp listening_port
bagname = "id:" + shardbag[ :configsvr_bag ]
bag = search( :rollups, bagname ).first
configsvr_list = nil
count = 0
configsvr_1 = bag[ :configsvr_1 ]
configsvr_2 = bag[ :configsvr_2 ]
configsvr_3 = bag[ :configsvr_3 ]
if !configsvr_1.nil?
count += 1;
hostname = configsvr_1[ :hostname ]
port = configsvr_1[ :port ]
if !hostname.empty? and !port.nil?
configsvr_list = hostname + ":" + port.to_s
end
end
if !configsvr_2.nil?
count += 1;
configsvr_2_hostname = configsvr_2[ :hostname ]
configsvr_2_port = configsvr_2[ :port ]
if !hostname.empty? and !port.nil?
if !configsvr_list.empty?
configsvr_list += ", "
end
configsvr_list = hostname + ":" + port.to_s
end
end
if !configsvr_3.nil?
count += 1;
configsvr_3_hostname = configsvr_3[ :hostname ]
configsvr_3_port = configsvr_3[ :port ]
if !hostname.empty? and !port.nil?
if !configsvr_list.empty?
configsvr_list += ", "
end
configsvr_list = hostname + ":" + port.to_s
end
end
if count != 1 or count != 3
puts "Configuration server count is %d; we need 1 or 3" % count
end
service "mongodb" do
supports :start => true, :stop => true, :restart => true, :status => true
action :nothing
end
cookbook_file "/etc/init/mongodb.conf" do
source "sharding-upstart.conf"
owner "root"
group "root"
mode 00644 # -rw-r--r--
end
template "/data/mongodb/mongodb.conf" do
source "sharding.conf.erb"
owner "mongodb"
group "mongodb"
mode 00644 # -rw-r--r--
variables( {
:port => listening_port,
:configsvr_list => configsvr_list
}
)
notifies :restart, resources( :service => "mongodb" ), :immediately
end
service "mongodb" do
provider Chef::Provider::Service::Upstart
action [ :enable, :start ]
end
bagname = "id:" + node[ :rollup ] # e.g.: "shard-1"
shardbag = search( :rollups, bagname ).first
bagname = "id:" + shardbag[ :replicaset_bag ]
bag = search( :rollups, bagname ).first
replSet_name = bag[ :name ]
replica_1 = bag[ :replica_1 ]
replica_2 = bag[ :replica_2 ]
replica_3 = bag[ :replica_3 ]
replica_4 = bag[ :replica_4 ]
if !replica_1.nil? and replica_1[ :primary ] == true
hostname = replica_1[ :hostname ]
port = replica_1[ :port ]
puts "replica_1 is primary"
elsif !replica_2.nil? and replica_2[ :primary ] == true
hostname = replica_2[ :hostname ]
port = replica_2[ :port ]
puts "replica_2 is primary"
elsif !replica_3.nil? and replica_3[ :primary ] == true
hostname = replica_3[ :hostname ]
port = replica_3[ :port ]
puts "replica_3 is primary"
elsif !replica_4.nil? and replica_4[ :primary ] == true
hostname = replica_4[ :hostname ]
port = replica_4[ :port ]
puts "replica_4 is primary"
end
if hostname.nil? or hostname.empty? or port.nil?
puts "Need hostname:port of one of the replica nodes to complete set-up"
end
mongo_command = "mongo --eval 'rs.addShard( \"#{replSet_name}/#{hostname}:#{port}\" )'"
execute "add-shard" do
command "#{mongo_command}"
retries 6
retry_delay 10
end
# --------------------------------------------------------------------
# TODO: Unless there's only one shard, we need to enable sharding and
# set up the shard key(s) using the MongoDB shell. These commands are:
# $ mongo --port <number> (port number out of shard bag)
#
# sh.enableSharding( "database-name" )
# sh.shardOn( "database-name.collection", { "field-name" : 1, "_id" : 1 } )
#
# The particulars should be governed by tuples in the node definition.
# "normal" :
# {
# "sharding" :
# {
# { "database" : <database-name> },
# { "collection" : <collection-name> },
# { "field1" : <field-name> },
# { "field2" : <field-name> },
# { "fieldn" : <field-name> )
# }
# }
#
# --or something like this. The following code is untested:
# --------------------------------------------------------------------
if !shardbag[ :sharding ].nil?
database_name = shardbag[ :sharding ][ :database ]
collection_name = shardbag[ :sharding ][ :collection ]
enable_command = 'sh.enableSharding( "#{database_name}" )'
mongo_command = "#{launch_shell} '#{enable_command}'"
execute "enable-shard" do
command "#{mongo_command}"
retries 6
retry_delay 10
end
# Now compose the shardOn() command. We'll consider up to 4 fields.
shardon_command = 'sh.shardOn( "#{database_name}.#{collection_name}", {'
field1 = shardbag[ :sharding ][ :field1 ]
field2 = shardbag[ :sharding ][ :field2 ]
field3 = shardbag[ :sharding ][ :field3 ]
field4 = shardbag[ :sharding ][ :field4 ]
if !field1.nil?
shardon_command += ' "#{field1}" : 1'
end
if !field2.nil?
shardon_command += ' "#{field2}" : 1'
end
if !field3.nil?
shardon_command += ' "#{field3}" : 1'
end
if !field4.nil?
shardon_command += ' "#{field4}" : 1'
end
shardon_command += ' "_id" : 1 }'
mongo_command = "#{launch_shell} '#{shardon_command}'"
execute "shardon" do
command "#{mongo_command}"
retries 6
retry_delay 10
end
end
bagname = "id:" + node[ :rollup ] # e.g.: "replicaset-1"
bag = search( :rollups, bagname ).first
hostname = bag[ :hostname ]
port = bag[ :port ]
execute "add-arbiter" do
command "mongo --eval 'rs.addArb( "#{hostname}":#{port}" )'"
retries 6
retry_delay 10
end
node.default[:mongodb][:package] = "mongodb-10gen" # this says we install 10-gen's package
# -----------------------------------------------------------------
# We'll be using the host IP address to make node names for MongoDB
# replica set specification.
# -----------------------------------------------------------------
require 'socket'
iphash = Socket.ip_address_list.find {|a| a.ipv4? ? !(a.ipv4_private? || a.ipv4_loopback?) : !(a.ipv6_sitelocal? || a.ipv6_linklocal? || a.ipv6_loopback?) }
$ip_address = iphash.ip_address
node.default[:nodeaddress] = $ip_address
puts $ip_address
node.default[:mongodb][:package] = "mongodb-10gen" # this says we install 10-gen's package
# -----------------------------------------------------------------
# We'll be using the host IP address to make node names for MongoDB
# replica set specification.
# -----------------------------------------------------------------
require 'socket'
iphash = Socket.ip_address_list.find {|a| a.ipv4? ? !(a.ipv4_private? || a.ipv4_loopback?) : !(a.ipv6_sitelocal? || a.ipv6_linklocal? || a.ipv6_loopback?) }
$ip_address = iphash.ip_address
node.default[:nodeaddress] = $ip_address
puts $ip_address
source :rubygems gem 'test-kitchen'
{
"name": "mongodb",
"description": "Installs/Configures mongodb-10gen my style",
"long_description": "",
"maintainer": "",
"maintainer_email": "",
"license": "To kill",
"platforms": { "ubuntu": ">= 12.04" },
"dependencies":
{
"apt": ">= 0.0.0",
"mongodb-10gen": ">= 2.4.5"
},
"recommendations": { },
"suggestions": { },
"conflicting": { },
"providing": { },
"replacing": { },
"attributes": { },
"groupings": { },
"recipes": { },
"version": "1.0.3"
}
name "mongodb" maintainer "" maintainer_email "" license "To kill" description "Installs/Configures mongodb-10gen" long_description IO.read( File.join( File.dirname( __FILE__ ), 'README.md' ) ) version "1.0.3" depends "apt" depends "mongodb-10gen" supports "ubuntu"
Description =========== Add apt repository and install mongodb-10gen. ## Platform * ubuntu ### Tested on * ubuntu 12.04(precise) Requirements ============ - OpscodeCookbook[apt] Attributes ========== ### group ['mongodb'] update your mongodb.conf values ### group ['mongodb']['config'] update your mongodb_config.conf values ### group ['mongodb']['router'] update your mongos.conf values Usage ===== ### Available recipes #### default - Add 10gen official repository and install newer stable mongodb. - **disable** autostart when install or serverboot. #### replica - set up /data/mongodb/mongodb.conf for any replica node #### arbiter - set up mongodb as arbiter for replica set. #### configsvr - set up mongodb configuration server. #### sharding - set up sharding router(mongos) node. Additions ========= Author ====== Author:: Russell Bateman (<[email protected]>)
. +-- attributes | `-- default.rb +-- files | `-- default | +-- mongodb | | +-- arbiter | | | `-- log | | +-- bin | | +-- configsvr | | | `-- log | | `-- sharding | | `-- log | +-- mongodb-upstart.conf | +-- multi-arbiter-upstart.conf | +-- multi-configsvr-upstart.conf | +-- multi-sharding-upstart.conf | +-- README.txt | `-- sharding-upstart.conf +-- Gemfile +-- metadata.json +-- metadata.rb +-- README.md +-- recipes | +-- add-arbiter.rb | +-- add-shard.rb | +-- arbiter.rb | +-- configsvr.rb | +-- default.rb | +-- README.txt | +-- replica.rb | +-- replicaset.rb | +-- sharding.rb | +-- test-add-shard.rb | +-- test-configsvr.rb | +-- test-replicaset.rb | +-- test-run.rb | `-- test-sharding.rb `-- templates `-- default +-- arbiter.conf.erb +-- configsvr.conf.erb +-- multi-arbiter.conf.erb +-- multi-configsvr.conf.erb +-- multi-sharding.conf.erb +-- replica.conf.erb `-- sharding.conf.erb 14 directories, 32 files