This article explains one way to handle management of several different, yet similar sites using the following software:
Phusion Passenger setup is already described in this article and applies here for the most part. The few differences are mentioned specifically. The high-level view of this configuration is as follows:
Puppet configuration is split into global and local segments
One puppetmaster server is selected as the master, or more correctly the primary host of the global puppet repository
Other puppetmaster servers (or slaves) pull the global part of their configuration from the global repository
- All puppet configuration and definition files are stored in Git repositories
All node definitions are stored in local Git repositories
On the filesystem level this configuration looks like this:
/etc/puppet: puppet configuration files, local puppet modules, node definitions
/etc/puppetglobal: global puppet modules
The important thing is to keep everything puppetmaster needs to run in /etc/puppet. A number of things can go wrong if you change the defaults, for example:
mod_passenger not working at all when trying to add command-line parameters for puppetmasterd into config.ru
puppetmasterd not respecting $modulepath, or any other variables for that matter
These happened on puppetmaster 0.25.4; newer puppetmaster versions may be better. With the above directory layout you have one foolproof last resort: symlink every module directory in /etc/puppetglobal/modules to /etc/puppet/modules. This way the /etc/puppet directory is completely self-contained. All you need to do is add symlinks for any new global modules you create.
The rationale for doing this split is simple:
- Forces one to write reusable puppet code
- Helps avoid having to reinvent the wheel
- Avoids manual rule migration hell
- Allows use of local modules where applicable
It is very important to write any local modules so that they only depend on global modules, not vice versa. Otherwise you'll end up having to migrate your local modules to all servers, meaning they're not really local and should really be in the global puppet configuration instead.
Setting up the environment
First, make sure that Puppetmaster and Passenger are configured properly on all puppetmaster servers, e.g. as described here. Then stop puppetmasterd and puppetd services:
$ /etc/init.d/apache2 stop $ /etc/init.d/puppet stop
This prevents them messing up the puppet configuration files while you're at work.
Master server configuration
If you don't have puppetmaster in production, all you need to do is:
$ cd /etc/puppet $ git init
If you have a production puppetmaster already, do something like this instead:
$ mv /etc/puppet /etc/puppetglobal $ mkdir -p /etc/puppet/manifests/nodes $ mkdir -p /etc/puppet/modules $ mv /etc/puppetglobal/puppet.conf /etc/puppet/ $ mv /etc/puppetglobal/manifests/site.pp /etc/puppet/manifests/ $ mv /etc/puppetglobal/manifests/nodes/* /etc/puppet/manifests/nodes/ $ cd /etc/puppet $ git init $ cd /etc/puppetglobal $ git init
After this you can migrate site-specific ("local") modules to /etc/puppet/modules.
Slave server configuration
Make the existing puppet configuration directory a git repo:
$ cd /etc/puppet $ git init
Then configure puppetmaster normally. Next checkout master's global git repository using SSH:
$ cd /etc $ git clone ssh://email@example.com/etc/puppetglobal puppetglobal Cloning into puppetglobal... firstname.lastname@example.org's password: remote: Counting objects: 1857, done. remote: Compressing objects: 100% (1567/1567), done. remote: Total 1857 (delta 726), reused 0 (delta 0) Receiving objects: 100% (1857/1857), 16.95 MiB | 1.12 MiB/s, done. Resolving deltas: 100% (726/726), done.
Make sure the SSH user has appropriate permissions to /etc/puppetglobal on primary puppetmaster server.
Puppet module migration
Although migrating the files is trivial, you may have to manually merge lots of code. However, if module/class names don't clash, you can do the migration incrementally by starting with all modules in /etc/puppet/modules and moving them over to /etc/puppetglobal/modules in time. Also make sure the fileserver references in your modules don't point to a static, fully-qualified fileserver URI - unless you can use one for all puppetmasters. For details, look here.
When you're writing puppet code, first determine whether it's should be global or local. In a nutshell, anything that contains sensitive and/or site-specific information should be placed into the local puppet modules; the rest should be global. Examples:
user module that creates users to each servers should probably be local
authorization module that adds authorized SSH keys (e.g. to allow rsync-based backups) should probably be local
apache module should probably be global
In case of local puppet code you'll just follow the basic Git workflow. If you're making global code changes, do them at the master server and fetch the changes to the slaves using Git, e.g. with
$ cd /etc/puppetglobal $ git pull --rebase origin
You should be able to do pushes to the master, too, if you know your way around Git.