Site-wide Configuration Management for Debian GNU/Linux

2012-03-29 11:03

I may be a FreeBSD fangirl, but my company is a Debian GNU/Linux shop. This tutorial looks at how I create site configurations with config-package-dev, package them, and make them available on the network.

Motivation

  • Exists within APT, so no tools are required on a target machine besides those shipped with the operating system.
  • Configuration packages make it easy to distribute a configuration across a site.
  • Separation of machine and configuration. Configurations can be installed over a machine of any state, and the machine returns to its previous state upon uninstallation thanks to dpkg-divert.
  • Simple codename separation for maintaining compatible but discrete packages across releases, also enabling seamless release upgrades that would otherwise fail due to incompatible configuration, e.g. apache2 during the upgrade from Debian 5 to 6.
  • Free (monetarily)
  • Free (philosophically)
  • Config-package-dev can build configuration packages that transform the in-place configuration files, leaving the configuration package agnostic to config file changes from upstream.

Readying the APT Repo server

Install nginx and reprepro, my favourite confusingly-named but extremely powerful Debian repository manager.

  1. apt-get install nginx reprepro

Create a local user repo who will run reprepro and under whose home directory we will host the repository, then create the directory skeleton for the repository itself.

  1. useradd -d /home/repo -m -G www-data -s /usr/bin/zsh repo
  2. mkdir -p /home/repo/public/debian/{conf,dists,incoming,indices,logs,pool,project,tmp}
  3. chown -R repo:www-data /home/repo/public

Key Generation

APT repositories can be signed with a GnuPG private key. To generate one, install gnupg along with the rng-tools package.

  1. apt-get install rng-tools gnupg

Start rngd, seeding it with /dev/urandom. Without rngd, GnuPG may take a very very long time to gather enough entropy for safe key generation, as I experienced on a VMware ESXi virtual machine.

  1. rngd -r /dev/urandom

To create the key, run gpg --gen-key as repo and follow the prompts. Be sure to place a passphrase on your new key when prompted, or anybody who gets ahold of the private key can sign with it.

Take a peek at gpg --list-secret-keys and make sure your key matches expectation.

gpg --list-secret-keys
  1. /home/repo/.gnupg/secring.gpg
  2. ------------------------
  3. sec 2048R/013CFE9D 2011-10-12
  4. uid Nicole Reid (Debian repository signing key for Dashie) <nreid@engauge.com>
  5. ssb 2048R/42AC4BBC 2011-10-12

Lastly, export the public component of the key to the document root of the repository so clients can access it.

  1. gpg --armor --export 013CFE9D --output > /home/repo/public/debian/dashie.gpg.key

Reprepro

Reprepro’s configuration lives within the root of the repository, under the HTTP document root. In this setup, it’s /home/repo/public/debian/conf.

conf/distribution

This file defines the distributions for which we host repositories. It may contain multiple distribution definition blocks, separated from each other by one blank line. Each codename release of Debian (Lenny, Squeeze, Wheezy, et cetera) is a separate distribution in this context.

conf/distribution
  1. Origin: Nicole Reid
  2. Label: Dashie
  3. Codename: squeeze
  4. Architectures: i386 amd64 source
  5. Components: main extra
  6. Description: Engauge.com Debian Squeeze configurations and packaged software.
  7. SignWith: 013CFE9D

This working example is a repository hosting the main and extra components for Debian 6.0 “Squeeze” on the i386 and amd64 architectures and signed by a GnuPG key with id 013CFE9D, as seen in the previous section.

conf/options

This file defines configuration for reprepro itself. Most of these are personal preference. They correspond to command line arguments to reprepro.

conf/options
  1. verbose
  2. basedir /home/repo/public/debian
  3. ask-passphrase

Nginx Configuration

Define a new virtual host to expose our repository on the network, making sure to deny access to the configuration and database subdirectories. I host with SSL, listening on port 80 only to issue redirects to the SSL host on port 443.

/etc/nginx/sites-available/apt.engauge.com
  1. server {
  2. listen 80;
  3. server_name apt.engauge.com;
  4. rewrite ^ https://apt.engauge.com$request_uri? permanent;
  5. }
  6. server {
  7. listen [::]:443 default ssl ipv6only=on;
  8. listen 443 default ssl;
  9. server_name apt.engauge.com;
  10. ssl_certificate /etc/ssl/certs/wildcard.engauge.com.crt;
  11. ssl_certificate_key /etc/ssl/private/wildcard.engauge.com.key;
  12. access_log /var/log/nginx/apt.access.log;
  13. autoindex on;
  14. location / {
  15. root /home/repo/public;
  16. index index.html;
  17. }
  18. location ~ /(.*)/conf {
  19. deny all;
  20. }
  21. location ~ /(.*)/db {
  22. deny all;
  23. }
  24. }

Enable it by symlinking your virtual host configuration file under sites-enabled, and reload nginx.

  1. ln -s /etc/nginx/sites-available/apt.engauge.com /etc/nginx/sites-enabled/apt.engauge.com
  2. service nginx reload

Packaging

Packaging is a complex topic, and newcomers would do well to read the Introduction to Debian Packaging.

You should have the package you’re configuring installed on your repository build machine, or file diversion might fail during package construction.

Prerequesites

Install a few necessary build dependencies.

  1. apt-get install config-package-dev build-essential cdbs debhelper wdiff debian-el \
  2. devscripts devscripts-el dh-make dpatch dpkg-awk dpkg-dev dpkg-dev-el equivs \
  3. fakeroot lintian quilt sbuild

Example Package: Configuration Transformation

This is a working example of a configuration transformation package that, when installed, will disable root logins in OpenSSH, force SSH 2 protocol, disable host-based authentication, and disallow empty passwords.

Create the directory structure for the package.

  1. mkdir -p dashie-openssh-server-config/debian

The control file names, declares dependencies for, and includes a description for our package.

dashie-openssh-server-config/debian/control
  1. Source: dashie-openssh-server-config
  2. Section: admin
  3. Priority: extra
  4. Maintainer: Nicole Reid <nreid@engauge.com>
  5. Build-Depends: openssh-server, cdbs (>= 0.4.23-1.1), debhelper (>= 6), config-package-dev (>= 4.5~)
  6. Standards-Version: 3.9.2
  7. Package: dashie-openssh-server-config
  8. Architecture: all
  9. Priority: extra
  10. Depends: openssh-server, ${misc:Depends}
  11. Provides: ${diverted-files}
  12. Conflicts: ${diverted-files}
  13. Description: OpenSSH server site configuration
  14. This package installs Engauge site configuration for OpenSSH
  15. server, a slightly more restricted configuration than stock.
  16. /!\ Will prevent remote root logins. Make sure you have /!\
  17. /!\ console access or a user that can su! /!\

rules is a Makefile where we configure config-package-dev for this package, telling it what on-filesystem files we wish to transform.

dashie-openssh-server-config/debian/rules
  1. #!/usr/bin/make -f
  2. DEB_DIVERT_EXTENSION = .dashie
  3. DEB_TRANSFORM_FILES_dashie-openssh-server-config += \
  4. /etc/ssh/sshd_config.dashie
  5. include /usr/share/cdbs/1/rules/debhelper.mk
  6. include /usr/share/cdbs/1/rules/config-package.mk

transform_sshd_config.dashie is named for the file it’s transforming, /etc/ssh/sshd_config.dashie, diverted to /etc/ssh/sshd_config by our package. My transform files are usually written in perl, but that’s not a requirement.

dashie-openssh-server-config/debian/transform_sshd_config.dashie
  1. #!/usr/bin/perl -0p
  2. s/^PermitRootLogin.*$/PermitRootLogin no/m or die;
  3. s/^Protocol.*$/Protocol 2/m or die;
  4. s/^HostbasedAuthentication.*$/HostbasedAuthentication no/m or die;
  5. s/^PermitEmptyPasswords.*$/PermitEmptyPasswords no/m or die;

postinst is a shell script called with the argument configure after package installation. In this example, it restarts the OpenSSH server to pick up its new configuration.

dashie-openssh-server-config/debian/postinst
  1. #! /bin/sh
  2. set -e
  3. #DEBHELPER#
  4. case "$1" in
  5. configure)
  6. service ssh restart
  7. ;;
  8. *)
  9. echo "$0: didn't understand being called with \`$1'" 1>&2
  10. exit
  11. ;;
  12. esac

The package changelog keeps a history of the package. It can be edited by hand or updated by a tool such as dch.

dashie-openssh-server-config/debian/changelog
  1. dashie-openssh-server-config (1.1) unstable; urgency=low
  2. * Include postinst to restart OpenSSHd after installation.
  3. -- Nicole Reid <nreid@engauge.com> Tue, 29 Nov 2011 10:17:42 -0500
  4. dashie-openssh-server-config (1.0) unstable; urgency=low
  5. * Initial release.
  6. -- Nicole Reid <nreid@engauge.com> Fri, 07 Oct 2011 11:37:03 -0400

compat is a very simple file, containing only the number of the debhelper version needed by your package. This package, for example, declares a need for debhelper (>= 6.0.0), the version shipped with Debian Lenny.

dashie-openssh-server-config/debian/compat
  1. 6

The copyright file is simply the license you prefer for the package.

dashie-openssh-server-config/debian/copyright
  1. Linux server site configuration package
  2. Author: Nicole Reid <nreid@engauge.com>
  3. Copyright © 2011 Nicole Reid <nreid@engauge.com>
  4. Permission is hereby granted, free of charge, to any person obtaining
  5. a copy of this software and associated documentation files (the
  6. "Software"), to deal in the Software without restriction, including
  7. without limitation the rights to use, copy, modify, merge, publish,
  8. distribute, sublicense, and/or sell copies of the Software, and to
  9. permit persons to whom the Software is furnished to do so, subject to
  10. the following conditions:
  11. The above copyright notice and this permission notice shall be
  12. included in all copies or substantial portions of the Software.
  13. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
  14. EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
  15. MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
  16. NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
  17. LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
  18. OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
  19. WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Example Package: Configuration Replacement

This is a working example of a configuration replacement package that, when installed, will install a systemwide zshrc, a custom ZSH prompt, and a blank .zshrc in the skeleton home directory. The blank .zshrc prevents ZSH from prompting users for first-run configuration, instead preferring our system-wide config.

Specify a dependency on ZSH in control, so the systemwide zshrc will exist for us to replace.

dashie-zsh-config/control
  1. Source: dashie-zsh-config
  2. Section: admin
  3. Priority: extra
  4. Maintainer: Nicole Reid <nreid@engauge.com>
  5. Build-Depends: zsh, cdbs (>= 0.4.23-1.1), debhelper (>= 6), config-package-dev (>= 4.5~)
  6. Standards-Version: 3.9.2
  7. Package: dashie-zsh-config
  8. Architecture: all
  9. Priority: extra
  10. Depends: zsh, ${misc:Depends}
  11. Provides: ${diverted-files}
  12. Conflicts: ${diverted-files}
  13. Description: ZSH site configuration
  14. This package installs Engauge site configuration for ZSH, Nicole's
  15. favourite shell. This configuration provides some nice defaults for
  16. anybody who cares to use ZSH.

<name-of-package>.install is a list of source and destination for files you want dh_install to install. This simple one-line example says every file in and under the packages’ files directory should be installed to the root of the filesystem.

dashie-zsh-config/dashie-zsh-config.install
  1. files/* /

In this packages rules, list the files the package will replace. In this case, the package installs /etc/zsh/zshrc.dashie and diverts it to /etc/zsh/zshrc.

dashie-zsh-config/rules
  1. #!/usr/bin/make -f
  2. DEB_DIVERT_EXTENSION = .dashie
  3. DEB_DIVERT_FILES_dashie-zsh-config += \
  4. /etc/zsh/zshrc.dashie
  5. include /usr/share/cdbs/1/rules/debhelper.mk
  6. include /usr/share/cdbs/1/rules/config-package.mk

This package has a files directory containing the installed files in their proper tree. The simple dh_install file above will copy this entire directory tree over the root of the filesystem.

tree /home/repo/dashie/squeeze/dashie-zsh-config/files
  1. /home/repo/dashie/squeeze/dashie-zsh-config/files
  2. ├── etc
  3. │   ├── skel
  4. │  │  └── .zshrc
  5. │   └── zsh
  6. │   └── zshrc.dashie
  7. └── usr
  8. └── share
  9. └── zsh
  10. └── functions
  11. └── Prompts
  12. └── prompt_nicole_setup
  13. 8 directories, 2 files

The other required files, changelog, compat, and license, are no different from the previous example.

Building

Run debuild -us -uc in the package’s top-level directory. The -us and -uc arguments tell dpkg-buildpackage not to cryptographically sign the source or the .changes file, just the package. If all goes well, you’ll find a __.deb a level up.

If the build is unsuccessful, example the verbose output for the cause.

Updating Existing Packages

When making changes to a package you’ve already built and distributed, comment your change with dch, using -i to increment the package version number, then debuild as usual. dch will automatically try to guess your name and email address, but you can specify them with the DEBFULLNAME and DEBEMAIL environment variables.

  1. export DEBEMAIL="nreid@engauge.com"
  2. export DEBFULLNAME="Nicole Reid"
  3. dch -i -m 'Hot new stuff for this version'
  4. debuild -us -uc

Add to Repository

Use reprepro to add the newly-built .deb to the repository. Specify the package’s component (main for packaged software, extra for configuration packages) and the codename distribution to target.

  1. reprepro --basedir ~/public/debian -C extra includedeb squeeze ~/dashie/squeeze/dashie-openssh-server-config_1.1_all.deb

When prompted, enter your GnuPG key’s passphrase. gpg-agent can help mitigate annoying successive password prompts.

Installation on Client Machines

With all that mess taken care of, configuring a client machine is as easy as adding your new repository to the APT sources list.

  1. echo 'deb https://apt.engauge.com/debian squeeze main extra
  2. deb-src https://apt.engauge.com/debian squeeze main extra' >> /etc/apt/sources.list.d/dashie.list

Enabling HTTPS transport for apt.

  1. apt-get install apt-transport-https

Adding the repo’s public key to the APT keyring.

  1. wget https://apt.engauge.com/dashie.gpg.key -O - | apt-key add -

…And running apt-get update.

Your packages should now be available for installation through APT.

aptitude search '~O Nicole'
  1. p apache-activemq - Apache ActiveMQ message broker.
  2. p dashie-atlhttp1-role - atlhttp1 role configuration
  3. p dashie-atlhttpdev1-role - atlhttpdev1 role configuration
  4. p dashie-atljenkins1-role - atljenkins1 role configuration
  5. p dashie-atlsql1-role - atlsql1 role configuration
  6. p dashie-atlsqldev1-role - atlhttp1 role configuration
  7. p dashie-kerberos-config - Kerberos 5 site configuration
  8. p dashie-ldap-config - OpenLDAP site configuration
  9. p dashie-mpt-status-config - mpt-status site configuration
  10. p dashie-nss-config - Name Service Switch site configuration
  11. i dashie-ntp-config - NTP site configuration
  12. i dashie-openssh-server-config - OpenSSH server site configuration
  13. p dashie-pam-config - PAM site configuration
  14. p dashie-samba-config - Samba server site configuration
  15. p dashie-sudo-config - Sudo site configuration
  16. i dashie-vim-config - Vim site configuration
  17. p dashie-wildcard.dev.engauge.co - *.dev.engauge.com SSL certificate
  18. p dashie-wildcard.engauge.com-ce - *.engauge.com SSL certificate
  19. i dashie-zsh-config - ZSH site configuration
  20. p gitorious - Gitorious git repository manager
  21. p ruby-enterprise - Ruby Enterprise Edition.

Automating

Set a few environment variables in the new user’s .zprofile or .profile, depending on your shell of choice. DEBEMAIL and DEBFULLNAME will automatically apply to dch changelogs generated as this user. Aliasing reprepro to reprepro with a basedir argument lets us easily run the command from any directory.

~/.zprofile or ~/.profile
  1. export DEBEMAIL="nreid@engauge.com"
  2. export DEBFULLNAME="Nicole Reid"
  3. alias reprepro="reprepro --basedir ~/public/debian"

You can permanently specify debuild arguments in your user’s .devscripts to save including them manually every time.

~/.devscripts
  1. DEBUILD_DPKG_BUILDPACKAGE_OPTS="-us -uc"

Keychain is a handy piece of software to keep gpg-agent alive. Install it with apt-get install keychain and add an entry with your key ID to your .zprofile or .profile.

~/.zprofile or ~/.profile
  1. eval `keychain --eval --agents gpg 013CFE9D`