The Debian/Ubuntu Platform Package Manager

This is to study how to install VAS on Debian/Ubuntu. (Pronounced /DAY bjan/.) On the day this was composed, I had only an Ubuntu box to work with. To skip all this primitive stuff and get on with some Debian package building, go here.


Platform details from uname et al...
	[email protected]:~# uname -a
	Linux redbox.vintela.com 2.6.15-23-386 #1 PREEMPT Tue May 23 13:49:40 UTC 2006 i686 GNU/Linux
	(kernel-name, nodename, kernel-release, kernel-version, machine, [processor, hardware-platform]
	operating-system)

In comparison with host taliesin, ...

	[email protected]:~> uname -a
	Linux taliesin.vintela.com 2.6.13-15.8-smp #1 SMP     Tue Feb 7  11:07:24 UTC 2006 i686 i686 i386 GNU/Linux
	Linux redbox.vintela.com   2.6.15-23-386   #1 PREEMPT Tue May 23 13:49:40 UTC 2006 i686           GNU/Linux

Back to redbox...

	[email protected]:~# uname -m         # (machine)
	i686

	[email protected]:~# uname -p         # (processor)
	unknown

	[email protected]:~# uname -i         # (hardware-platform)
	unknown

...and to see how we can discover the names “Debian” or “Ubuntu”...

	[email protected]:/etc# cat /etc/debian_version
	testing/unstable

Finally, from http://abock.org/2005/09/20/it-must-be-rocket-science/, we discovered the /etc/lsb-release file.

	[email protected]:~# ls -l /etc/lsb-release
	-rw-r--r-- 1 root root 101 2006-05-22 02:43 lds-release
	[email protected]:~# cat /etc/lsb-release
	DISTRIB_ID=Ubuntu
	DISTRIB_RELEASE=6.06
	DISTRIB_CODENAME=dapper
	DISTRIB_DESCRIPTION="Ubuntu 6.06 LTS"

Presumably, the following would come from an actual Debian host:

	# cat /etc/lsb-release
	DISTRIB_ID=Debian
	DISTRIB_RELEASE=3.1
	DISTRIB_CODENAME=sarge
	DISTRIB_DESCRIPTION="Debian GNU/Linux"

Another Ubuntu description seen...

	DISTRIB_DESCRIPTION="Ubuntu (The Hoary Hedgehog Release)"

...so, the codename and description strings are pretty useless.

However, reading deeper, it appears that the lsb_release -a command is better...

	[email protected]:/etc# lsb_release -a
	No LSB modules are available.
	Distributor ID: Ubuntu
	Description:    Ubuntu 6.06 LTS
	Release:        6.06
	Codename:       dapper

..., but /etc/issue is much easier to parse and a good last resort anyway if the lsb_release command isn’t present (they said it not I). This is what we’re going to use in the installation script:

	[email protected]:/etc# cat /etc/issue
	Ubuntu 6.06 LTS \n \l

Package manager (dpkg) details...

rpm is present, but nothing is installed...

	[email protected]:~# rpm
	RPM version 4.4.1
	Copyright (C) 1998-2002 - Red Hat, Inc.
	This program may be freely redistributed under the terms of the GNU GPL

	[email protected]:~# rpm
	(no reponse because no packages installed)

This is one of the proper Debian package managers (the lowest-level; the other one is apt)...

	[email protected]:~# dpkg -l
	Desired=Unknown/Install/Remove/Purge/Hold
	| Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed
	|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
	||/ Name           Version        Description
	+++-==============-==============-============================================
	ii  acpi           0.09-1         displays information on ACPI devices
	(many packages listed...)

	[email protected]:/etc# ll | grep dpkg
	drwxr-xr-x  3 root   root        96 2006-06-07 15:01 dpkg
	-rw-r--r--  1 root   root       470 2005-09-29 11:40 nsswitch.conf.dpkg-dist
	-rw-r--r--  1 root   root       552 2006-05-12 11:42 pam.conf.dpkg-dist

Compare with host taliesin...

	[email protected]:~> ll /etc | grep dpkg
	(nothing)

dpkg commands...

To install...

	# dpkg --install package-path
	# dpkg -i package-path

To list installed packages...

	# dpkg -list package-name-pattern
	# dpkg -l package-name-pattern

Unused by the install script, this command lists the files installed to the system by the package:

	# dpkg -listfiles package-name
	# dpkg -L package-name

To upgrade...

	# dpkg --update-avail package-path

There is also dpkg -A package-path which I was using, but according to the man page, that might not be as useful as the one listed here.

To uninstall (remove)...

	# dpkg -remove package-name
	# dpkg -r package-name

I heard it asserted by someone in the vim newsgroup that the only command that truly uninstalls a package is. This is because the remove command doesn’t remove the configuration files.

	# dpkg --purge package-name
	# dpkg -P package-name

Lists all packages containing .

	# dpkg - l | grep 

Show where everything is installed.

	# dpkg -L 

Debian package manager apt...

Ubuntu/Debian package manager:

	[email protected]:~> apt-cache search element
	[email protected]:~> apt-get   install package-name


dpkg tricks...


Building dpkgs...

16 January 2006

The package is usually built using dpkg-buildpackage, which reads several files, to be located in the Debian subdirectory of a source tree (repository).

Other tools exist also; files I believe refers to a list of participating files used at points in the building process.

debhelper is a suite of programs to help write and maintain a rules file. It obviates a lot of cutting and pasting.

The Common Debian Build System (CDBS) exists to make Debian package creation easier, in particular, the rules step. However, VAS is already compiled by the time a package is built. It’s unclear whether this will be helpful.

Once built, a Debian package can be checked for common problems using lintian.

Imperatives

There seem to be some imperatives when creating a package for Debian/Ubuntu distribution. This is explained in the Debian Policy Manual. This includes categorization into one of three sections, main, contributory and non-free. Refer to the manual, 2.4 Sections, on this. Each package needs a priority value (2.5 Priorities) including required, important, standard, optional and extra. These mostly refer to Debian OS components; VAS would probably be strictly “extra.”

The package name is specified by Package in the control field and survives all the way into the .deb file.

The version is recorded by Version and is in adequate format for dpkg to discern whether packages are being up- or down-graded, etc.

A maintaining person or group must be specified by Maintainer; this is an e-mail address, i.e.: Quest Software, Inc. <[email protected]>.

In addition to the above, every package must have:

The package installation scripts are not supposed to produce output that isn’t strictly necessary for the user to see.

(I left off reading at the end of chapter 4 because Jeff wasn’t serious about doing native-package building for Debian.)


Rebuild Aptitude source lists...

Sometimes, it's useful to remove apt lists and rebuild them. sudo/root access is needed: Do this:

# sudo bash
$ cd /var/lib/apt
$ mv lists lists.old
$ apt-get update

Look also at the output from

$ apt-key list

Building Debian packages

Here are some notes on building Debian packages. Be sure to see Building dpkgs... above too.

Links

To get started, you don't need anything special to build a Debian package except dpkg and whatever you're building into the package, which is pretty much your own business.

The Debian package consists of:

  1. Control file
  2. Dependencies
  3. Conflict (sort of "anti-dependencies")
  4. Scripts

Control file

This is basically a manifest in much the same way as MANIFEST.MF for a JAR or WAR file. Inside, there's a build number (version) that you'll want to increment automatically from whatever system (like ant) you're building the package with.

Package: accountmgr
Priority: optional
Maintainer: Acme group
Section: accountmgr
Architecture: amd64
Version: 2.09.113
Size: 30000
Installed-Size: 30000
Depends: tomcat6 (>= 6.0)
Description: Acme Account Services package

Procedure

Follow these steps, probably in ant. This is very fuzzy still; I'm working on a good methodology for these notes.

  1. Create a filesystem modeling the installation.
  2. Copy files, including the main ones (like a WAR) into the filesystem
  3. Copy scripts into a script area (this is a subset of the previous step)
  4. Fix up permissions (made necessary by ant whose commands do not deal very well with filesystem permissions issues.
  5. Build the package using dpkg --build package-root. This creates package.deb.
  6. Add detail to the package thus: dpkg --info package.deb.
  7. Tuck it into some Aptitude repository.

Aptitude repository

This is proprietary; you choose a repository belonging to your company. Target hosts (hosts that install your software) set up their Aptitude repository source lists to include your repository whence their Aptitude commands find your new package and install, upgrade or remove it. The commands are:

$ sudo apt-get update
$ sudo apt-get install package
$ sudo apt-get update package
$ sudo apt-get purge package

Building Debian packages—a practical example

I downloaded MongoDB to repair the post-install script. I'm going to do this and keep it around for use instead of trying to get it from Aptitude.

  1. Download the Debian package—without installing it.

    This is especially easy if you've got a place where it's already downloaded. Check the path /var/cache/apt/archives for the package. Otherwise (and I haven't done this), you might be able to get it via this command, which typically won't work if you do happen already to have it:

    $ sudo apt-get --download-only mongodb-10gen
    
  2. Next, copy the package to a nice place where you can work on it, e.g.: /home/user/Downloads/mongodb. Make that your current working directory.
  3. Create a new subdirectory where the package contents will go, e.g.: mongodb-package. You should now see the following in /home/user/Downloads/mongodb
    ~/Downloads/mongodb $ ll
    drwxr-xr-x 3 russ russ 4096 Jul 29 14:21 .
    drwxr-xr-x 5 russ russ 4096 Jul 29 14:16 ..
    -rw-r--r-- 1 russ russ 87724864 Jul 29 14:21 mongodb-10gen_2.4.5_amd64.deb
    drwxr-xr-x 6 russ russ 4096 Jul 29 14:01 mongodb-package
    
  4. Then, extract the package to it:
    ~/Downloads/mongodb $ dpkg-deb -x mongodb-10gen_2.4.5_amd64.deb ./mongodb-package
    

    You should now see the entire package contents extracted in your subdirectory:

    ~/Downloads/mongodb $ tree
    .
    |-- mongodb-10gen_2.4.5_amd64.deb
    `-- mongodb-package
    |-- DEBIAN
    | |-- conffiles
    | |-- control
    | |-- md5sums
    | |-- postinst
    | |-- postrm
    | |-- preinst
    | `-- prerm
    |-- etc
    | |-- init
    | | `-- mongodb.conf
    | |-- init.d
    | | `-- mongodb -> /lib/init/upstart-job
    | `-- mongodb.conf
    |-- usr
    | |-- bin
    | | |-- bsondump
    | | |-- mongo
    | | |-- mongod
    | | |-- mongodump
    | | |-- mongoexport
    | | |-- mongofiles
    | | |-- mongoimport
    | | |-- mongooplog
    | | |-- mongoperf
    | | |-- mongorestore
    | | |-- mongos
    | | |-- mongostat
    | | `-- mongotop
    | |-- sbin
    | `-- share
    | |-- doc
    | | `-- mongodb-10gen
    | | |-- changelog.gz
    | | `-- copyright
    | |-- lintian
    | | `-- overrides
    | | `-- mongodb-10gen
    | `-- man
    | `-- man1
    | |-- bsondump.1.gz
    | |-- mongo.1.gz
    | |-- mongod.1.gz
    | |-- mongodump.1.gz
    | |-- mongoexport.1.gz
    | |-- mongofiles.1.gz
    | |-- mongoimport.1.gz
    | |-- mongooplog.1.gz
    | |-- mongoperf.1.gz
    | |-- mongorestore.1.gz
    | |-- mongos.1.gz
    | |-- mongosniff.1.gz
    | |-- mongostat.1.gz
    | `-- mongotop.1.gz
    `-- var
    `-- lib
    `-- mongodb
    18 directories, 41 files
    
  5. Next, modify what's bugging you. In my case, this was the postinst script.
  6. Finally, reassemble the package:
    ~/Downloads/mongodb # dpkg-deb -b ./mongodb-package/ mongodb-10gen_2.4.5_amd64.deb
    dpkg-deb: building package `mongodb-10gen' in `mongodb-10gen_2.4.5_amd64.deb'.
    

To examine the reassembly and/or compare it to the original, you can use Nautilus (the GUI file system explorer) by finding the new (and old) package, right-clicking on it and choosing Open with Archive Manager. Then, if you want to see the contents of a text file, right-click the text file and choose gvim (etc.) to see it. You can't modify them there since everything is read-only.

The resulting package should install without trouble anywhere you copy it using the Debian package manager (rather than apt-get):

$ sudo dpkg -i mongodb-10gen_2.4.5_amd64.deb

The package problem I fixed...

For my own record: what I did to the postinst script 'cause I'll likely have to do it again.

Basically, the problem is that this script greps /etc/passwd to see if user mongodb already exists and if it doesn't, tries to create it. This doesn't work if your server's managed with LDAP and happens already to define user mongodb. Don't know why powers that be decided to create such a user, but it's in LDAP and not in /etc/passwd. adduser doesn't work just on what's in or not in that local file. The presence of this user makes me have to fix this script and run dpkg by hand everywhere I wish to install. Otherwise, apt-get install fails because dpkg fails. No automatic updates either.

I commented out the first line here in italics and added the line in bold, then rebuilt the package which worked.

21 case "$1" in
22 configure)
23 # create a mongodb group and user
24 #if ! grep -q mongodb /etc/passwd; then
25 if ! id mongodb ; then
26 adduser --system --no-create-home mongodb
27 addgroup --system mongodb
28 adduser mongodb mongodb
29 fi