The Debian/Ubuntu Platform Package Manager


Debian package manager details

For Debian/Ubuntu, see documentation at http://www.opensourcemanuals.org/manual/dpkg. These are the commands we’re going to use in an installation script...

Install:

   dpkg -i package-file.deb

Uninstall (remove):

   dpkg -r package-name

Update:

   dpkg -A package-file.deb

Information/ontology (search):

   dpkg -C

List information (less than clearly indicated in the man page):

   dpkg -l package-name

This is to study how to install VAS on Debian/Ubuntu. (Pronounced /DAY bjan/.) On the day this was composed, I had only an Ubuntu box to work with. To skip all this primitive stuff and get on with some Debian package building, go here.


Platform details from uname et al...
	root@redbox:~# uname -a
	Linux redbox.vintela.com 2.6.15-23-386 #1 PREEMPT Tue May 23 13:49:40 UTC 2006 i686 GNU/Linux
	(kernel-name, nodename, kernel-release, kernel-version, machine, [processor, hardware-platform]
	operating-system)

In comparison with host taliesin, ...

	russ@taliesin:~> uname -a
	Linux taliesin.vintela.com 2.6.13-15.8-smp #1 SMP     Tue Feb 7  11:07:24 UTC 2006 i686 i686 i386 GNU/Linux
	Linux redbox.vintela.com   2.6.15-23-386   #1 PREEMPT Tue May 23 13:49:40 UTC 2006 i686           GNU/Linux

Back to redbox...

	root@redbox:~# uname -m         # (machine)
	i686

	root@redbox:~# uname -p         # (processor)
	unknown

	root@redbox:~# uname -i         # (hardware-platform)
	unknown

...and to see how we can discover the names “Debian” or “Ubuntu”...

	root@redbox:/etc# cat /etc/debian_version
	testing/unstable

Finally, from http://abock.org/2005/09/20/it-must-be-rocket-science/, we discovered the /etc/lsb-release file.

	root@redbox:~# ls -l /etc/lsb-release
	-rw-r--r-- 1 root root 101 2006-05-22 02:43 lds-release
	root@redbox:~# cat /etc/lsb-release
	DISTRIB_ID=Ubuntu
	DISTRIB_RELEASE=6.06
	DISTRIB_CODENAME=dapper
	DISTRIB_DESCRIPTION="Ubuntu 6.06 LTS"

Presumably, the following would come from an actual Debian host:

	# cat /etc/lsb-release
	DISTRIB_ID=Debian
	DISTRIB_RELEASE=3.1
	DISTRIB_CODENAME=sarge
	DISTRIB_DESCRIPTION="Debian GNU/Linux"

Another Ubuntu description seen...

	DISTRIB_DESCRIPTION="Ubuntu (The Hoary Hedgehog Release)"

...so, the codename and description strings are pretty useless.

However, reading deeper, it appears that the lsb_release -a command is better...

	root@redbox:/etc# lsb_release -a
	No LSB modules are available.
	Distributor ID: Ubuntu
	Description:    Ubuntu 6.06 LTS
	Release:        6.06
	Codename:       dapper

..., but /etc/issue is much easier to parse and a good last resort anyway if the lsb_release command isn’t present (they said it not I). This is what we’re going to use in the installation script:

	root@redbox:/etc# cat /etc/issue
	Ubuntu 6.06 LTS \n \l

Package manager (dpkg) details...

rpm is present, but nothing is installed...

	root@redbox:~# rpm
	RPM version 4.4.1
	Copyright (C) 1998-2002 - Red Hat, Inc.
	This program may be freely redistributed under the terms of the GNU GPL

	root@redbox:~# rpm
	(no reponse because no packages installed)

This is one of the proper Debian package managers (the lowest-level; the other one is apt)...

	root@redbox:~# dpkg -l
	Desired=Unknown/Install/Remove/Purge/Hold
	| Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed
	|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
	||/ Name           Version        Description
	+++-==============-==============-============================================
	ii  acpi           0.09-1         displays information on ACPI devices
	(many packages listed...)

	root@redbox:/etc# ll | grep dpkg
	drwxr-xr-x  3 root   root        96 2006-06-07 15:01 dpkg
	-rw-r--r--  1 root   root       470 2005-09-29 11:40 nsswitch.conf.dpkg-dist
	-rw-r--r--  1 root   root       552 2006-05-12 11:42 pam.conf.dpkg-dist

Compare with host taliesin...

	russ@taliesin:~> ll /etc | grep dpkg
	(nothing)

dpkg commands...

To install...

	# dpkg --install package-path
	# dpkg -i package-path

To list installed packages...

	# dpkg -list package-name-pattern
	# dpkg -l package-name-pattern

Unused by the install script, this command lists the files installed to the system by the package:

	# dpkg -listfiles package-name
	# dpkg -L package-name

To upgrade...

	# dpkg --update-avail package-path

There is also dpkg -A package-path which I was using, but according to the man page, that might not be as useful as the one listed here.

To uninstall (remove)...

	# dpkg -remove package-name
	# dpkg -r package-name

I heard it asserted by someone in the vim newsgroup that the only command that truly uninstalls a package is. This is because the remove command doesn’t remove the configuration files.

	# dpkg --purge package-name
	# dpkg -P package-name

Lists all packages containing .

	# dpkg - l | grep 

Show where everything is installed.

	# dpkg -L 

Debian package manager apt...

Ubuntu/Debian package manager. For a long time, I called this "Aptitude" thinking that apt was just short of that word. After a decade of believing that, I discovered during a presentation that it stands for advanced packaging tool, though Aptitude does exist in relation to apt. The relation is simply that Aptitude is a front-end with user interace to the advanced packaging tool's command-line interface. Also, though I've never seen proof of this, Aptitude at some point extended its coverage to Red Hat packaing (RPM). Simply put, I mistook what Aptitude is and I don't think I've ever actually used it. I've only ever used apt.

	russ@taliesin:~> apt-cache search element
	russ@taliesin:~> apt-get   install package-name

dpkg tricks...


Building dpkgs...

16 January 2006

The package is usually built using dpkg-buildpackage, which reads several files, to be located in the Debian subdirectory of a source tree (repository).

Other tools exist also; files I believe refers to a list of participating files used at points in the building process.

debhelper is a suite of programs to help write and maintain a rules file. It obviates a lot of cutting and pasting.

The Common Debian Build System (CDBS) exists to make Debian package creation easier, in particular, the rules step. However, VAS is already compiled by the time a package is built. It’s unclear whether this will be helpful.

Once built, a Debian package can be checked for common problems using lintian.

Imperatives

There seem to be some imperatives when creating a package for Debian/Ubuntu distribution. This is explained in the Debian Policy Manual. This includes categorization into one of three sections, main, contributory and non-free. Refer to the manual, 2.4 Sections, on this. Each package needs a priority value (2.5 Priorities) including required, important, standard, optional and extra. These mostly refer to Debian OS components; VAS would probably be strictly “extra.”

The package name is specified by Package in the control field and survives all the way into the .deb file.

The version is recorded by Version and is in adequate format for dpkg to discern whether packages are being up- or down-graded, etc.

A maintaining person or group must be specified by Maintainer; this is an e-mail address, i.e.: Quest Software, Inc. <[email protected]>.

In addition to the above, every package must have:

The package installation scripts are not supposed to produce output that isn’t strictly necessary for the user to see.

(I left off reading at the end of chapter 4 because Jeff wasn’t serious about doing native-package building for Debian.)


Rebuild apt-get source lists...

Sometimes, it's useful to remove apt lists and rebuild them. sudo/root access is needed: Do this:

	# sudo bash
	$ cd /var/lib/apt
	$ mv lists lists.old
	$ apt-get update

Look also at the output from

	$ apt-key list

Preventing packages from being updated...

Block Ubuntu from ever updating Cassandra. This is because I'm working on a plug-in that only works with a, specific version of Cassandra.

# apt-mark hold cassandra
cassandra set on hold.

Building Debian packages

Here are some notes on building Debian packages. Be sure to see Building dpkgs... above too.

Links

To get started, you don't need anything special to build a Debian package except dpkg and whatever you're building into the package, which is pretty much your own business.

The Debian package consists of:

  1. Control file
  2. Dependencies
  3. Conflict (sort of "anti-dependencies")
  4. Scripts

Control file

This is basically a manifest in much the same way as MANIFEST.MF for a JAR or WAR file. Inside, there's a build number (version) that you'll want to increment automatically from whatever system (like ant) you're building the package with.

    Package: accountmgr
    Priority: optional
    Maintainer: Acme group
    Section: accountmgr
    Architecture: amd64
    Version: 2.09.113
    Size: 30000
    Installed-Size: 30000
    Depends: tomcat6 (>= 6.0)
    Description: Acme Account Services package

Procedure

Follow these steps, probably in ant. This is very fuzzy still; I'm working on a good methodology for these notes.

  1. Create a filesystem modeling the installation.
  2. Copy files, including the main ones (like a WAR) into the filesystem
  3. Copy scripts into a script area (this is a subset of the previous step)
  4. Fix up permissions (made necessary by ant whose commands do not deal very well with filesystem permissions issues.
  5. Build the package using dpkg --build package-root. This creates package.deb.
  6. Add detail to the package thus: dpkg --info package.deb.
  7. Tuck it into some apt-get repository.

Advanced packaging tool repository (apt)

This is proprietary; you choose a repository belonging to your company. Target hosts (hosts that install your software) set up their repository source lists to include your repository whence their commands find your new package and install, upgrade or remove it. The commands are:

    $ sudo apt-get update
    $ sudo apt-get install package
    $ sudo apt-get update package
    $ sudo apt-get purge package

Building Debian packages—a practical example

I downloaded MongoDB to repair the post-install script. I'm going to do this and keep it around for use instead of trying to get it from apt-get.

  1. Download the Debian package—without installing it.

    This is especially easy if you've got a place where it's already downloaded. Check the path /var/cache/apt/archives for the package. Otherwise (and I haven't done this), you might be able to get it via this command, which typically won't work if you do happen already to have it:

        $ sudo apt-get --download-only mongodb-10gen
    
  2. Next, copy the package to a nice place where you can work on it, e.g.: /home/user/Downloads/mongodb. Make that your current working directory.
  3. Create a new subdirectory where the package contents will go, e.g.: mongodb-package. You should now see the following in /home/user/Downloads/mongodb
        ~/Downloads/mongodb $ ll
        drwxr-xr-x 3 russ russ     4096 Jul 29 14:21 .
        drwxr-xr-x 5 russ russ     4096 Jul 29 14:16 ..
        -rw-r--r-- 1 russ russ 87724864 Jul 29 14:21 mongodb-10gen_2.4.5_amd64.deb
        drwxr-xr-x 6 russ russ     4096 Jul 29 14:01 mongodb-package
    
  4. Then, extract the package to it:
        ~/Downloads/mongodb $ dpkg-deb -x mongodb-10gen_2.4.5_amd64.deb ./mongodb-package
    

    You should now see the entire package contents extracted in your subdirectory:

        ~/Downloads/mongodb $ tree
        .
        |-- mongodb-10gen_2.4.5_amd64.deb
        `-- mongodb-package
            |-- DEBIAN
            |   |-- conffiles
            |   |-- control
            |   |-- md5sums
            |   |-- postinst
            |   |-- postrm
            |   |-- preinst
            |   `-- prerm
            |-- etc
            |   |-- init
            |   |   `-- mongodb.conf
            |   |-- init.d
            |   |   `-- mongodb -> /lib/init/upstart-job
            |   `-- mongodb.conf
            |-- usr
            |   |-- bin
            |   |   |-- bsondump
            |   |   |-- mongo
            |   |   |-- mongod
            |   |   |-- mongodump
            |   |   |-- mongoexport
            |   |   |-- mongofiles
            |   |   |-- mongoimport
            |   |   |-- mongooplog
            |   |   |-- mongoperf
            |   |   |-- mongorestore
            |   |   |-- mongos
            |   |   |-- mongostat
            |   |   `-- mongotop
            |   |-- sbin
            |   `-- share
            |       |-- doc
            |       |   `-- mongodb-10gen
            |       |       |-- changelog.gz
            |       |       `-- copyright
            |       |-- lintian
            |       |   `-- overrides
            |       |       `-- mongodb-10gen
            |       `-- man
            |           `-- man1
            |               |-- bsondump.1.gz
            |               |-- mongo.1.gz
            |               |-- mongod.1.gz
            |               |-- mongodump.1.gz
            |               |-- mongoexport.1.gz
            |               |-- mongofiles.1.gz
            |               |-- mongoimport.1.gz
            |               |-- mongooplog.1.gz
            |               |-- mongoperf.1.gz
            |               |-- mongorestore.1.gz
            |               |-- mongos.1.gz
            |               |-- mongosniff.1.gz
            |               |-- mongostat.1.gz
            |               `-- mongotop.1.gz
            `-- var
                `-- lib
                    `-- mongodb
    
        18 directories, 41 files
    
  5. Next, modify what's bugging you. In my case, this was the postinst script.
  6. Finally, reassemble the package:
    ~/Downloads/mongodb # dpkg-deb -b ./mongodb-package/ mongodb-10gen_2.4.5_amd64.deb
    dpkg-deb: building package `mongodb-10gen' in `mongodb-10gen_2.4.5_amd64.deb'.
    

To examine the reassembly and/or compare it to the original, you can use Nautilus (the GUI file system explorer) by finding the new (and old) package, right-clicking on it and choosing Open with Archive Manager. Then, if you want to see the contents of a text file, right-click the text file and choose gvim (etc.) to see it. You can't modify them there since everything is read-only.

The resulting package should install without trouble anywhere you copy it using the Debian package manager (rather than apt-get):

    $ sudo dpkg -i mongodb-10gen_2.4.5_amd64.deb

The package problem I fixed...

For my own record: what I did to the postinst script 'cause I'll likely have to do it again.

Basically, the problem is that this script greps /etc/passwd to see if user mongodb already exists and if it doesn't, tries to create it. This doesn't work if your server's managed with LDAP and happens already to define user mongodb. Don't know why powers that be decided to create such a user, but it's in LDAP and not in /etc/passwd. adduser doesn't work just on what's in or not in that local file. The presence of this user makes me have to fix this script and run dpkg by hand everywhere I wish to install. Otherwise, apt-get install fails because dpkg fails. No automatic updates either.

I commented out the first line here in italics and added the line in bold, then rebuilt the package which worked.

    21 case "$1" in
    22     configure)
    23     # create a mongodb group and user
    24         #if ! grep -q mongodb /etc/passwd; then
    25        if ! id mongodb ; then
    26         adduser --system --no-create-home mongodb
    27         addgroup --system mongodb
    28         adduser mongodb mongodb
    29         fi