Translation(s): none


This is a tutorial how to create a Debian package for Astropy Affiliated Packages. The tutorial consists of three parts:

  1. Preparation

  2. Packaging
  3. Upload and advanced packaging

Note that there is a Sprint Meeting in Heidelberg at 2015-08-14 where we organize a packaging tutorial.


1. Preparation

Previous page

2. Packaging

File and git layout

All debian specific packaging is done within the debian subdirectory, which is created on the top directory of the original package. The content of the original package is not changed directly (exceptions see below).

The standard git layout for debian packaging consists of three branches:

Install upstream files

Debian needs to have the source tar file with a specific name. So, download the latest packages from pypi and link it to the appropriate file name:

$ wget https://pypi.python.org/packages/source/w/wcsaxes/wcsaxes-0.4.tar.gz
$ ln -s wcsaxes-0.4.tar.gz wcsaxes_0.4.orig.tar.gz
$ cd wcsaxes

Inject all source files into the upstream branch of git, which needs to be created first. Finally, we store the metadata of the tar file in the pristine-tar branch:

$ git checkout -b upstream
$ tar xf ../wcsaxes_0.4.orig.tar.gz --strip-components=1
$ ls
ah_bootstrap.py  CHANGES.md  ez_setup.py  PKG-INFO   setup.cfg  wcsaxes
astropy_helpers  docs        licenses     README.md  setup.py
$ git add --all
$ git commit -m "Initial upstream version 0.4"
$ git tag upstream/0.4
$ pristine-tar commit ../wcsaxes_0.4.orig.tar.gz

Finally, we switch back into our master branch:

$ git checkout -b master

Create the debian specific files

Create the directory debian for the Debian specific files:

$ mkdir debian

Static files

There are two files that show the compatibility version and the format of the debian source package: debian/compat and debian/source/format. They can be created the following way:

$ echo 9 > debian/compat
$ mkdir debian/source
$ echo "3.0 (quilt)" > debian/source/format

debian/control

debian/control is the main configuration file for the Debian package. It contains a section for the source, and one section for each binary package. The sections should be separated by an empty line.

First the section for the package source and build process. This defines, for example, the build dependencies (which need to be adjusted for your package):

Source: wcsaxes
Section: python
Priority: optional
Maintainer: Debian Astronomy Team <debian-astro-maintainers@lists.alioth.debian.org>
Uploaders: Ole Streicher <olebole@debian.org>
Build-Depends: debhelper (>= 9),
               dh-python,
               python-all,
               python-astropy,
               python-astropy-helpers,
               python-matplotlib,
               python-setuptools,
               python3-all,
               python3-astropy,
               python3-astropy-helpers,
               python3-setuptools
Standards-Version: 3.9.8
Homepage: https://github.com/astrofrog/wcsaxes
Vcs-Git: https://anonscm.debian.org/debian-astro/packages/wcsaxes.git
Vcs-Browser: https://anonscm.debian.org/cgit/debian-astro/packages/wcsaxes.git
X-Python-Version: >= 2.7
X-Python3-Version: >= 3.3

Then follow sections for each binary package. We have will have one package for Python-2 and a package for Python-3. Per policy, the Python-2 package should be prefixed with python-, and the Python-3 package with python3-:

Package: python-wcsaxes
Architecture: all
Depends: ${misc:Depends}, ${python:Depends}
Description: Framework for plotting astronomical and geospatial data (Python 2)
 WCSAxes is a framework for making plots of Astronomical data in
 Matplotlib. It is affiliated with the Astropy project and is intended for
 inclusion in the Astropy package once stable.
 .
 This package contains the Python 2 version of the package.

The common and Python dependencies are calculated automatically, so one just specifies ${misc:Depends} and ${python:Depends} here. Other depenencies (which cannot be found automatically) can be added here manually.

The architecture is set to all, since the package is architecture independent (only byte code, no C-code). If the package contains C extensions that are compiled, replace all with any. This will build the package individually for each available Debian platform.

The Python-3 package is almost the same:

Package: python3-wcsaxes
Architecture: all
Depends: ${misc:Depends}, ${python3:Depends}
Description: Framework for plotting astronomical and geospatial data (Python 3)
 WCSAxes is a framework for making plots of Astronomical data in
 Matplotlib. It is affiliated with the Astropy project and is intended for
 inclusion in the Astropy package once stable.
 .
 This package contains the Python 3 version of the package.

debian/copyright

Debian is quite strict when it comes to the freeness of your code. Only packages that follow the "Debian Free Software Guidelines" are accepted. That requires a very careful documentation of the copyright of the package.

wcsaxes is (as most astropy affiliated packages) BSD licensed, so the debian/copyright looks like this:

Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: wcsaxes
Upstream-Contact: Thomas Robitaille <thomas.robitaille@gmail.com>
Source: https://github.com/astrofrog/wcsaxes

Files: *
Copyright: (c) 2014, Thomas P. Robitaille
 2011-2014, Astropy Developers
 2015 Ole Streicher <olebole@debian.org> (Debian files)
License: BSD-3-Clause
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are met:
 .
 * Redistributions of source code must retain the above copyright notice, this
   list of conditions and the following disclaimer.
 .
 * Redistributions in binary form must reproduce the above copyright notice,
   this list of conditions and the following disclaimer in the documentation
   and/or other materials provided with the distribution.
 .
 * Neither the name of the Astropy Team nor the names of its contributors may
   be used to endorse or promote products derived from this software without
   specific prior written permission.
 .
 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 POSSIBILITY OF SUCH DAMAGE.

For affiliated packages, there may be a licenses subdirectory in the upstream source which contains the used licenses. This can be used to start.

However, you must carefully check that the copyright is actually correct! Look into each individual source file to see the copyright holder and if it may have a different license. Do not forget bundled third party libraries, documentation files or data files. If in doubt, contact upstream to get a clear answer. All files need to be documented here, even if they are not used for the package.

Clarifying the license is one of the major jobs when packaging, and probably the most annoying one. Don't be sloppy here: each package is re-checked by Debians ftp-masters when it is uploaded. If the ftp-masters are not convinced that everything is fine (and they are picky!), they will reject the package, causing a substantial delay to get the package into Debian.

debian/changelog

This file is used to document the changes during the lifetime of the packge. In the build process, it is used to retrieve the version and the revision of the package.

A template can be created with the command

$ dch --create --package wcsaxes -v 0.4-1

This will open your editor:

wcsaxes (0.4-1) unstable; urgency=low

  * Initial release (Closes: #XXXXXX)

 -- Ole Streicher <olebole@debian.org> Sun, 10 May 2015 13:25:07 +0200

Replace the bug number by the number that was given to your ITP bug and save the file. For the first Debian package version, the changelog should just have this one entry.

The build scripts will take the version (0.4) and revision number (1) from here.

debian/watch

For packages hosted on pypi, the easiest way to get the watch file is via a specialized URL: http://pypi.debian.net/wcsaxes/watch This gives the following file:

version=3
opts=uversionmangle=s/(rc|a|b|c)/~$1/ \
http://pypi.debian.net/wcsaxes/wcsaxes-(.+)\.(?:zip|tgz|tbz|txz|(?:tar\.(?:gz|bz2|xz)))

debian/rules

This file contains the recipe to build the package. As for many other standard installation procedures, this is already builtin in the Debian tools. So this file is quite simple:

#!/usr/bin/make -f
#export DH_VERBOSE=1

export PYBUILD_NAME=wcsaxes
export http_proxy=127.0.0.1:9

%:
        dh $@ --with python2,python3 --buildsystem=pybuild

(Note that the last line is indented by tab, not by spaces, since this is a Makefile)

The http_proxy variable is set so that a connection to the internet is avoided during the build (all requirements have to be fullfilled locally by Debian Policy).

Finall, set the executable flag for debian/rules:

$ chmod a+x debian/rules

debian/patches

As already mentioned, we will not use the astropy_helpers that were in the upstream tar file but use the one that is provided by Debian. To change this, one needs to change the auto_use flag of the ah_bootstrap section in setup.cfg. This can be done by creating a patch for this file. First prepare the patch file:

$ mkdir debian/patches
$ quilt new use_system_astropy_helpers.patch
$ echo "Author: Ole Streicher <olebole@debian.org>" | quilt header -a
$ echo "Description: Use astropy_helpers provided by the system" | quilt header -a
$ quilt add setup.cfg

Now you are ready to edit the file setup.cfg. After you finished (change the flag from True to False), update the patch with the command

$ quilt refresh

This all will create a file debian/patches/use_system_astropy_helpers.patch which should look like this:

Author: Ole Streicher <olebole@debian.org>
Description: Use astropy_helpers provided by the system
--- a/setup.cfg
+++ b/setup.cfg
@@ -12,7 +12,7 @@
 norecursedirs = build docs/_build

 [ah_bootstrap]
-auto_use = True
+auto_use = False

 [metadata]
 package_name = wcsaxes

To un-apply all patches, do a

$ quilt pop -a

to re-apply them

$ quilt push -a

Omit the -a option to apply or un-apply just one patch instead of all.

Summary listing of all debian files

This is the list of all files we created so far:

$ ls -lR debian/
ls -lR debian/
total 24
-rw-r--r-- 1 ole ole  151 Jun  8 18:06 changelog
-rw-r--r-- 1 ole ole    2 Jun  8 18:04 compat
-rw-r--r-- 1 ole ole 1705 Jun  8 18:05 control
-rw-r--r-- 1 ole ole 1834 Jun  8 18:05 copyright
drwxr-xr-x 1 ole ole   76 Jun  9 08:39 patches
-rwxr-xr-x 1 ole ole  230 Jun  8 18:07 rules
drwxr-xr-x 1 ole ole   12 Jun  8 18:04 source
-rw-r--r-- 1 ole ole  137 Jun  8 18:06 watch

debian/patches:
total 8
-rw-r--r-- 1 ole ole  33 Jun  9 08:39 series
-rw-r--r-- 1 ole ole 267 Jun  8 18:10 use_system_astropy_helpers.patch

debian/source:
total 4
-rw-r--r-- 1 ole ole 12 Jun  8 18:04 format

After this, you should commit your first changes:

$ git add debian/
$ git commit -m "Initial Debian files"

Build the package

Once you created all needed files, you are ready to start a first build. As mentioned before, the build process is done with the "pbuilder" program that creates a chrooted clean Debian environment.

If you did not update this environment for a while, do this now:

$ sudo pbuilder update

Then the command to create the package is simply

$ pdebuild

Since the chroot needs root privilegues, you will be asked for your password.

Packaging will take some time, since the system is setup from scratch using only the packages the were specified as build dependencies. The build log is printed to the screen and also stored in the parent directory as ../wcsaxes_0.4-1_amd64.build.

If the build was successfull, the package files are created in /var/cache/pbuilder/result. These are:

If you have installed all requirements on your machine, you can now install the Debian packages with

$ sudo dpgk -i /var/cache/pbuilder/result/python-wcsaxes_0.4-1_all.deb

respectively

$ sudo dpgk -i /var/cache/pbuilder/result/python-wcsaxes_0.4-1_all.deb

If you run into missing dependencies, do a

$ sudo apt-get install -f

afterwards.

Lintian

Once your package builds completely, you should check it against a number of common mistakes. The main tool here is "lintian". Just run it with the .changes file as argument:

$ lintian /var/cache/pbuilder/result/wcsaxes_0.4-1_amd64.changes

All errors and warnings shown there should be fixed before the package can be uploaded. There are, however, sometimes false warnings, for example if some that way on (good) purpose, or just because of an outdated lintian. For example, in the case of wcsaxes, one will get the following warnings:

W: python-wcsaxes: image-file-in-usr-lib usr/lib/python2.7/dist-packages/wcsaxes/tests/baseline_images/changed_axis_units.png
[...]
W: python3-wcsaxes: image-file-in-usr-lib usr/lib/python3/dist-packages/wcsaxes/tests/baseline_images/update_clip_path_rectangular.png

These warnings come from an old requirement that all architecture independent data should go into /usr/share/. Since for python packages all files (architecture dependent as well as architecture independent) reside in the same package directory under /usr/lib/pythonX/dist-packages, this warning is not valid here.

If you don't understand the short message, you can just google it, or you specify the -i option; then lintian displays a longer explanation.

Lintian warnings that should be ignored on purpose, can be silenced by creating a file debian/<package>.lintian-overrides. In the case of wcsaxes, the file debian/python-wcsaxes.lintian-overrides has the following content:

# Python packages install all files into a common subdir by purpose.
python-wcsaxes: image-file-in-usr-lib

The file debian/python3-wcsaxes.lintian-overrides is similar. Note that you must give the reason why you disabled the lintian message as a comment.

Lintian can do much more tests, so to create a good package it is highly recommended to run it with the arguments -E -I --pedantic as well:

$ lintian -E -I --pedantic /var/cache/pbuilder/result/wcsaxes_0.4-1_amd64.changes
P: wcsaxes source: debian-watch-may-check-gpg-signature
N: 44 tags overridden (44 warnings)

3. Upload and advanced packaging

Next page


See also

There is quite a lot documentation available that you can check for further information. The most important pages are

The policies describe the basic rules to follow when packaging:


CategoryPackaging