Building Python Debug Extensions with dh
Using dh (the debhelper command sequencer), the process of creating both regular and -dbg packages for Python extensions can be automated to a larger extent. Because there is no canonical way to do this (yet) and even the basic objectives are sometimes misunderstood, this is often implemented wrong.
This page attempts to clarify the objectives, and to document one possible implementation. Of the various known approaches, the one taken by Jakub Wilk in python-djvu was chosen, as it is comparatively simple yet satisfies all of the objectives listed below. Portions of the Ubuntu Debug Build PyDbgBuilds documentation have been adapted as well.
This guide assumes that the developer is already familiar with packaging Python extensions in general, and with dh and python-support in particular.
This guide also assumes that you are developing for Debian's unstable distribution, and therefore may utilize features yet present only there. If you are developing for an older distribution or intend to provide backports for your package, #Alternatives lists alternative instructions for some of the steps described below. In particular, this guide assumes the following package versions:
python (>= 2.6)
debhelper (>= 8.1.0)
lintian (>= 2.5.0)
Goals
- a binary package containing the stripped extension, built for the regular interpreter
We will call this package python-foo.
- a binary package containing an extension built for the debug interpreter,
as well as the stripped debugging symbols for use with gdb
We will call this package python-foo-dbg.
- Full unit tests (where available) after the build process for both the regular an the debug extensions.
debian/control
You must Build-Depend on debhelper (>= 8.1.0~) (see the warning below in Build for earlier versions) and python-all-dbg. This will trigger support for building debug extensions in dh. You must also Build-Depend on python-support (>= 0.90).
python-foo-dbg must Depend on python-foo (= ${binary:Version}). python-foo-dbg must have Priority: extra, Section: debug.
python-foo-dbg should Recommend the debug interpreter python-dbg. A Depends is not strictly necessary because by installing python-foo-dbg, someone might merely be interested in the stripped debugging symbols for use with gdb and not care for the debug extension.
If python-foo Depends on other Python extensions, then python-foo-dbg requires the -dbg packages of those extensions for use with the debug interpreter. Therefore, if python-foo were to Depend on python-bar, then python-foo-dbg should at least Recommend: python-bar-dbg. A Depends is not strictly necessary for the same reasons as listed in the previous point. A Build-Depends, however, is necessary if you intend to run build-time unit tests.
Here is an example for how debian/control would look like including all of the above:
Source: python-foo Section: python Priority: optional Maintainer: Foo Bar <foo.bar@example.com> Build-Depends: python-all-dev, python-all-dbg, debhelper (>= 8.1.0~), python-support (>= 0.90), python-bar, python-bar-dbg Standards-Version: 3.9.1 Package: python-foo Architecture: any Depends: ${python:Depends}, ${misc:Depends}, ${shlibs:Depends}, python-bar Description: Python support for FOO This is the long package description. Package: python-foo-dbg Section: debug Priority: extra Architecture: any Depends: ${python:Depends}, ${misc:Depends}, ${shlibs:Depends}, python-foo (= ${binary:Version}) Recommends: python-dbg, python-bar-dbg Description: Python support for FOO (debug extension) This is the long package description for the debug package.
Build
With python-all-dbg in Build-Depends, dh_auto_build in debian/rules will automatically run the build process twice for every supported Python version: first with the debug interpreter, then with the regular interpreter.
Assuming that
python2.5 and python2.6 are present,
that python-foo supports both,
that python2.6 is the default Python version and that
dh_auto_build detects a distutils-based setup,
dh_auto_build in debian/rules will perform:
python2.5-dbg setup.py build python-dbg setup.py build python2.5 setup.py build python setup.py build
Unit Tests
Unit tests for all of the built extensions are performed by overriding dh_auto_test in debian/rules and running them manually. For each run, an appropriate PYTHONPATH must be set; this requires a bit of hackery because the pathname for the debug build directory changed after python-2.5 (from lib_d.* to lib.*-pydebug).
Assuming that the unit tests are run with the script mytests.py within the top directory, one way to accomplish this could be:
# Get the supported Python versions PYVERS = $(shell pyversions -r -v) # Callable functions to determine the correct PYTHONPATH pythonpath = $$(ls -d $(CURDIR)/build/lib.*-$(1)) pythonpath_dbg = $$(ls -d $(CURDIR)/build/lib_d.*-$(1) 2>/dev/null || ls -d $(CURDIR)/build/lib.*$(1)-pydebug) override_dh_auto_test: ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS))) set -e -x;\ for py in $(PYVERS); do \ PYTHONPATH=$(call pythonpath,$$py) python$$py mytests.py ;\ PYTHONPATH=$(call pythonpath_dbg,$$py) python$$py-dbg mytests.py ;\ done endif
Install
Just as for #Build above, dh_auto_install in debian/rules will automate most of the installation process for us (additional options passed by dh_auto_install have been omitted here for brevity):
python2.5-dbg setup.py install python-dbg setup.py install python2.5 setup.py install python setup.py install
Because our source package is now building multiple binary packages, dh_auto_install will install everything to debian/tmp by default. We therefore need to move those files to their intended destinations. Specifically, everything should go into python-foo except for *_d.so: those go into python-foo-dbg. Two possible ways to accomplish this are:
Option 1: *.install files
Using *.install files, dh_install in debian/rules can move them to the correct destinations for us:
debian/python-foo.install
usr/lib/python*/*-packages/*/*[!_][!_].so usr/lib/python*/*-packages/*/*.py ... etc ...
debian/python-foo-dbg.install
usr/lib/python*/*-packages/*/*_d.so
Option 2: override_dh_install
If your source package doesn't build any binary packages other than python-foo and python-foo-dbg, a simpler approach can be used. It does not require *.install files; overriding dh_install in debian/rules is sufficient:
override_dh_install: # Install everything excluding the *_d.so debug extensions to python-foo dh_install -X"*_d.so" "debian/tmp/*" -p python-foo # Install the debug extensions to python-foo-dbg dh_install "debian/tmp/usr/lib/python*/*-packages/foo/*_d.so" -p python-foo-dbg # Continue with regular dh_install dh_install
The last step is to include the stripped debugging symbols in the -dbg package, which dh_strip will automatically do for us with the --dbg-package option.
You currently still need to manually rename a directory within the installation path, namely pyshared to pymodules. See #576014
override_dh_strip: ifeq (,$(filter nostrip,$(DEB_BUILD_OPTIONS))) # Stripped symbols go into python-foo-dbg dh_strip --dbg-package=python-foo-dbg # Rename pyshared to pymodules cd debian/python-foo-dbg/usr/lib/debug/usr/lib && mv pyshared pymodules endif
Results
- python-foo contains a stripped regular extension
- → Goal A achieved
- python-foo-dbg will contain an extension built for the debug interpreter,
- → Goal B achieved
as well as the stripped debugging symbols for use with gdb.
- → Goal C achieved
- Unit tests (if available) have been performed for both the regular extensions and the debug extensions:
- → Goal D achieved
See Also
http://svn.debian.org/viewsvn/collab-maint/deb-maint/python-support/trunk/README
/usr/share/perl5/Debian/Debhelper/BuildSystem/python-distutils.pm
/usr/share/doc/python/README.debug
Alternatives
As previously mentioned, the above assumes that you are developing for unstable, and has corresponding package dependencies for debhelper et al. If this is not the case, you may need to modify some of the steps according to the rules below.
debhelper (<< 8.1.0)
At build time, if your package contains any scripts, the shebang will be changed to #!/usr/bin/python-dbg, which you'll have to change back manually (see 589759). The following workaround supplied by Joey Hess should work in those cases:
override_dh_auto_build: dh_auto_build -- --force