Building Python Debug Extensions with dh

Using dh (the debhelper command sequencer), the process of creating both regular and -dbg packages for Python extensions can be automated to a larger extent. Because there is no canonical way to do this (yet) and even the basic objectives are sometimes misunderstood, this is often implemented wrong.

This page attempts to clarify the objectives, and to document one possible implementation. Of the various known approaches, the one taken by Jakub Wilk in python-djvu was chosen, as it is comparatively simple yet satisfies all of the objectives listed below. Portions of the Ubuntu Debug Build PyDbgBuilds documentation have been adapted as well.

This guide assumes that the developer is already familiar with packaging Python extensions in general, and with dh and python-support in particular.

This guide also assumes that you are developing for Debian's unstable distribution, and therefore may utilize features yet present only there. If you are developing for an older distribution or intend to provide backports for your package, #Alternatives lists alternative instructions for some of the steps described below. In particular, this guide assumes the following package versions:

Goals

  1. a binary package containing the stripped extension, built for the regular interpreter
    • We will call this package python-foo.

  2. a binary package containing an extension built for the debug interpreter,
  3. as well as the stripped debugging symbols for use with gdb

    • We will call this package python-foo-dbg.

  4. Full unit tests (where available) after the build process for both the regular an the debug extensions.

debian/control

Here is an example for how debian/control would look like including all of the above:

Source: python-foo
Section: python
Priority: optional
Maintainer: Foo Bar <foo.bar@example.com>
Build-Depends:
    python-all-dev,
    python-all-dbg,
    debhelper (>= 8.1.0~),
    python-support (>= 0.90),
    python-bar,
    python-bar-dbg
Standards-Version: 3.9.2

Package: python-foo
Architecture: any
Depends:
    ${python:Depends},
    ${misc:Depends},
    ${shlibs:Depends},
    python-bar
Description: Python support for FOO
 This is the long package description.

Package: python-foo-dbg
Section: debug
Priority: extra
Architecture: any
Depends:
    ${python:Depends},
    ${misc:Depends},
    ${shlibs:Depends},
    python-foo (= ${binary:Version})
Recommends:
    python-dbg,
    python-bar-dbg
Description: Python support for FOO (debug extension)
 This is the long package description for the debug package.

Build

With python-all-dbg in Build-Depends, dh_auto_build in debian/rules will automatically run the build process twice for every supported Python version: first with the debug interpreter, then with the regular interpreter.

Assuming that

  1. python2.6 and python2.7 are present,

  2. that python-foo supports both,

  3. that python2.7 is the default Python version and that

  4. dh_auto_build detects a distutils-based setup,

dh_auto_build in debian/rules will perform (additional options passed by dh_auto_build have been omitted here for brevity):

Unit Tests

Unit tests for all of the built extensions are performed by overriding dh_auto_test in debian/rules and running them manually. For each run, an appropriate PYTHONPATH must be set. Assuming that the unit tests are run with the script mytests.py within the top directory, one way to accomplish this could be:

# Get the supported Python versions
PYVERS = $(shell pyversions -r -v)

override_dh_auto_test:
ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))
        set -e -x;\
        for py in $(PYVERS); do \
                PYTHONPATH=$(CURDIR)/build/lib.*-$$py  python$$py mytests.py ;\
                PYTHONPATH=$(CURDIR)/build/lib.*-$$py-pydebug python$$py-dbg mytests.py ;\
        done
endif

Install

Just as for #Build above, dh_auto_install in debian/rules will automate most of the installation process for us (additional options passed by dh_auto_install have been omitted here for brevity):

Because our source package is now building multiple binary packages, dh_auto_install will install everything to debian/tmp by default. We therefore need to move those files to their intended destinations. Specifically, everything should go into python-foo except for *_d.so: those go into python-foo-dbg. Two possible ways to accomplish this are:

Option 1: *.install files

Option 2: override_dh_install

If your source package doesn't build any binary packages other than python-foo and python-foo-dbg, a simpler approach can be used. It does not require *.install files; overriding dh_install in debian/rules is sufficient:



The last step is to include the stripped debugging symbols in the -dbg package, which dh_strip will automatically do for us with the --dbg-package option.

You currently still need to manually rename a directory within the installation path, namely pyshared to pymodules. Lintian will warn about that, see 576014

override_dh_strip:
ifeq (,$(filter nostrip,$(DEB_BUILD_OPTIONS)))
        # Stripped symbols go into python-foo-dbg
        dh_strip --dbg-package=python-foo-dbg
        
        # Rename pyshared to pymodules
        cd debian/python-foo-dbg/usr/lib/debug/usr/lib && mv pyshared pymodules
endif

Results

See Also

Alternatives

As previously mentioned, the above assumes that you are developing for unstable, and has corresponding package dependencies for debhelper et al. If this is not the case, you may need to modify some of the steps according to the rules below.

debhelper (<< 8.1.0)

At build time, if your package contains any scripts, the shebang will be changed to #!/usr/bin/python-dbg, which you'll have to change back manually (see 589759). The following workaround supplied by Joey Hess should work in those cases:

python (<< 2.6)

Building debug extensions for Python versions 2.5 and earlier or 2. requires a bit of hackery because the pathname for the debug build directory changed in 2.6 (from lib_d.* to lib.*-pydebug).

Assuming that the unit tests are run with the script mytests.py within the top directory:

# Get the supported Python versions
PYVERS = $(shell pyversions -r -v)

# Callable functions to determine the correct PYTHONPATH
pythonpath = $$(ls -d $(CURDIR)/build/lib.*-$(1))
pythonpath_dbg = $$(ls -d $(CURDIR)/build/lib_d.*-$(1) 2>/dev/null || ls -d $(CURDIR)/build/lib.*$(1)-pydebug)

override_dh_auto_test:
ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))
        set -e -x;\
        for py in $(PYVERS); do \
                PYTHONPATH=$(call pythonpath,$$py) python$$py mytests.py ;\
                PYTHONPATH=$(call pythonpath_dbg,$$py) python$$py-dbg mytests.py ;\
        done
endif


CategoryDebianDevelopment