Differences between revisions 82 and 83
Revision 82 as of 2016-11-04 01:04:55
Size: 18677
Editor: eriberto
Revision 83 as of 2017-01-09 21:04:18
Size: 19056
Editor: BarryWarsaw
Deletions are marked like this. Additions are marked like this.
Line 191: Line 191:
sphinx-build might try to access the Internet to fetch intersphinx inventory files; {{{http_proxy}}} set to will prevent that. sphinx-build might try to access the Internet to fetch intersphinx inventory files; {{{http_proxy}}} (and {{{https_proxy}}}?) set to will prevent that.

Another option is to add a patch to use locally installed intersphinx inventory files, but this will be specific to each package. There's no general patch that works for everything, and your package might not even use intersphinx mappings.
[[https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830186|Debian bug #830186]] describes one such approach.

Style Guide for Packaging Python Libraries

NOTE: Also see Python/AppStyleGuide for alternative single package debian/rules.

This is a wiki page so of course you are free to edit it. However, if you are making changes in recommendations, it is requested that you first contact the debian-python mailing list to discuss your changes, unless you're sure they are uncontroversial (e.g. fixing typos). Thanks!

The purpose of this document is to provide a quick style guide for packaging new Python libraries, or adding Python 3 support to existing packages (assuming that upstream supports it too). It is not intended to supplant the Debian Python Policy and in fact, if you have not read that document, stop now and go read it. Instead, this page captures in example form, best practices for simple Python libraries that you might find on the Python Package Index. A few assumptions about the Python package you are working on are being made:

  • The package is setuptools/distribute based. It should have a working setup.py (or for >= Python 3.3 setup.cfg) file and should generally be installable without modification into a virtualenv.

  • Pure-Python or simple extension modules only. If you can do python setup.py install you'll be fine. For more complex building instructions, you will have to adapt these guidelines.

  • The package supports at least Python 2.6.
  • The package has a test suite.
  • The package has some Sphinx-based documentation.

For packages which already support Python 2 and want to add Python 3 support, and/or modernize the packaging as described in this page, it's easiest to do so if your package already uses debhelper (v8 or newer), dh_python2 explicitly, and source format 3.0 (quilt).

The design of this document is to capture best-practices that you can cargo cult into your own packages. It will forever be a work in progress.


There are many ways to build Python libraries for Debian, and each package presents its own corner cases and challenges. In modern Debian (e.g. unstable as of November 2013) and derivatives, the easiest way to build Python library packages is with the pybuild debhelper build system. This link has details on pybuild, including more examples for your cargo culting pleasure.

If you can't use pybuild, say because you want to be able to backport your package to distribution versions before pybuild was introduced, you may be interested in the historical version of this document, which contains all the gory details you had to implement manually, but which pybuild now takes care of for you.

The examples below are taken from the enum34 package, but modified for generality.

Note: If you are building a Python 3-only package (i.e. there's no Python 2 support) then it is highly recommended you explicitly set the dh build system to pybuild. By default, if there is a setup.py, dh will set the build system to python_distutils which only supports Python 2 and you'll get errors during package build as it tries to invoke some nonexistent Python 2 tools.



One very important thing to get right is the Build-Depends line in the source package stanza. setuptools/distribute-based packages have the nasty habit of downloading dependencies from PyPI if they are needed at python setup.py build time. If the package is available from the system (as would be the case when Build-Depends is up-to-date), then distribute will not try to download the package, otherwise it will try to download it. This is a huge no-no, and pybuild internally sets the http_proxy and https_proxy environment variables (to to prevent this from happening.

dh_python2 and dh_python3 will correctly fill in the installation dependencies (via ${python:Depends} and ${python3:Depends} respectively), but it cannot fill in the build dependencies. Take extra care in getting this right, and double check your build logs for illegal access to pypi.python.org.

You'll want to have at least the following build dependencies:

  • debhelper (>= 10)

  • dh-python
  • python-all
  • python-setuptools

If the upstream package supports Python 3, you'll need these dependencies to also build a Python 3 version:

  • python3-all
  • python3-setuptools

If the package has documentation in reStructuredText format, which often uses Sphinx as well, add these dependencies:

  • python-docutils
  • python-sphinx

Python versions

If upstream supports Python 3, you'll want both of these lines in the source package stanza:

X-Python-Version: >= 2.6
X-Python3-Version: >= 3.2

If the d/control file already support a Python 2 version of the package, but you now want to add Python 3 support, copy the Python 2 binary package stanza and paste it below for a Python 3 version. Change the binary package name prefix from python- to python3-. Change the Depends line in the new stanza from including ${python:Depends} to ${python3:Depends}. Change the short description of the stanzas to make it clear which binary package is the Python 2 version and which is the Python 3 version. Here is an example of the two stanzas you might have:

Package: python-foo
Architecture: all
Depends: ${python:Depends}, ${misc:Depends}
Suggests: python-foo-doc
Description: Python frobnicator (Python 2)
 This package frobnicates a Python-based doohicky so that you no longer
 need to kludge a dingus to make the doodads work.
 This package installs the library for Python 2.

Package: python3-foo
Architecture: all
Depends: ${python3:Depends}, ${misc:Depends}
Suggests: python-foo-doc
Description: Python frobnicator (Python 3)
 This package frobnicates a Python-based doohicky so that you no longer
 need to kludge a dingus to make the doodads work.
 This package installs the library for Python 3.

Documentation package

Usually the same upstream documentation will serve equally well for users of the Python 2 and Python 3 runtime packages. Notice the Suggests line above. If this is the case, it's best to put those files into a separate python-foo-doc package, like so:

Package: python-foo-doc
Architecture: all
Section: doc
Depends: ${sphinxdoc:Depends}, ${misc:Depends}
Description: Python frobnicator (common documentation)
 This package frobnicates a Python-based doohicky so that you no longer
 need to kludge a dingus to make the doodads work.
 This is the common documentation package.


The contents of this file depends on whatever build system you choose, however we highly recommend the pybuild build system for its simplicity (at least for setup-based packages, i.e. the majority of those on PyPI) and its great support for dual Python 2 and Python 3 builds.

Here is a debian/rules file for you to start with. First, I'll show you the whole thing, then I'll explain it line by line.

#! /usr/bin/make -f

#export DH_VERBOSE = 1
export PYBUILD_NAME = foo

        dh $@ --with python2,python3 --buildsystem=pybuild

The file starts with the standard #! line. I like adding the (commented out when uploading) DH_VERBOSE line because it can make build problems easier to debug.

The next line is:

export PYBUILD_NAME = foo

pybuild supports a number of variables which can be used to control such things as the destination directory for build artifacts for both the Python 2 and Python 3 builds. The defaults generally do the right thing, but you do need to at least tell pybuild the name of your package. Here, we're telling it that the name is foo. This should match the module name, so for example, in enum34, you'd see:

export PYBUILD_NAME = enum34

even though the binary packages that get produced are python-enum34 and python3-enum34.

The next line is the standard debhelper-based catch-all rule which is used to get the whole build started:

        dh $@ --with python2,python3 --buildsystem=pybuild

What's important to note here is that both of the dh_python2 and dh_python3 helpers are being invoked, and also that the build system that dh will use is pybuild. There's a lot of magic packed into this line, and the pybuild manpage goes into more detail, but essentially what this does is:

  • Builds the Python 2 package for all supported Python 2 versions.
  • Builds the Python 3 package for all supported Python 3 versions.
  • Automatically detects the command to run the test suite.
  • Runs the Python 2 test suite.
  • Runs the Python 3 test suite.

Once all that's done, it properly installs all the files into binary packages.

If upstream does not yet support Python 3, omit the python3 helper for now.

And that's it! You usually won't need debian/python-foo.install or debian/python3-foo.install files even if you have multiple binary packages, because again, pybuild does the magic for you. You might need these if there are some non-standard installation tricks you need to implement.

You may or may not need this file. It can be useful for Sphinx-based documentation, by providing the original reST files used to build the HTML documentation. Here is an example debian/python-foo.links file (it does not need a Python 3 version).

usr/share/doc/python-foo/html/_sources usr/share/doc/python-foo/rst

Sphinx documentation

If your package has Sphinx documentation, add the sphinxdoc helper like so:

        dh $@ --with python2,python3,sphinxdoc --buildsystem=pybuild

If you need to build the Sphinx documentation (usually from .rst or .md files), add:

override_dh_auto_build: export http_proxy=
override_dh_auto_build: export https_proxy=
        PYTHONPATH=. sphinx-build -N -bhtml docs/ build/html # HTML generator
        PYTHONPATH=. sphinx-build -N -bman docs/ build/man # Manpage generator

sphinx-build might try to access the Internet to fetch intersphinx inventory files; http_proxy (and https_proxy?) set to will prevent that.

Another option is to add a patch to use locally installed intersphinx inventory files, but this will be specific to each package. There's no general patch that works for everything, and your package might not even use intersphinx mappings. Debian bug #830186 describes one such approach.


Make sure that you have a working debian/watch file, for referencing upstream distribution tarballs. Usually, uscan is sufficient to download tarballs from the declarations in debian/watch (or, if you're using svn-buildpackage, this can be done automatically with svn-buildpackage --svn-download-orig), but if the source needs to be repackaged, you should then provide a get-orig-source (policy) target in debian/rules which does the job.

The URLs many existing packages have been using are, or will soon, be broken. PyPI changed the way index listings for package directories work and in fact, these will disappear at some point in the future. Thus, it is highly recommended that you update your existing packages to the new debian/watch format described here.

Since probably for most of our packages, the original tarball comes from the Python Package Index (a.k.a. PyPI or the Cheeseshop), here is a sample debian/watch file that you might be able to adapt for your own package:

opts=uversionmangle=s/(rc|a|b|c)/~$1/ \

Notice the use of pypi.debian.net instead of pypi.python.org. We're now running a redirector for PyPI to adjust for changes in the format of package listings on the latter site. In fact, it's trivial to get a valid debian/watch file for your package. Just do this from the parent of your debian directory:

$ curl -o debian/watch http://pypi.debian.net/mypackage/watch


If you need to override the dh_python2/3 defaults, do it with an override method in your debian/rules file. Something like this will work:

        dh_python3 --shebang=/usr/bin/python3

If you need to override the default testing regime (e.g. to use py.test), you can do it one of two ways. Either:


or you can override the dh_auto_test rule:

        dh_auto_test -- --test-pytest

Building python -dbg packages

Debian packages two versions of the Python interpreter, one configured for production use, and one configured for debugging purposes. Extensions have to built for each of these, they are binary incompatible. The python -dbg packages should contain

  • the unstripped extension built for the debug version of the Python interpreter.
  • the stripped symbols of the extension built for the normal Python interpreter, found in /usr/lib/debug.

you have to override the dh_strip command as in:

        dh_strip -ppython-foo  --dbg-package=python-foo-dbg
        dh_strip -ppython3-foo --dbg-package=python3-foo-dbg

This doesn't call dh_strip for the python -dbg packages (leaves these unstripped), and puts the debug information from the normal packages into the corresponding -dbg packages.


It's good idea to supply CI (DEP-8) tests for package. These tests check to make sure your built module can be imported, and that they are working in case any runtime dependencies change. autodep8 have support for Python packaging. To enable it, just add to debian/control:

Testsuite: autopkgtest-pkg-python

This will check if Python{,3} module, named same as package, can be imported from system. You could write more complex test yourself.

You should check these tests locally by installing autopkgtest and autodep8 package. Then run autopkgtest command (see the manpage for details).


Executables and library packages

Let's say you have a Python package which results in Python 2 and 3 libraries, and a Python 3 executable. What is the best practices for naming and organizing your binary packages?

Clearly you want to at least have:

  • python-foo -- the Python 2 library
  • python3-foo -- the Python 3 library

but where should the /usr/bin/foo script go? You could put it in python3-foo but you CANNOT put it in python-foo or for that matter any binary package that starts with the python- prefix. dh_python3 refuses to rewrite shebang lines for any executable in a binary package that doesn't start with "python3-". (Similar constraints apply for dh_python2 and packages that don't start with "python-", and dh_pypy for packages that don't start with "pypy-".) This means that something like python-foo-cli or python-foo-bin is unacceptable.

Here are some recommendations. We do not have a standard (though maybe we should):

  • foo - this is nice because it parallels the /usr/bin/foo executable. In all likelihood, if the /usr/bin script name doesn't collide, then the foo binary package won't either.

  • python-foo - if the script is Python 2.x compatible.

  • python3-foo - if the script is Python 3.x compatible.

Python 3.3/3.4 unittest fixers for 2to3

Python 3.4 added a new 2to3 fixer which rewrites deprecated unittest assert methods to the current preferred names. If your upstream package uses 2to3 to provide a Python 3 compatible version, and uses these deprecated assert names, what this means is that the Python 3.3 and Python 3.4 version of those files will differ. dh_python3 will rightly refuse to copy files to the common location (i.e. /usr/lib/python3/dist-packages/<pkg>) when the files differ, so you'll end up with Python 3.4 specific paths in your python3- binary package in addition to the common location. lintian will complain.

There are a few ways to solve this, and maintainers may have their own preferences:

  • You can add a debian/<pkg>.pyremove file containing the paths to remove (but see bug 741381). E.g. the contents of python3-genshi.pyremove are:

/genshi/* 3.4
/Genshi*.egg-info/* 3.4
  • You could add an override in your d/rules file which does the removal. Something like this perhaps:

        rm -rf debian/python3-foo/usr/lib/python3.?
  • You could add a quilt patch that renames the deprecated methods in the upstream source. This would make 2to3 produce the same output for both Python 3.3 and 3.4 and avoid the problem altogether.

MANIFEST.in issues and missing files

If the upstream package requires some non-standard files to be installed, it must include a MANIFEST.in file. Sometimes these files are omitted from the original tarball because they're mostly useful for creating the sdist. Case in point: a package contained a bunch of doctests in .rst files but python setup.py sdist does not include .rst files by default. Thus, when pybuild went to build the package locally, the .rst files were omitted and the test suite failed because it could not find the doctest.

The solution for this is to either ensure that upstream includes the MANIFEST.in file, or that you add one before pybuild runs.


  • describe egg related information, what should go in the binary package and what shouldn't.