#language en ~-[[DebianWiki/EditorGuide#translation|Translation(s)]]: English-~ ---- <> == Links == [[Python|Main Python page]] == Introduction == Pybuild is a newer (as in 2013) and more modular build system for Python packages. Features: * Build Python2, Python3, PyPy with a one-liner dh command * Support nose/pytest and tox testing Note: New Python2 packages should not be uploaded to unstable. See [[Python/2Removal|Python2 Removal]] == debian/control == {{{dh-python}}} is a mandatory build dependency. The pybuild dh integration script (not pybuild itself) uses the {{{Build-Depends}}} to figure out which interpreters it should build for. If the package provides Python 2.X modules it should build-depend on python-all (or python-all-dev in case of Python extensions), if it builds extensions with debug symbols: python-all-dbg; if it provides Python 3 script - python3 needs to be added to Build-Depends, etc. If you want to provide only python3 packages you can build-depend on python3-all (python3-all-dev, python3-all-dbg). One very important thing to get right is the {{{Build-Depends}}} line in the source package stanza. setuptools/distribute-based packages have the nasty habit of downloading dependencies from PyPI if they are needed at {{{python setup.py build}}} time. If the package is available from the system (as would be the case when {{{Build-Depends}}} is up-to-date), then distribute will not try to download the package, otherwise it will try to download it. This is a huge no-no, and pybuild internally sets the {{{http_proxy}}} and {{{https_proxy}}} environment variables (to {{{http[s]://127.0.0.1:9/}}}) to prevent this from happening. {{{dh_python2}}} and {{{dh_python3}}} will correctly fill in the installation dependencies (via {{{${python:Depends}}} and {{{${python3:Depends}}} respectively), but it cannot fill in the build dependencies. Take extra care in getting this right, and double check your build logs for illegal access to {{{pypi.python.org}}}. == debian/rules == Here is a {{{debian/rules}}} file for you to start with. First, I'll show you the whole thing, then I'll explain it line by line. {{{ #!/usr/bin/make -f #export DH_VERBOSE=1 export PYBUILD_NAME=foo %: dh $@ --with python2,python3 --buildsystem=pybuild }}} The file starts with the standard {{{#!}}} line. I like adding the (commented out when uploading) {{{DH_VERBOSE}}} line because it can make build problems easier to debug. The next line is: {{{ export PYBUILD_NAME=foo }}} pybuild supports a number of variables which can be used to control such things as the destination directory for build artifacts for both the Python 2 and Python 3 builds. The defaults generally do the right thing, but you do need to at least tell pybuild the name of your package. Here, we're telling it that the name is {{{foo}}}. This should match the module name, so for example, in enum34, you'd see: {{{ export PYBUILD_NAME=enum34 }}} even though the binary packages that get produced are {{{python-enum34}}} and {{{python3-enum34}}}. The next line is the standard debhelper-based catch-all rule which is used to get the whole build started: {{{ %: dh $@ --with python2,python3 --buildsystem=pybuild }}} What's important to note here is that both of the {{{dh_python2}}} and {{{dh_python3}}} helpers are being invoked, and also that the ''build system'' that dh will use is pybuild. There's a lot of magic packed into this line, and the pybuild manpage goes into more detail, but essentially what this does is: * Builds the Python 2 package for all supported Python 2 versions. * Builds the Python 3 package for all supported Python 3 versions. * Automatically detects the command to run the test suite. * Runs the Python 2 test suite. * Runs the Python 3 test suite. Once all that's done, it properly installs all the files into binary packages. You usually won't need {{{debian/python-foo.install}}} or {{{debian/python3-foo.install}}} files even if you have multiple binary packages, because again, pybuild does the magic for you (if PYBUILD_NAME is correctly set). You might need these if there are some non-standard installation tricks you need to implement. === CUSTOMIZATION === Pybuild can be configured via command line or CDBS-like - by exporting environment variables in debian/rules that look like this: {{{ PYBUILD_OPTION PYBUILD_OPTION_$interpreter PYBUILD_OPTION_$interpreter$version }}} For example: * If you don't want to use debian/*.install files you can set DESTDIR for each interpreter (note that you can use "python2" for all python2.X interpreters and "python3" for all python3.X ones, this doesn't include -dbg interpreters, though): {{{ export PYBUILD_DESTDIR_python2=debian/python-foo/ export PYBUILD_DESTDIR_python2-dbg=debian/python-foo-dbg/ export PYBUILD_DESTDIR_python3=debian/python3-foo/ export PYBUILD_DESTDIR_python3-dbg=debian/python3-foo-dbg/ export PYBUILD_DESTDIR_python3.1-dbg=debian/spam-pkg-with-python3.1-only/ export PYBUILD_DESTDIR_pypy=debian/pypy-foo/ }}} * If you want to add some custom arguments to Python 3's install command: {{{ export PYBUILD_INSTALL_ARGS_python3=--install-scripts=/usr/share/foo/ }}} * You can disable (almost) any action via --disable / DISABLE env. var: {{{ export PYBUILD_DISABLE=configure export PYBUILD_DISABLE_python3.1=test }}} * If you want to disable Build-Depends detection: {{{ export PYBUILD_INTERPRETERS=python{version} python{version}-dbg export PYBUILD_VERSIONS=2.7 3.3 }}} * If auto detection didn't detect the right build system: {{{ export PYBUILD_SYSTEM=distutils }}} * If you want to add a custom command before or after each iteration: {{{ export PYBUILD_BEFORE_BUILD=echo {version} >> '{dir}/enabled' export PYBUILD_AFTER_INSTALL=rm -rf '{destdir}/{install_dir}/foo/tests' }}} * If your build system is not yet supported, you can use custom build system: {{{ export PYBUILD_SYSTEM=custom export PYBUILD_CLEAN_ARGS=rm -rf {build_dir}/python{version}/ export PYBUILD_CONFIGURE_ARGS=./configure --python={version} export PYBUILD_BUILD_ARGS=make build --dir={build_dir} --python={version} export PYBUILD_INSTALL_ARGS=make --destdir={destdir} --python={version} --install-dir={install_dir} export PYBUILD_TEST_ARGS_python2=cd {build_dir}; nosetests export PYBUILD_TEST_ARGS_python3=cd {build_dir}; python{version} -m discover }}} * If your package contains tests that are not installed/converted by distutils yet need 2to3 conversion: {{{ export PYBUILD_BEFORE_TEST=cp -r {dir}/tests {build_dir} export PYBUILD_BEFORE_TEST_python3=cp -r {dir}/tests {build_dir}; 2to3 -w {build_dir}/tests export PYBUILD_AFTER_TEST=rm -rf {build_dir}/tests }}} * To disable a specific test (pytest), note the `'-k $exp'` {{{ export PYBUILD_TEST_ARGS=-k 'not test_that_needs_internet' }}} * To drop more than one test add additional tests for exclusion by using `and not test_foo` to the variable. {{{ export PYBUILD_TEST_ARGS=-k 'not test_1 and not test_2' }}} * To [[https://www.debian.org/doc/packaging-manuals/python-policy/index.html#document-python-programs|package a program]] instead of a module: {{{ export PYBUILD_INSTALL_ARGS=--install-lib=/usr/share/packagename/ --install-scripts=/usr/share/packagename/ }}} == Resources == * Pybuild [[https://manpages.debian.org/testing/dh-python/pybuild.1.en.html|manpage]] * Pybuild announcement [[https://lists.debian.org/debian-python/2013/01/msg00009.html|email]] Examples of packages using Pybuild on codesearch: [[http://codesearch.debian.net/search?q=path%3Adebian%2Frules+pybuild|search]] introduction to pybuild and Python packaging talk at [[https://summit.debconf.org/debconf14/meeting/77/introduction-to-pybuild-and-python-packaging/|DebConf14]]: [[http://annex.debconf.org/debconf-share/debconf14/slides/introduction_to_pybuild_and_Python_packaging.pdf|slides]], [[http://meetings-archive.debian.net/pub/debian-meetings/2014/debconf14/webm/introduction_to_pybuild_and_Python_packaging.webm|video]]