Differences between revisions 26 and 27
Revision 26 as of 2014-09-30 11:54:27
Size: 11364
Editor: ?Beuc
Comment: use 'apt-get source'
Revision 27 as of 2014-09-30 13:21:19
Size: 11445
Editor: ?Beuc
Comment: Use a subshell rather than pushd/popd
Deletions are marked like this. Additions are marked like this.
Line 206: Line 206:
pushd apr-1.5.1/
dch --bpo "No changes."
pdebuild --debbuildopts '-v1.4.6-3+deb7u1' \
  --use-pdebuild-internal --buildresult /usr/src/backports/wheezy/ \
  --pbuildersatisfydepends /usr/lib/pbuilder/pbuilder-satisfydepends-experimental \
  -- --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz \
     --bindmounts /usr/src/backports/wheezy/ \
     --hookdir /usr/src/backports/wheezy/
popd
(
    c
d apr-1.5.1/
    dch --bpo "No changes."
    pdebuild --debbuildopts '-v1.4.6-3+deb7u1' \
        --use-pdebuild-internal --buildresult /usr/src/backports/wheezy/ \
        --pbuildersatisfydepends /usr/lib/pbuilder/pbuilder-satisfydepends-experimental \
        -- --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz \
           --bindmounts /usr/src/backports/wheezy/ \
           --hookdir /usr/src/backports/wheezy/
)
Line 216: Line 217:
pushd apr-util-1.5.3/
dch --bpo "No changes."
pdebuild --debbuildopts '-v1.4.1-3' \
  --use-pdebuild-internal --buildresult /usr/src/backports/wheezy/ \
  --pbuildersatisfydepends /usr/lib/pbuilder/pbuilder-satisfydepends-experimental \
  -- --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz \
     --bindmounts /usr/src/backports/wheezy/ \
     --hookdir /usr/src/backports/wheezy/
popd
(
    c
d apr-util-1.5.3/
    dch --bpo "No changes."
    pdebuild --debbuildopts '-v1.4.1-3' \
        --use-pdebuild-internal --buildresult /usr/src/backports/wheezy/ \
        --pbuildersatisfydepends /usr/lib/pbuilder/pbuilder-satisfydepends-experimental \
        -- --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz \
           --bindmounts /usr/src/backports/wheezy/ \
           --hookdir /usr/src/backports/wheezy/
)
Line 228: Line 230:
pushd apache2-2.4.10/
dch --bpo "Note: depends on backported libapr."
#dch -a "Others changes you made."
pdebuild --debbuildopts '-v2.2.22-13+deb7u3' \
  --use-pdebuild-internal --buildresult /usr/src/backports/wheezy/ \
  --pbuildersatisfydepends /usr/lib/pbuilder/pbuilder-satisfydepends-experimental \
  -- --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz \
     --bindmounts /usr/src/backports/wheezy/ \
     --hookdir /usr/src/backports/wheezy/
popd
(
    c
d apache2-2.4.10/
    dch --bpo "Note: depends on backported libapr."
    pdebuild --debbuildopts '-v2.2.22-13+deb7u3' \
        --use-pdebuild-internal --buildresult /usr/src/backports/wheezy/ \
        --pbuildersatisfydepends /usr/lib/pbuilder/pbuilder-satisfydepends-experimental \
        -- --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz \
           --bindmounts /usr/src/backports/wheezy/ \
           --hookdir /usr/src/backports/wheezy/
)

This page describes how to compile a backport in the "proper" way, one which tries to respect the rules for the backports.

This assumes you will build a backport that will be uploaded to the official backports repository. For simpler recipes, look for SimpleBackportCreation for your own informal backports or AutomateBackports for your own personnal backporting archive.

Using pbuilder

Your first step is to configure pbuilder and, optionnally, cowbuilder. The remaining of this howto will assume you will use cowbuilder, as this is what is in use here and is more efficient. We also like to use git-buildpackage to avoid building a .dsc file first, so to build straight from the source.

A full pbuilder or cowbuilder is beyond the scope of this manual, but there are specific issues you should not miss here. See git-pbuilder for details of use of cowbuilder/pbuilder with got-buildpackage.

Modifying the package

To make a proper backport, you need to modify the package according to the rules. A simpler version:

  • Use wheezy-backports or squeeze-backports as distribution.

  • Append ~bpo${debian_release}+${build_int} to the version number (consider using dch --bpo), e.g. 1.2.3-4 now becomes 1.2.3-4~bpo70+1 for wheezy or 1.2.3-4~bpo60+1 for squeeze.

  • Include all changelog entries since the last version on debian-backports or since stable if it's the first version (using -v).

This is just a summary of the technical aspects of the rules. It is really important that you read up on the rules and follow them.

Installing the toolchain

apt-get install cowbuilder pbuilder

Building for different distributions

Then enable pbuilder to build for different distributions easily, by following those instructions.

At this point you should be able to build a package using:

DIST=squeeze ARCH=amd64 pbuilder build -sa -v1.0 foo.dsc

With git-buildpackage this becomes:

cd package
DIST=squeeze ARCH=amd64 git-buildpackage --git-builder=git-pbuilder  -sa -v1.0

Note that this option can be written to your ~/.gbp.conf file as such: builder = /usr/bin/git-pbuilder.

Another example, with cowbuilder:

DIST=wheezy ARCH=i386 pdebuild --pbuilder cowbuilder --debbuildopts "-sa -v1.0"

Uploading the backport

The main upload queue now accepts backports directly, so this configuration isn't required anymore.

In dput.conf:

[bpo]
fqdn = backports-master.debian.org
incoming = /pub/UploadQueue/
method = ftp
login = anonymous
allow_dcut = 1

Then upload away to bpo. All DDs should have access to that repository, but need to ask first, on Request Tracker.

Building multi-dependencies packages

At this point you are able to build and upload backports of existing packages easily. The tricky part comes up when you have multiple dependencies to upload at once. pbuilder chroots do not use packages from backports by default, so that's the first thing we need to fix. Add this to your pbuilderrc:

OTHERMIRROR="deb http://www.backports.org/debian/ $DIST-backports main"

Then update the chroot with the new sources.list line:

sudo DIST=squeeze ARCH=amd64 cowbuilder --update --override-config

Then you could build the packages one at a time: backport one, upload it, wait for it to show up in the backports archive, build the second one, etc. But this is really time consuming and could take a long time for big package suites. It could also mean a lot of dependent packages be uploaded to backports while the package that needs it is not there, which is bad practice.

What you need is to be able to use the packages you're building locally in the chroot. This is again described in the PbuilderTricks page, but we'll show our own way here.

Add this to your pbuilderrc:

OTHERMIRROR="deb http://www.backports.org/debian/ $DIST-backports main|deb file:///home/anarcat/dist/build-area/ ./"
HOOKDIR=/usr/lib/pbuilder/hooks
BINDMOUNTS="/home"

The above assumes you are building your packages in /home/anarcat/dist/package and the build results end up in /home/anarcat/dist/build-area/. This can be freely changed, but it will only work if it is under the BINDMOUNTS.

Then you need to add a hook that will run apt-get update before the package is built, in /usr/lib/pbuilder/hooks/D90update:

/usr/bin/apt-get update

The HOOKDIR variable can be changed if you want to put your hook file somewhere else, but the file name is important.

Then the only bit missing is making sure the /home/anarcat/dist/build-area/ directory is a valid archive. Other howtos make that part of the hook, but since we are not running as root (see below), this is not practical for us. We simply use the following command on a as-needed basis:

dpkg-scanpackages . /dev/null | bzip2 -c  > Packages.bz2

It is in the following makefile to make things easier:

all: Packages.bz2 Sources.bz2

Packages.bz2::
        dpkg-scanpackages . /dev/null | bzip2 -c  > Packages.bz2

Sources.bz2::
        dpkg-scansources . /dev/null | bzip2 -c  > Sources.bz2

Other tricks

We have found the following options to be useful in pbuilderrc:

PDEBUILD_PBUILDER="cowbuilder"

BUILDUSERID="500"
BUILDUSERNAME="anarcat-pbuilder"

AUTO_DEBSIGN=yes

This is the ~/.gbp.conf (git-buildpackage config):

[DEFAULT]
# tell git-buildpackage howto clean the source tree
cleaner = fakeroot debian/rules clean
postbuild = lintian $GBP_CHANGES_FILE
# this is how we invoke pbuilder, arguments passed to git-buildpackage will be
# passed to dpkg-buildpackge in the chroot
builder = /usr/bin/git-pbuilder

[git-buildpackage]
export-dir = ../build-area/

[git-import-orig]
dch = False

Self-contained example for Apache 2.4

# Prepare local repo
mkdir -p /usr/src/backports/wheezy/
touch /usr/src/backports/wheezy/Packages
cat <<'EOF' > /usr/src/backports/wheezy/D70results
#!/bin/bash
# Make sure local repo is updated before building
cd /usr/src/backports/wheezy/
dpkg-scanpackages . /dev/null > Packages
# It would be better to sign but that's complex in the context of pbuilder
# Also 'pbuilder --allow-untrusted' only works with the default satisfydepends
echo 'APT::Get::AllowUnauthenticated "true";' > /etc/apt/apt.conf.d/99AllowUnauthenticated
apt-get update
EOF
chmod 755 /usr/src/backports/wheezy/D70results

# Create initial environment
sudo pbuilder --create --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz \
  --distribution wheezy \
  --othermirror "deb http://security.debian.org/ wheezy/updates main|deb http://http.debian.net/debian wheezy-backports main|deb file:///usr/src/backports/wheezy ./" \
  --bindmounts /usr/src/backports/wheezy/
# Update regularly:
# sudo pbuilder --update --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz --bindmounts /usr/src/backports/wheezy/

# Setup identity
export DEBEMAIL="you@debian.org" 
export DEBFULLNAME="Your Name"

# Configure build
#export DEB_BUILD_OPTIONS="parallel=$(nproc) nocheck"
export DEB_BUILD_OPTIONS="parallel=$(nproc)"

# Add source for 'apt-get source'
echo "deb-src http://ftp.fr.debian.org/debian/ jessie main contrib" \
  | sudo tee /etc/apt/sources.list.d/jessie-src.list
sudo apt-get update

# Dependencies to add in our local repo
apt-get source apr/jessie
(
    cd apr-1.5.1/
    dch --bpo "No changes."
    pdebuild --debbuildopts '-v1.4.6-3+deb7u1' \
        --use-pdebuild-internal --buildresult /usr/src/backports/wheezy/ \
        --pbuildersatisfydepends /usr/lib/pbuilder/pbuilder-satisfydepends-experimental \
        -- --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz \
           --bindmounts /usr/src/backports/wheezy/ \
           --hookdir /usr/src/backports/wheezy/
)
apt-get source apr-util/jessie
(
    cd apr-util-1.5.3/
    dch --bpo "No changes."
    pdebuild --debbuildopts '-v1.4.1-3' \
        --use-pdebuild-internal --buildresult /usr/src/backports/wheezy/ \
        --pbuildersatisfydepends /usr/lib/pbuilder/pbuilder-satisfydepends-experimental \
        -- --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz \
           --bindmounts /usr/src/backports/wheezy/ \
           --hookdir /usr/src/backports/wheezy/
)

# Apache 2.4 itself
apt-get source apache2/jessie
(
    cd apache2-2.4.10/
    dch --bpo "Note: depends on backported libapr."
    pdebuild --debbuildopts '-v2.2.22-13+deb7u3' \
        --use-pdebuild-internal --buildresult /usr/src/backports/wheezy/ \
        --pbuildersatisfydepends /usr/lib/pbuilder/pbuilder-satisfydepends-experimental \
        -- --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz \
           --bindmounts /usr/src/backports/wheezy/ \
           --hookdir /usr/src/backports/wheezy/
)

# Sadly there's no hook to do that *after* the packages are installed in --buildresult
(cd /usr/src/backports/wheezy && dpkg-scanpackages . /dev/null > Packages)

If something goes wrong and you need to test manually:

sudo pbuilder --login --basetgz /var/cache/pbuilder/base-wheezy-bpo.tar.gz --bindmounts /usr/src/backports
apt-get update
echo 'APT::Get::AllowUnauthenticated "true";' > /etc/apt/apt.conf.d/99AllowUnauthenticated
cd /usr/src/backports/sources/apache2-2.4.10/
apt-get install pbuilder devscripts fakeroot
/usr/lib/pbuilder/pbuilder-satisfydepends-experimental
debuild ...

Notes:

  • That's for the low-level technical part. But as discussed it turns out pushing a major Apache version backport wasn't such a good idea :)

  • Not using cowbuilder, since it complexifies the commands, and since a few packages fail to build in hard-links corner-case situations 1

See also



CategoryPackaging