This is a tutorial for git-pbuilder.

git-pbuilder is part of the package git-buildpackage and the usage is very similar to cowbuilder (see also cowbuilder).

git-pbuilder needs sudo rights!

Usage

First, an environment needs to be created, then it can be used by git-buildpackage, with, for instance, the gbp buildpackage --git-pbuilder option.

Initialization and Variants

You can create base images for all architectures (e.g. i386 or amd64) and distributions (e.g. Wheezy, Jessie or Sid (unstable), Ubuntu xxx, ...) that your hardware can run on.

Normal Usage

The easiest usage is to call no further options, that will create a Sid (unstable) build environment with the architecture you are currently running on (if you are using i386, the environment will also be created for i386):

git-pbuilder create

The base build image is created in /var/cache/pbuilder/base.cow/

Creating other Architecture

If you want to explicitly create an environment for i386 (while running on architecture amd64) you have to tell that git-pbuilder:

ARCH=i386 git-pbuilder create

The base build image is created in /var/cache/pbuilder/base-sid-i386.cow/

Creating Packages for other Distributions

Mainly packaging is done in the Sid (unstable) distribution. But sometimes you want to create packages for the stable or old-stable release, so you have to tell this also while git-pbuilder is creating the environment. If you want to build for the Wheezy distribution then you have to call git-pbuilder like this:

DIST=wheezy git-pbuilder create

The base image is created in /var/cache/pbuilder/base-wheezy-[your-platform].cow/. A base image for the (old-old-stable) Squeeze Distribution is created similar, just change the argument for DIST.

Using a Mirror

If you use git-pbuilder (or git-buildpackage) very often it's better to use a local mirror to save downloadtime and reduce traffic. If you have set up a local apt proxy the you can tell git-pbuilder to use it. Let's say you want to create a base image for Squeeze and architecture amd64 with a caching proxy with the IP 192.168.110.4 on port 3142 (like apt-cacher-ng), so it's a combination of all variants from above:

DIST=squeeze ARCH=amd64 git-pbuilder create --mirror=http://192.168.110.4:3142/ftp.de.debian.org/debian

The base build image is created in /var/cache/pbuilder/base-squeeze-amd64.cow/

Updating

If your build environment is quite old you have to update it before you can use it. To update the base image just run:

git-pbuilder update

The update for the possible other distributions or architectures is similar like the creation of it. Update the Wheezy environment for the same architecture you are currently running on would be:

DIST=wheezy git-pbuilder update

or for Squeeze on i386

DIST=squeeze ARCH=i368 git-pbuilder update

If you have used a mirror while creating the base images it will be used. So remember that if you have various networks you working, the update will fail if the mirror isn't reachable. To use another mirror you have to set the variable MIRRORSITE to a valid mirror site.

MIRRORSITE="http://10.101.0.10:3142/ftp.debian.org/debian" git-pbuilder update

Installing Extra Packages

Sometimes you have to install extra packages to the base image. This is helpful if you work off-line in some cases or you want to speed up the packaging. The workflow for that is similar to cowbuilder.

git-pbuilder login --save-after-login
# first step, update the package list
root@host:/# apt-get update
# then you can install any package
root@host:/# apt-get install vim screen less

You have to repeat these steps for every base image you use if you need the same behaviour in your various build environments.

DIST=wheezy ARCH=amd64 git-pbuilder login --save-after-login
...

Using Local Packages

Sometimes the package you are trying to build build-depends on a library you just packaged, and is not available in the official repositories. This page explains how to solve this with pbuilder in general. With git-pbuilder, after having created the /etc/pbuilderrc (or /root/.pbuilderrc) and D05deps as instructed, you need to call

git-pbuilder update --override-config

Tips

Use of eatmydata

You can install the package eatmydata to improve the speed of your builds.

git-pbuilder login --save-after-login
root@host:/# apt-get update
root@host:/# apt-get install eatmydata

Using ccache

If you are often building the same package with a big source then it is useful to speed up a second build with ccache. To do so you have to tell pbuilder the needed environment for the use of ccache inside the chroot. The ccache cache directory has to be placed somewhere in your file system, suggested place is /var/cache/pbuilder/ccache but you can also put it under /home/ccache for example in case you have more free space there. But please do not use a NFS or CIFS share! The executing right for this directory needs to be set to a+w so the user pbuilder (who the ccache will be run by) can create the needed subdirectories there. If the directory doesn't exist create and set/correct the permissions on it.

sudo mkdir -p /var/cache/pbuilder/ccache
sudo chmod a+w /var/cache/pbuilder/ccache

Next you have to tweak your /etc/pbuilderrc (or $HOME/.pbuilderrc). Fill in the following part.

export CCACHE_DIR="/var/cache/pbuilder/ccache"
export PATH="/usr/lib/ccache:${PATH}"
EXTRAPACKAGES="ccache"
BINDMOUNTS="${CCACHE_DIR}"

That's all, on the next run git-pbuilder will use ccache.

Changing standard ccache options

Without further options ccache will use a cache size of 1GB and endlessly amount of cached files. Depending on the package you build you will set up other maximums there. The needed caching size depends on the size of object files the build will produce. You have to investigate here. So maybe you wanna set the caching size to 4GB. This has to be done in the chroot, so the only way to do this is a hook script. You need a hook script of type A because it has to be set before the build starts. The script is quite easy, put the following context as file A10set_ccache_options in your hook directory.

# A10_set_ccache_options
# setting needed options to ccache
# possible options can be found on http://ccache.samba.org/manual.html#_options

# increase the ccache caching size
ccache -M 4G
# output the current statistics
ccache -s

With this hook script get a similar output right before build starts like this.

cache directory             /home/ccache
cache hit (direct)                     6
cache hit (preprocessed)               1
cache miss                           982
called for link                       57
called for preprocessing              26
compile failed                        18
preprocessor error                     8
bad compiler arguments                 2
unsupported source language            9
autoconf compile/link                133
unsupported compiler option            3
no input file                         24
files in cache                      2300
cache size                         619.5 Mbytes
max cache size                       4.0 Gbytes

Creating a specific base chroot

You can spend a lot of time with waiting for the prepared chroot if you are building packages with a big list of dependencies, even if the packages are cached inside the pbuilder apt directory. This is annoying and in case of developing and tuning the package unnecessary. The build would be much quicker if the used chroot has already installed all dependencies. As git-pbuilder can pass cowbuilder arguments as well the easiest way is to tell git-pbuilder which base chroot cowbuilder should use. But before that you have to create your desired chroot. To do this just copy the base directory to a new directory and name it as you want. You have to respect one rule, the new directory must start with 'base-'. So for example if you want to create a new base chroot based on the default sid/unstable chroot copy the /var/cache/pbuilder/buildd/base.cow to /var/cache/pbuilder/buildd/base-$your_package.cow.

sudo cp -a /var/cache/pbuilder/base.cow /var/cache/pbuilder/base-my_package.cow

Next you need to login into this new chroot and install all the needed dependencies persistently. For this you need a package list for apt-get or dpkg --set-selections.

DIST=my_package git-pbuilder login --save-after-login

Now install the needed packages and log out.

# apt-get install $(list of packages)
#     #or with `dpkg --set-selections`
# dpkg --set-selections < packagelist # created with 'dpkg --get selections \* > /tmp/packagelist'

You can now use this prepared chroot with the git-buildpackage option --git-dist=.

git-buildpackage --git-dist=my_package ... other options ...

Troubleshooting

Slow copying and removing of the COW directory

What cowbuilder does is:

cp -al /var/cache/pbuilder/base.cow /tmp/new
rm -rf /tmp/[new]

Of course cowbuilder uses a different location than /tmp/[new]. You need to optimize those 2 commands on your computer. They should take around 0.2s each. If not, try to use the ext3 filesystem, for more details, see our benchmarks.