This is a tutorial for git-pbuilder.

git-pbuilder is part of the package git-buildpackage and the usage is very similar to cowbuilder (see also cowbuilder).

git-pbuilder needs sudo rights!

Usage

First, an environment needs to be created, then it can be used by git-buildpackage, with, for instance, the gbp buildpackage --git-pbuilder option.

Initialization and Variants

You can create base images for all architectures (mostly i386 or amd64) and distribution (mostly Squeeze, Wheezy or Testing/SID, Ubuntu xxx, ...) that your hardware can run on.

Normal Usage

The easiest usage is to call no further options, that will create a Testing/SID build environment with the architecture you are currently running on (if you are using i386, the environment will also be created for i386):

git-pbuilder create

The base build image is created in /var/cache/pbuilder/base.cow/

Creating other Architecture

If you want explicitly create a environment for i386 (but running on architecture amd64) you have to tell that git-pbuilder:

ARCH=i386 git-pbuilder create

The base build image is created in /var/cache/pbuilder/base-sid-i386.cow/

Creating Packages for other Distributions

Mainly package creating is done in the Testing distribution. But sometimes you want to create packages for the stable or old-stable release, so you have to tell this also while git-pbuilder is creating the environment. If you want to build for the Wheezy distribution then you have to call git-pbuilder like this:

DIST=wheezy git-pbuilder create

The base image is created in /var/cache/pbuilder/base-wheezy-[your-platform].cow/. The creating of a base image for the (old-old-stable) Squeeze Distribution is similar, just change the argument for DIST.

Using a Mirror

If you use git-pbuilder (or git-buildpackage) very offen it's better to use a local mirror to save downloadtime and reduce traffic. If you have set up a local Apt proxy the you can tell git-pbuilder to use it. Let's say you want to create a base image for Squeeze and architecture amd64 with a caching proxy with the IP 192.168.110.4 on port 3142 (like apt-cacher-ng), so it's a combination of all variants from above:

DIST=squeeze ARCH=amd64 git-pbuilder create --mirror=http://192.168.110.4:3142/ftp.de.debian.org/debian

The base build image is created in /var/cache/pbuilder/base-squeeze-amd64.cow/

Updating

If the date of creation of your build environment is quite older you have to update it before you can use it. To update the base image just run:

git-pbuilder update

The update for the possible other distributions or architectures is similar like the creation of it. Update the Wheezy environmnt for the same architecture you are currently running on would be:

DIST=wheezy git-pbuilder update

or for Squeeze on i386

DIST=squeeze ARCH=i368 git-pbuilder update

If you have used a mirror while creating the base images it will be used! So remember that if you have various networks you working, the update will fail if the mirror isn't reachable.

Installing Extra Packages

Sometimes you have to install extra packages to the base image. This is helpful if you work offline in some cases or you want to speed up the packaging. The workflow for that is similar to cowbuilder.

git-pbuilder login --save-after-login
# first step, update the package list
root@host:/# apt-get update
# then you can install any package
root@host:/# apt-get install vim screen less

You have to repeat this steps for every base image you use if you needs the same behaviour in your various build environments.

DIST=wheezy ARCH=amd64 git-pbuilder login --save-after-login
...

Using Local Packages

Sometimes the package you are trying to build-depends on a library you just packaged, and is not available in the official repositories. This page explains how to solve this with pbuilder in general. With git-pbuilder, after having created the /etc/pbuilderrc (or /root/.pbuilderrc) and D05deps as instructed, you need to call

git-pbuilder update --override-config

Tips

Use of eatmydata

You can install the package eatmydata to improve the speed of your builds.

git-pbuilder login --save-after-login
root@host:/# apt-get update
root@host:/# apt-get install eatmydata

Using ccache

If you building often the same package which has a bigger source then it useful to speed up a second build with ccache. To do so you have to tell pbuilder the needed environment for the use of ccache inside the chroot. The ccache cache directory has to be placed somethere in your file system, suggest place is /var/cache/pbuilder/ccache but you can put it also under /home/ccache for example in case you have more free space there. But please do not use a NFS or CIFS share! The executing right for this directory needs to be set to a+w so the user pbuilder (which the ccache will run) can create the needed subdirectories there. If not exists create the directory and set/correct the permissions on it.

sudo mkdir -p /var/cache/pbuilder/ccache
sudo chmod a+w /var/cache/pbuilder/ccache

Next you have to tweak your /etc/pbuilderrc (or $HOME/.pbuilderrc). Fill in the following part.

export CCACHE_DIR="/var/cache/pbuilder/ccache"
export PATH="/usr/lib/ccache:${PATH}"
EXTRAPACKAGES="ccache"
BINDMOUNTS="${CCACHE_DIR}"

That's all, next run while using git-pbuilder it will use ccache.

Changing standard ccache options

Without further options ccache will use a cache size of 1GB and endlessly amount of cached files. Depending on the package you build you will set up other maximums there. The needed caching size depends on the size of object files the build will produce. You have to investigate here. So maybe you wanna set the caching size to 4GB. This has to done in the chroot, so the only way to do this is a hook script. You need a hook script from typ A because the setting has to be set before the build starts. The script is right easy, put the following context as file A10set_ccache_options in your hook directory.

# A10_set_ccache_options
# setting needed options to ccache
# possible options can be found on http://ccache.samba.org/manual.html#_options

# increase the ccache caching size
ccache -M 4G
# output the current statistics
ccache -s

With this hook script get a similar output right before build starts like this.

cache directory                     /home/ccache
cache hit (direct)                     6
cache hit (preprocessed)               1
cache miss                           982
called for link                       57
called for preprocessing              26
compile failed                        18
preprocessor error                     8
bad compiler arguments                 2
unsupported source language            9
autoconf compile/link                133
unsupported compiler option            3
no input file                         24
files in cache                      2300
cache size                         619.5 Mbytes
max cache size                       4.0 Gbytes

Creating a specific base chroot

You can spend a lot of time with waiting for the prepared chroot if you building packages with a big list of dependencies, even if the packages are cached inside the pbuilder apt directory. This annoying and in case of developing an tuning the package unnecessary. The build would be much quicker if the used chroot has already installed all dependencies. As git-pbuilder can pass cowbuilder arguments as well the easiest way is to tell git-pbuilder which base chroot cowbuilder should use. But before that you have to create your desired chroot. To do this just copy the base directory to a new directory and name it as you want. You have to respect on rule, the new directory must start with 'base-'. So for example if you want to create a new base chroot based on the default sid/unstable chroot copy the /var/cache/pbuilder/buildd/base.cow to /var/cache/pbuilder/buildd/base-$your_package.cow.

sudo cp -a /var/cache/pbuilder/base.cow /var/cache/pbuilder/base-my_package.cow

Next you need to login into this new chroot and install all the needed dependencies persistently. For this you need a package list for apt-get or dpkg --set-selections.

DIST=my_package git-pbuilder login --save-after-login

Now install the needed packages and log out.

# apt-get install $(list of packages)
#     #or with `dpkg --set-selections`
# dpkg --set-selections < packagelist # created with 'dpkg --get selections \* > /tmp/packagelist'

You can now use this prepared chroot with the option --git-dist= of git-buildpackage.

git-buildpackage --git-dist=my_package ... other options ...

Troubleshooting

Slow copying and removing of the COW directory

What cowbuilder does is:

cp -al /var/cache/pbuilder/base.cow /tmp/new
rm -rf /tmp/[new]

Of course cowbuilder uses a different location than /tmp/[new]. You need to optimize those 2 commands on your computer. They should take around 0.2s each. If not, try to use the ext3 filesystem, for more details, see our benchmarks.