Got a spare moment? Please migrate this to our new webpages

#language en

Status of this page, post January 2018

/!\ Note: as of (at least) January 2018, this documentation appears to be obsolete. The process documented below did not work for a current unstable pbuilder enviroment, due to problems installing packages and assigning the appropriate locale. See, which points you at the reprotest tool on GitHub. The reprotest tool is apparently the correct mechanism to use today.

/!\ Current experiments are done in Debian unstable. We maintain a set of modified packages needed to make other packages reproducible. These packages allow to conduct experiments and test modifications locally or in our continuous integration platform. The goal is to get our changes accepted by the respective maintainers after proving them bug-free and fruitful.


/!\ This page is outdated/obsolete. Read the two paragraphs *above* and don't read anything below! ;)

Warning, again

Please read the stuff above.

APT repository

Our modified packages can be found in the following APT archive, which is signed by 49B6 5747 36D0 B637 CC37 01EA 5DB7 CA67 EA59 A31F:

deb ./
deb-src ./

Git repositories

Several Git repositories have been created on Alioth. Commit notifications are sent to a dedicated mailing list.

The QA infrastructure itself is managed in a different git repository: qa/

Straightforward patches — or packages not using Git as their version control system — have often been sent directly through the BTS.

Modified packages

The following packages have been modified to enable reproducible builds of other packages:


The pu/reproducible_builds branch adding BUILD_PATH_PREFIX_MAP support.

Please see for instructions how to build dpkg from git.

Usage example

If you don't have a pbuilder setup, you can install the package and create an unstable base:

sudo apt-get install pbuilder
sudo pbuilder --create --distribution unstable

If you have a pbuilder already setup, it's fairly easy to setup an environment with the custom toolchain:

## for pbuilder
sudo cp /var/cache/pbuilder/base.tgz /var/cache/pbuilder/base-reproducible.tgz
sudo pbuilder --login --save-after-exec --basetgz /var/cache/pbuilder/base-reproducible.tgz
## for cowbuilder
sudo cowbuilder --create --distribution sid --basepath /var/cache/pbuilder/base-reproducible.cow
sudo cowbuilder --login --save-after-exec --basepath /var/cache/pbuilder/base-reproducible.cow
## then, for both:
/usr/lib/apt/apt-helper download-file /etc/apt/trusted.gpg.d/ SHA256:9e2b4442528422882e9d364ca9e0a31e08b610775e986d2fdbf409198d06f4f4
echo 'deb ./' > /etc/apt/sources.list.d/reproducible.list
apt update
apt dist-upgrade
# locales-all is needed by
# disorderfs is needed by for detecting readdir-related issues
apt install locales-all disorderfs
exit 0 # exit the pbuilder/cowbuilder login shell

Once that's done, two options:

  1. Use the prebuilder script from the misc.git repository.

    To rebuild a source package, pass the name (or name stem) of its .dsc file:

     % ls *.dsc
     % /path/to/misc/prebuilder/ -b … hello
     % ls *.dsc
     % /path/to/misc/prebuilder/ -b … h*.dsc

    From a package source directory, dpkg-source -b ./ generates (or updates) the .dsc file in the parent directory.

  2. Manually, through the following process:

apt-get source --download-only acl
mkdir b1 b2
sudo DEB_BUILD_OPTIONS=nocheck pbuilder --build --debbuildopts '-b' --basetgz /var/cache/pbuilder/base-reproducible.tgz acl_*.dsc
dcmd cp /var/cache/pbuilder/result/acl_*.changes b1
sudo dcmd rm /var/cache/pbuilder/result/acl_*.changes
sudo DEB_BUILD_OPTIONS=nocheck pbuilder --build --debbuildopts '-b' --basetgz /var/cache/pbuilder/base-reproducible.tgz acl_*.dsc
dcmd cp /var/cache/pbuilder/result/acl_*.changes b2
sudo dcmd rm /var/cache/pbuilder/result/acl_*.changes

(for cowbuilder, change pbuilder to cowbuilder, --basetgz to --basepath and $path.tgz to $path.cow.

Important: this will likely produce false-positive results, packages which are not reproducible might appear reproducible, as several variations (eg TZ or the locales) are not introduced when building like this. So you really want to use the first option ;)

diffoscope is useful to check the result:

diffoscope --html $output_file b1/*.changes b2/*.changes

Adding a package to the APT archive

Somehow you need to copy the package files to /home/groups/reproducible/htdocs/debian/ on Remeber that you have to do a sourceful upload, with binaries and sources.

You can either use dcmd:

dcmd scp ../packagename_version\~reproducible_amd64.changes

or add the following snipped to your (old-style dput, but compatible also with dput-ng):

fqdn                    =
incoming                = /home/groups/reproducible/htdocs/debian/
method                  = scp
allow_dcut              = 0
allow_unsigned_uploads  = 0
allowed_distributions   = .*

then you can just use

dput reproducible ../packagename_version\~reproducible_amd64.changes

to upload the built package.

Then, on, run make from within /home/groups/reproducible/htdocs/debian/ to run apt-ftparchive and gpg to refresh the Release file, the Packages and Sources files

Guidelines for adding a package to the APT archive

Scheduling packages to rebuild on jenkins

Packages can be scheduled for immediate rebuild by anybody in the reproducible builds team by running the script in /srv/home/groups/reproducible/ on The script requires the mandatory -s parameter specifying the suite. Example:

/srv/home/groups/reproducible/ -s unstable mypackage1 mypackage2

It also allows some interesting tweaking of the scheduling (e.g. enable/disable notification at the start/end of the build, saving artifacts, etc) and allows selecting what packages to rebuild by doing a query itself. Run the script with the --help switch to see which options and filters are available.

Finding packages to schedule for rebuild

After a toolchain problem is fixed, the affected packages should be rebuilt. To find which packages are to be rebuilt, you need to calculate the intersection of all unreproducible packages and those which should now be fixed or have their debbindiff changed as a result of the toolchain upload to the experimental repository.

If the packages affected by the fixed issue are categorized under a single issue, you can just go on and do (or ask for) a rescheduling. The rescheduling script has enough power to get all unreproducible packages affected by a given issue.


1. get a list of all unreproducible source packages:

curl --location > reproducible.json
jq --raw-output '.[] | select(.status == "unreproducible") | .package' < reproducible.json | sort -u > all-unreproducible

2. get a list of all affected source packages by finding those that build-depend on the fixed binary package. For example, python-sphinx (you need to have the deb-src lines on your /etc/apt/sources.list):

grep-dctrl -F Build-Depends python-sphinx --or -F Build-Depends-Indep python-sphinx --or -F Build-Depends-Arch python-sphinx /var/lib/apt/lists/*_debian_dists_sid_main_source_Sources -s Package -n | sort -u > all-affected

3. calculate the intersection:

comm -12 all-affected all-unreproducible > schedule-to-rebuild

Other way to get a list of packages to schedule are grepping for some patter the .buildinfos or debbindiff files. For that you need either to download all the files locally (the misc.git repository contains scripts to keep a synced copy of those files) or ask to somebody with shell access to reproducible.d.n to run the grep for you.

Once in a way or another you obtain a list of packages to reschedule, you can now post it on IRC to get a rebuild triggered as soon as somebody with the required privileges reads your message.

You should not immediately reschedule packages for rebuild after a toolchain upload, since the build chroot has to be updated (or regenerated) to notice the fixed package versions. The chroot gets updated every 4 hours. Also it's better to check whether the chroot has knowledge of the new package by scheduling a single small affected packages, and check its .buildinfo to see whether the fixed package has been picked up.