36044
Comment:
|
36171
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
{{{#!wiki important Got a spare moment? Please migrate this [[https://reproducible-builds.org/|to our new webpages]]…}}} |
Got a spare moment? Please migrate this to our new webpages…
With free software, anyone can inspect the source code for malicious flaws. But Debian provide binary packages to its users. The idea of “deterministic” or “reproducible” builds is to empower anyone to verify that no flaws have been introduced during the build process by reproducing byte-for-byte identical binary packages from a given source.
More information about reproducible builds in general are available at reproducible-builds.org.
Contents
Why do we want reproducible builds?
- Allow independent verifications that a binary matches what the source intended to produce.
- Should reproducible uploads become mandatory, then the incentive of an attacker to compromise the system of a developer with upload rights is lowered because it is not anymore possible for the developer to upload a binary that does not match the uploaded sources.
- Additionally, the incentive for this kind of attack is further lowered because an attacker now has to compromise all machines that can check the reproducibility of the uploaded source.
- Finally, with a sufficiently large body of independent (geographically and administratively) machines, reproducible builds can help find systems which are compromised in a way to produce binaries with altered functionality.
Help Multi-Arch: same packages co-installation (as they need every matching file to be byte identical).
- Be able to generate debug symbols for packages which do not have a “debug package”.
- Ensure packages can be built from source. The archive could be made to only accept reproducible uploads: the maintainer would stop uploading .deb files but keep them referenced in the .changes. A build would then build the source. Only if the hash matches the upload gets accepted.
- Allow file-level deduplication on Debian mirror sites, or maybe snapshots.d.o, of .deb files whose contents didn't really change between versions.
- Allow .deb deltas to be smaller.
Packages with build profiles must offer the exact same functionality for all profiles. Reproducible builds could be use to verify that it is the case.
Making sure that Architecture:all packages are build identically on different build architectures.
Validate cross-builds against native builds.
- Find embedded code copies (when packages should be reproducible because a toolchain package got fixed but are not because they use an embedded copy instead)
- Run builds in environments that trace things like system calls, file system or network access for QA or general analytical purposes. Reproducible builds help to ensure that the used tracing method had no influence on the produced binary.
allow diverse double compilation to verify compiler integrity: if one can compile gcc with gcc and clang (and any other compiler in Debian capable of compiling gcc) and then recompile gcc again with the gcc compiler packages created in the first step. The resulting packages should be bit by bit identical.
- diverse double compile (bootstrap) a base Debian on different distributions to make sure that also secondary build input like coreutils and C library are not affected by the Ken Thompson problem (this also allows to verify that hardware is not compromised using cross compilation on different platforms)
- proprietary binary blobs can verify that they are used by unmodified free software (example: Firefox Encrypted Media Extensions binary blob requires an untampered-with Firefox)
- Allow Debian package maintainers to verify that their packaging related changes to a source package (like switching the build system to debhelper/dh or upgrading the compat level) do not introduced unexpected side effects.
Reproducing builds
There are two sides to the problem: the build environment needs to be recorded during the initial build, and the same environment needs to be reproduced for later rebuilds.
Recording the environment
Information on a build will be recorded in a new control file with extension `.buildinfo`.
Reproduce the build environment
This is work-in-progress.
See Fun with buildinfo (2017).
See also srebuild (2015). The srebuild program is a sbuild wrapper which finds a timestamp from snapshot.debian.org containing all versions of the binary packages in a .buildinfo file and then carries out the build with the right versions installed.
References
Reproducibility in science: as defined by Wikipedia
Gitian: a secure software distribution method
Bazel: build tool that specifies all direct build dependencies
http://rbm.boklm.eu/: the Reproducible Build Manager
- Deterministic virtual machines:
Reflections on Trusting Trust, by Ken Thompson
Fully Countering Trusting Trust through Diverse Double-Compiling (DDC) - a PhD dissertation on how to use reproducible builds to counter the "trusting trust" attack on compilers
Is that really the source code for this software? by Jos van den Oever on blogs.kde.org (2013-06-19). Compare reproducing tar from the Debian, Fedora and OpenSUSE packages.
Deterministic Builds Part Two: Technical Details by Mike Perry
Verifying the source code for binaries by Jake Edge in Linux Weekly News.
Colin Watson's answer on ubuntu-devel to “Will Ubuntu use "reproducible builds" as debian is planning to do?”
guardianproject wiki:
Why and How of Reproducible Builds: Distrusting Our Own Infrastructure for Safer Software Releases, Seth Schoen and Mike Perry at Mozilla San Francisco, 2014-11-05
Challenges and implications of verifiable builds for security-critical open-source software by Xavier de Carné de Carnavalet and Mohammad Mannan, in ACSAC '14
The CIA Campaign to Steal Apple’s Secrets by Jeremy Scahill and Josh Begley, in The Intercept. Specifically (S//NF) Strawhorse: Attacking the MacOS and iOS Software Development Kit
Spy agencies target mobile phones, app stores to implant spyware by Amber Hildebrandt and Dave Seglins
Dave Cheney on Reproducible Builds in Go at GDG Berlin Golang, 20 April 2015
Triangle of Secure Code Delivery, Taylor Hornby, July 24, 2014
- Misc. upstream discussions:
Octave: bug report and mailing list thread
groff: mailing list thread
GHC (Glasgow Haskell Compiler): #4012
Groovy: GROOVY-6308
coreboot: mailing list thread and patches
libreboot: Feature #16: Reproducible Builds
mono and dotnet: https://github.com/dotnet/roslyn/commit/04462c44e30dfa91267581abdb029f3102796486, https://github.com/mono/ikvm-fork/commit/cd4bed9dd6540c380177c5b9c72f4d020f1b138f and https://github.com/mono/mono/commit/a803d17038c0fcc8b40b12744801a87ceddb15ba
Mozilla projects: Firefox/Iceweasel Thunderbird/Icedove
- Misc. distribution discussions:
OpenWrt: mailing list thread
NetBSD has a MKREPRO build switch (see http://man.netbsd.org/HEAD/usr/share/man/html5/mk.conf.html) and MKREPRO_TIMESTAMP should be set to SOURCE_DATE_EPOCH. Also see http://gnats.netbsd.org/50116, http://gnats.netbsd.org/50120 and http://gnats.netbsd.org/50121.
Guix has set up a challenge where every user can verify that the binaries from the server correctly correspond to the sources.
openSUSE are discussing reproducible builds too but are used to a different meaning of the term. mail 1, mail 2
https://github.com/hardenedlinux/grsecurity-reproducible-build
More in the publicity and Presentations sections.
Publicity
This section lists URLs, people, and dates for when other people have publicly expressed interest, or shared information about, the project.
Mike Perry, 2013-08-20: Deterministic Builds Part One: Cyberwar and Global Compromise
Jake Edge, 2013-08-21: Security software verifiability
Holger Levsen, 2014-09-26: Reproducible builds? I never did any - manually
Lunar, 2014-12-29: Reproducible builds against RC bugs
Lunar, 2015-01-15: 80%
Jake Edge, 2015-01-21: Lots of progress for Debian's reproducible builds (discussion on LWN, HN, reddit)
Kristian Kißling, 2015-01-27: Debian bringt Reproducible Builds voran in linux-magazin.de (Note: the article contains several misunderstandings.)
Hanno Böck, 2015-02-02: "Vertrauen durch reproduzierbare Build-Prozesse" pages 1, 2 in Golem.de (German)
Hans-Joachim Baader, 2015-02-16: Reproduzierbare Builds in Debian nähern sich in pro-linux.de (German)
2015-02-16: Debian Project Reaches 83% Reproducible Builds for Source Packages in softpedia.com.
Tannhausser, 2015-02-17: Debian mejora su seguridad con los reproducible builds in La Mirada del Replicante (Spanish)
Darren Pauli, 2015-02-23: Debian on track to prove binaries' origins in The Register
Arun, 2015-02-24: Debian working on reproducible builds in ?TuxDiary
2015-02-24: Debian está trabajando en compilaciones reproducibles in Detrás del pingüino (Spanish)
2015-02-27: Debian security initiative for reproducible builds reaches milestone in ?TechRepublic by James Sanders
2015-05-06: http://www.linux-magazin.de/Ausgaben/2015/06/Reproducible-Builds in Linux-Magazin 06/2015 by Daniel Stender (in German)
2015-05-10: Reproducible builds on Debian for GSoC 2015 by dhole
2015-09-06: How Debian Is Trying to Shut Down the CIA and Make Software Trustworthy Again, J.M. Porup, Motherboard
2015-09-10: How Debian and other open-source projects are making software more trustworthy, Chris Hoffman, PCWorld.
2016-01-21: Hello tests.reproducible-builds.org by h01ger
2016-03-31: Reproducible Signal builds for Android (though it's not bit by bit identical and needs an apkdiff tool to claim it's reproducibility)
2016-04-04: Establishing Correspondence Between an Application and its Source Code - How Combining Two Completely Separate Open Source Projects Can Make Us All More Secure by Emily Ratliff
2017-05-03: Construcciones Reproducibles, in Software Gurú magazine, number 54 (Mexico), by Gunnar Wolf
Weekly reports
Stretch cycle
2015-05-03: Reproducible builds: first week in Stretch cycle by Lunar
2015-05-11: Reproducible builds: week 2 in Stretch cycle by Lunar
2015-05-17: Reproducible builds: week 3 in Stretch cycle by Lunar
2015-05-25: Reproducible builds: week 4 in Stretch cycle by Lunar
2015-06-01: Reproducible builds: week 5 in Stretch cycle by Lunar
2015-06-08: Reproducible builds: week 6 in Stretch cycle by Lunar
2015-06-15: Reproducible builds: week 7 in Stretch cycle by Lunar
2015-06-22: Reproducible builds: week 8 in Stretch cycle by Lunar
2015-06-29: Reproducible builds: week 9 in Stretch cycle by Lunar
2015-07-07: Reproducible builds: week 10 in Stretch cycle by Lunar
2015-07-12: Reproducible builds: week 11 in Stretch cycle by Lunar
2015-07-20: Reproducible builds: week 12 in Stretch cycle by Lunar
2015-07-26: Reproducible builds: week 13 in Stretch cycle by Lunar
2015-08-03: Reproducible builds: week 14 in Stretch cycle by Lunar
2015-08-10: Reproducible builds: week 15 in Stretch cycle by Lunar
2015-08-16: Reproducible builds: week 16 in Stretch cycle by Lunar
2015-08-25: Reproducible builds: week 17 in Stretch cycle by Lunar
2015-09-01: Reproducible builds: week 18 in Stretch cycle by Lunar.
2015-09-06: Reproducible builds: week 19 in Stretch cycle by Lunar.
2015-09-14: Reproducible builds: week 20 in Stretch cycle by Lunar.
2015-09-21: Reproducible builds: week 21 in Stretch cycle by Lunar.
2015-09-27: Reproducible builds: week 22 in Stretch cycle by Lunar.
2015-10-04: Reproducible builds: week 23 in Stretch cycle by Lunar.
2015-10-14: Reproducible builds: week 24 in Stretch cycle by Lunar.
2015-10-18: Reproducible builds: week 25 in Stretch cycle by Lunar.
2015-10-26: Reproducible builds: week 26 in Stretch cycle by Lunar.
2015-11-02: Reproducible builds: week 27 in Stretch cycle by Lunar.
2015-11-09: Reproducible builds: week 28 in Stretch cycle by Lunar.
2015-11-15: Reproducible builds: week 29 in Stretch cycle by Lunar.
2015-11-23: Reproducible builds: week 30 in Stretch cycle by Lunar.
2015-12-01: Reproducible builds: week 31 in Stretch cycle by Lunar.
2015-12-11: Reproducible builds: week 32 in Stretch cycle by Lunar.
2015-12-14: Reproducible builds: week 33 in Stretch cycle by Lunar.
2015-12-20: Reproducible builds: week 34 in Stretch cycle by Lunar.
2016-01-03: Reproducible builds: week 35 in Stretch cycle by Lunar.
2016-01-03: Reproducible builds: week 36 in Stretch cycle by Lunar.
2016-01-14: Reproducible builds: week 37 in Stretch cycle by Lunar.
2016-01-17: Reproducible builds: week 38 in Stretch cycle by Lunar.
2016-01-24: Reproducible builds: week 39 in Stretch cycle by Lunar.
2016-02-02: Reproducible builds: week 40 in Stretch cycle by Lunar.
2016-02-08: Reproducible builds: week 41 in Stretch cycle by Lunar.
2016-02-14: Reproducible builds: week 42 in Stretch cycle by Lunar.
2016-02-21: Reproducible builds: week 43 in Stretch cycle by Lunar.
2016-03-05: Reproducible builds: week 44 in Stretch cycle by Lunar.
2016-03-10: Reproducible builds: week 45 in Stretch cycle by Lunar.
2016-03-14: Reproducible builds: week 46 in Stretch cycle by Lunar.
2016-03-21: Reproducible builds: week 47 in Stretch cycle by Lunar.
2016-03-26: Reproducible builds: week 48 in Stretch cycle.
2016-04-02: Reproducible builds: week 49 in Stretch cycle.
2016-04-09: Reproducible builds: week 50 in Stretch cycle.
2016-04-16: Reproducible builds: week 51 in Stretch cycle.
2016-04-23: Reproducible builds: week 52 in Stretch cycle.
2016-04-30: Reproducible builds: week 53 in Stretch cycle.
2016-05-07: Reproducible builds: week 54 in Stretch cycle.
2016-05-14: Reproducible builds: week 55 in Stretch cycle.
2016-05-14: Reproducible builds: week 55 in Stretch cycle.
2016-05-21: Reproducible builds: week 56 in Stretch cycle.
2016-05-28: Reproducible builds: week 57 in Stretch cycle.
2016-06-04: Reproducible builds: week 58 in Stretch cycle.
2016-06-11: Reproducible builds: week 59 in Stretch cycle.
2016-06-18: Reproducible builds: week 60 in Stretch cycle.
2016-06-25: Reproducible builds: week 61 in Stretch cycle.
2016-07-02: Reproducible builds: week 62 in Stretch cycle.
2016-07-23: Reproducible builds: week 65 in Stretch cycle.
2016-08-06: Reproducible builds: week 67 in Stretch cycle.
2016-08-13: Reproducible builds: week 68 in Stretch cycle.
2016-08-20: Reproducible builds: week 69 in Stretch cycle.
2016-08-27: Reproducible builds: week 70 in Stretch cycle.
2016-09-03: Reproducible builds: week 71 in Stretch cycle.
2016-09-10: Reproducible builds: week 72 in Stretch cycle.
2016-09-17: Reproducible builds: week 73 in Stretch cycle.
2016-09-24: Reproducible builds: week 74 in Stretch cycle.
2016-10-01: Reproducible builds: week 75 in Stretch cycle.
2016-10-08: Reproducible builds: week 76 in Stretch cycle.
2016-10-15: Reproducible builds: week 77 in Stretch cycle.
2016-10-22: Reproducible builds: week 78 in Stretch cycle.
2016-10-29: Reproducible builds: week 79 in Stretch cycle.
2016-11-05: Reproducible builds: week 80 in Stretch cycle.
2016-11-12: Reproducible builds: week 81 in Stretch cycle.
2016-11-19: Reproducible builds: week 82 in Stretch cycle.
2016-11-26: Reproducible builds: week 83 in Stretch cycle.
2016-12-03: Reproducible builds: week 84 in Stretch cycle.
2016-12-10: Reproducible builds: week 85 in Stretch cycle.
2016-12-17: Reproducible builds: week 86 in Stretch cycle.
2016-12-24: Reproducible builds: week 87 in Stretch cycle.
2016-12-31: Reproducible builds: week 88 in Stretch cycle.
2017-01-07: Reproducible builds: week 89 in Stretch cycle.
2017-01-14: Reproducible builds: week 90 in Stretch cycle.
2017-01-21: Reproducible builds: week 91 in Stretch cycle.
2017-01-28: Reproducible builds: week 92 in Stretch cycle.
2017-02-04: Reproducible builds: week 93 in Stretch cycle.
2017-02-11: Reproducible builds: week 94 in Stretch cycle.
2017-02-18: Reproducible builds: week 95 in Stretch cycle.
2017-02-25: Reproducible builds: week 96 in Stretch cycle.
2017-03-04: Reproducible builds: week 97 in Stretch cycle.
2017-03-11: Reproducible builds: week 98 in Stretch cycle.
2017-03-18: Reproducible builds: week 99 in Stretch cycle.
2017-03-25: Reproducible builds: week 100 in Stretch cycle.
GSoC 2015: akira
2015-06-07: GSoC 2015 Week 2: Move forward reproducible builds by akira
2015-06-15: GSoC 2015 Week 3: Move forward reproducible builds by akira
2015-06-21: GSoC 2015 Week 4: Move forward reproducible builds by akira
2015-06-29: GSoC 2015 Week 5: Move forward reproducible builds by akira
2015-07-06: GSoC 2015 Week 6: Move forward reproducible builds by akira
2015-07-13: GSoC 2015 Week 7: Move forward reproducible builds by akira
2015-07-20: GSoC 2015 Week 8: Move forward reproducible builds by akira
2015-07-27: GSoC 2015 Week 9: Move forward reproducible builds by akira
2015-08-03: GSoC 2015 Week 10: Move forward reproducible builds by akira
2015-08-03: GSoC 2015 Week 11: Move forward reproducible builds by akira
2015-08-19: GSoC 2015 Week 12: Move forward reproducible builds by akira
GSoC 2015: Dhole
2015-06-07: GSoC 2015 Week 2: Move forward reproducible builds by dhole
2015-06-14: GSoC 2015 Week 3: Move forward reproducible builds by dhole
2015-06-21: GSoC 2015 Week 4: Move forward reproducible builds by dhole
2015-06-28: GSoC 2015 Week 5: Move forward reproducible builds by dhole
2015-07-06: GSoC 2015 Week 6: Move forward reproducible builds by dhole
2015-07-13: GSoC 2015 Week 7: Move forward reproducible builds by dhole
2015-07-20: GSoC 2015 Week 8: Move forward reproducible builds by dhole
2015-07-25: GSoC 2015 Week 9: Move forward reproducible builds by dhole
2015-07-31: GSoC 2015 Week 10: Move forward reproducible builds by dhole
2015-08-07: GSoC 2015 Week 11: Move forward reproducible builds by dhole
2015-08-16: GSoC 2015 Week 12: Move forward reproducible builds by dhole
Related projects
CARE monitors the execution of the specified command to create an archive that contains all the material required to re-execute it in the same context.
Further work
Having reproducible builds allows us to trust binary packages better, because it becomes easier to have:
- diversity of build location and jurisdiction - build packages in more than one location, including the developer's
- diversity of build hardware, in case of hardware bugs, or malicious implants - a mix of VMs, some real hardware, different CPU manufacturers, different date of manufacture and supplier
- diversity of people - multiple signatures on a .changes file
- diversity of kernels, explained below
Kernel packages
Special features of kernel packages (including bootloaders and hypervisors) - GRUB2, Xen, linux, kfreebsd...
- we put huge trust in them - kernels are the ultimate target of any rootkit, able to completely hide from userland
- a kernel image built for amd64, if the build system is portable and reproducible enough, will be the same whether built from linux-amd64 or kfreebsd-amd64
- or maybe from different kernel versions - for example, a jessie build chroot on a wheezy host system
Then we would be better protected from something that could affect many systems at once, such as a kernel vulnerability; or widespread infection by a rootkit, which now must be compatible with more than one type of kernel to go unnoticed.