Differences between revisions 373 and 388 (spanning 15 versions)
Revision 373 as of 2019-05-01 11:14:48
Size: 6484
Editor: HolgerLevsen
Comment: hint what salsa is
Revision 388 as of 2019-12-27 21:56:29
Size: 8031
Editor: ?AurélienCouderc
Comment: fix broken links below icons
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
{{{#!wiki important
Got a spare moment? Please migrate this [[https://reproducible-builds.org/|to our new webpages]]…}}}
Line 6: Line 9:
||<style="width: 25%;vertical-align: top;text-align: center">[[ReproducibleBuilds/About|{{attachment:ReproducibleBuilds/rb-about.png|About}}]] <<BR>> [[ReproducibleBuilds/About|About]] ||<style="width: 25%;vertical-align: top;text-align: center">[[ReproducibleBuilds/Howto|{{attachment:ReproducibleBuilds/rb-howto.png|Howto}}]] <<BR>> [[ReproducibleBuilds/Howto|Make a package reproducible]] ||<style="width: 25%;vertical-align: top;text-align: center">[[ReproducibleBuilds/Contribute|{{attachment:ReproducibleBuilds/rb-contribute.png|Contribute}}]] <<BR>> [[ReproducibleBuilds/Contribute|How to help]] ||<style="width: 25%;vertical-align: top;text-align: center">[[ReproducibleBuilds/ExperimentalToolchain|{{attachment:ReproducibleBuilds/rb-toolchain.png|Toolchain}}]] <<BR>> [[ReproducibleBuilds/ExperimentalToolchain|Experimental toolchain]] ||
||<style="width: 25%;vertical-align: top;text-align: center">[[ReproducibleBuilds/History|{{attachment:ReproducibleBuilds/rb-history.png|History}}]] <<BR>> [[ReproducibleBuilds/History|Project history]] ||<style="width: 25%;vertical-align: top;text-align: center">[[https://salsa.debian.org/reproducible-builds/|{{attachment:ReproducibleBuilds/rb-alioth.png|Salsa / Gitlab}}|class=]] <<BR>> [[https://salsa.debian.org/reproducible-builds/|Salsa project|class=]] ||<style="width: 25%;vertical-align: top;text-align: center">[[https://bugs.debian.org/cgi-bin/pkgreport.cgi?usertag=reproducible-builds@lists.alioth.debian.org|{{attachment:ReproducibleBuilds/rb-bugs.png|Bugs}}|class=]] <<BR>> [[https://bugs.debian.org/cgi-bin/pkgreport.cgi?usertag=reproducible-builds@lists.alioth.debian.org|Bug reports|class=]] ||<style="width: 25%;vertical-align: top;text-align: center">[[https://reproducible.debian.net/|{{attachment:ReproducibleBuilds/rb-jenkins.png|Jenkins}}|class=]] <<BR>> [[https://reproducible.debian.net/|Continuous integration|class=]] ||
||<style="width: 25%;vertical-align: top;text-align: center">[[ReproducibleBuilds/About|{{attachment:ReproducibleBuilds/rb-about.png|About}}]] <<BR>> [[ReproducibleBuilds/About|About]] ||<style="width: 25%;vertical-align: top;text-align: center">[[ReproducibleBuilds/Howto|{{attachment:ReproducibleBuilds/rb-howto.png|Howto}}]] <<BR>> [[ReproducibleBuilds/Howto|Make a package reproducible]] ||<style="width: 25%;vertical-align: top;text-align: center">[[https://reproducible-builds.org/contribute/|{{attachment:ReproducibleBuilds/rb-contribute.png|Contribute}}]] <<BR>> [[https://reproducible-builds.org/contribute/|How to help]] ||<style="width: 25%;vertical-align: top;text-align: center">[[ReproducibleBuilds/ExperimentalToolchain|{{attachment:ReproducibleBuilds/rb-toolchain.png|Toolchain}}]] <<BR>> [[ReproducibleBuilds/ExperimentalToolchain|Experimental toolchain]] ||
||<style="width: 25%;vertical-align: top;text-align: center">[[ReproducibleBuilds/History|{{attachment:ReproducibleBuilds/rb-history.png|History}}]] <<BR>> [[ReproducibleBuilds/History|Project history]] ||<style="width: 25%;vertical-align: top;text-align: center">[[https://salsa.debian.org/reproducible-builds/|{{attachment:ReproducibleBuilds/rb-alioth.png|Salsa / Gitlab}}|class=]] <<BR>> [[https://salsa.debian.org/reproducible-builds/|Salsa project / Gitlab|class=]] ||<style="width: 25%;vertical-align: top;text-align: center">[[https://bugs.debian.org/cgi-bin/pkgreport.cgi?usertag=reproducible-builds@lists.alioth.debian.org|{{attachment:ReproducibleBuilds/rb-bugs.png|Bugs}}|class=]] <<BR>> [[https://bugs.debian.org/cgi-bin/pkgreport.cgi?usertag=reproducible-builds@lists.alioth.debian.org|Bug reports|class=]] ||<style="width: 25%;vertical-align: top;text-align: center">[[https://reproducible.debian.net/|{{attachment:ReproducibleBuilds/rb-jenkins.png|Jenkins}}|class=]] <<BR>> [[https://reproducible.debian.net/|Continuous integration|class=]] ||
Line 26: Line 29:
 * [[DebianBug:894441|#894441: binNMUs should be replaced by easy "no-change-except-debian/changelog-uploads"]] (as of 20190302, 12% of all binaries in Buster were binNMUs.)
 * [[DebianBug:900837|#900837: Mass-rebuild of packages for reproducible builds]] (blocked by #894441)
 * [[DebianBug:774415|#774415]] devscripts: please add the srebuild wrapper for reproducible builds - package it standalone? https://salsa.debian.org/reproducible-builds/debian-rebuilder-setup/blob/master/builder/srebuild has another variant
 * User-facing interfaces (see proof-of-concept in [[DebianBug:863622|#863622: apt: warn when installing packages that are not reproducible]]
Line 31: Line 34:
 * [[DebianBug:900918|#900918: debian-installer: Please make the generated images reproducible]], see also [[DebianBug:920631|#920631]] and [[DebianBug:920676|#920676]]
 * User-facing interfaces (see proof-of-concept in [[DebianBug:863622|#863622: apt: warn when installing packages that are not reproducible]]
 * [[DebianBug:900837|#900837: Mass-rebuild of packages for reproducible builds]] (blocked by #894441):
 * [[DebianBug:894441|#894441: binNMUs should be replaced by easy "no-change-except-debian/changelog-uploads"]] (as of 20190302, 12% of all binaries in Buster were binNMUs.)
Line 34: Line 37:
 * currently debian-policy says "packages ''should'' be reproducible", though we aim for "packages ''must'' be reproducible" though it's still a long road until we'll be there: currently (Oct 2018) there are more than 1250 unreproducible packages in Buster, thus if policy would be changed today, 1250 packages would need to be kicked out of Buster (well, or fixed) immediatly, so this policy change right now is not feasable.  * Currently debian-policy says "packages ''should'' be reproducible", though we aim for "packages ''must'' be reproducible" though it's still a long road until we'll be there: currently (Oct 2018) there are more than 1250 unreproducible packages in Buster, thus if policy would be changed today, 1250 packages would need to be kicked out of Buster (well, or fixed) immediatly, so this policy change right now is not feasible.
Line 46: Line 49:
 * [[DebianBug:929397|#929397: ftp.d.o: please upload LTS .buildinfo files to ftp-master]] (this is not relevant yet, as Jessie is the LTS release, while only dpkg from Stretch and newer produces .buildinfo files.)

= Also related =

 * [[DebianBug:895346|#895346]] [devscripts] devscripts: dcmd --buildinfo is not documented
 * [[DebianBug:869567|#869567]] [devscripts] debsign: doesn't sign multiple .buildinfo in the same changes
 * [[DebianBug:898961|#898961]] [devscripts] dscverify: accept .buildinfo from a build with unsigned .dsc which later was signed
 * [[DebianBug:807270|#807270]] [devscripts] mk-origtargz: create reproducible tarballs and --mtime option
 * [[DebianBug:852365|#852365]] [sbuild] sbuild: append-to-version may overwrite incorrect .buildinfo
 * [[DebianBug:923987|#923987]] [sbuild] Should also send the buildinfo in the build mail
Line 49: Line 63:
= Next = = Even more =
Line 55: Line 69:
= Solved issues =

 * [[DebianBug:900918|#900918: debian-installer: Please make the generated images reproducible]], see also [[DebianBug:920631|#920631]] and [[DebianBug:920676|#920676]]
 * [[DebianBug:844431|#844431: debian-policy: packages should build reproducible]]
 * many more we fixed between 2014 and 2019, when this section was created. Hopefully some like those in dpkg will be added here eventually.
Line 56: Line 76:
CategoryDebianDevelopment
CategoryDeveloper CategoryPackaging

Got a spare moment? Please migrate this to our new webpages

https://reproducible-builds.org

It should be possible to reproduce, byte for byte, every build of every package in Debian. More information about reproducible builds in general are available at reproducible-builds.org.

About
About

Howto
Make a package reproducible

Contribute
How to help

Toolchain
Experimental toolchain

History
Project history

Salsa / Gitlab
Salsa project / Gitlab

Bugs
Bug reports

Jenkins
Continuous integration

Status

Reproducible builds of Debian as a whole is still not a reality, though individual reproducible builds of packages are possible and being done. So while we are making very good progress, it is a stretch to say that Debian is reproducible.

  • Most packages built in sid today are reproducible under a fixed build-path and environment.

  • We have a new control file *.buildinfo that records the build environment, see deb-buildinfo for reference. Older design drafts are here.

  • We have a continuous integration platform that builds and immediately rebuilds packages. With this we can detect problems related to timestamps, file ordering, CPU usage, (pseudo-)randomness and other things.

  • We are examining packages and sorting out common problems.

  • Many patches have already been submitted, and we are continuously writing new ones.

  • You can check which packages installed on your system are still unreproducible by using the reproducible-check script in the devscripts package.

Big outstanding issues

These are the critical items necessary to have reproducible builds for at least the required packages of Debian

Annoying but not major

Nice to have

  • Tighten up the Policy definition of "reproducible" to be stricter about environment variables and build paths.
  • Discuss which environment variables we should blacklist or whitelist, 876055.

  • #929397: ftp.d.o: please upload LTS .buildinfo files to ftp-master (this is not relevant yet, as Jessie is the LTS release, while only dpkg from Stretch and newer produces .buildinfo files.)

Also related

  • #895346 [devscripts] devscripts: dcmd --buildinfo is not documented

  • #869567 [devscripts] debsign: doesn't sign multiple .buildinfo in the same changes

  • #898961 [devscripts] dscverify: accept .buildinfo from a build with unsigned .dsc which later was signed

  • #807270 [devscripts] mk-origtargz: create reproducible tarballs and --mtime option

  • #852365 [sbuild] sbuild: append-to-version may overwrite incorrect .buildinfo

  • #923987 [sbuild] Should also send the buildinfo in the build mail

There are many other possible nice-to-haves, e.g., making builds independent of their build directory, making it possible to create archive formats (like tar.gz and zip) with different tools yet result in the same byte order, etc. Many of those are valuable, but they shouldn't distract from getting the results of reproducible builds out to users.

Even more

For more concrete tasks to be done, look at how to contribute.

Statistics from the continuous integration platform

Solved issues


CategoryDeveloper CategoryPackaging