Differences between revisions 1 and 8 (spanning 7 versions)
Revision 1 as of 2015-01-06 17:19:12
Size: 2684
Editor: Lunar
Comment: move old issues away from howto
Revision 8 as of 2016-05-09 19:46:13
Size: 2809
Editor: Infinity0
Comment: sphinx timestamps now use S_D_E
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
<<TableOfContents()>>
Line 10: Line 11:
We are thus going to make mandatory to build package in a directory named
like `/usr/src/debian/hello-2.8-1`.

As a bonus, this means that it will be easier to unpack packages in this
canonical location for use with tools looking at the source code like `gdb`.
We are thus making the build path part of the build environment, and record it in [[ReproducibleBuilds/BuildinfoSpecification|.buildinfo files]].
Line 23: Line 19:
== {data,control}.tar.{gz,xz,bz2} will store files in readdir order == == {data,control}.tar.{gz,xz,bz2} will store files in filesystem order ==
Line 25: Line 21:
This is dependent on an accident of filesystem layout at build time, so it would
sometimes not be reproducible.
Changes to DebianPts:dpkg are discussed in Bug:719845. [[http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=39;att=0;bug=719845|Test case patch for pkg-tests]]. [[http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=719845#54|Patches that fork `sort` to get a stable order for files in control and data archives]].
Line 28: Line 23:
We should probably fix this in dpkg by sorting the contents of the tar files. == Files generated by debhelper depend on filesystem order ==
Line 30: Line 25:
Changes are discussed in Bug:719845. [[http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=39;att=0;bug=719845|Test case patch for pkg-tests]]. [[http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=719845#54|Patches that fork `sort` to get a stable order for files in control and data archives]].


== Files generated by debhelper depend on readdir order ==

At least the shlibs files generated by dh_makeshlibs depends on the order returned by find(1). There might be other debhelper programs with the same issue.
Several components of DebianPts:debhelper generate output which depends on the filesystem order. See Bug:774100, Bug:774102, and Bug:775020.
Line 53: Line 43:
piuparts, which uses xmlto to generate documentation, is affected by this. Tentative fix is in the [[https://anonscm.debian.org/cgit/reproducible/libxslt.git/log/?h=pu/reproducible_builds|pu/reproducible_builds branch]].

== Files in data.tar contain timestamps ==

 * [[ReproducibleBuilds/TimestampInLongLinkTarMembers|“Long link” Tar members contain a timestamp]]
 * [[ReproducibleBuilds/TimestampInGhcInterfaces|Timestamps in GHC .hi files]]
 * [[ReproducibleBuilds/TimestampsInManpagesGeneratedBySphinx|Timestamps in manpages generated by Sphinx]]

Here's some details about issues that were preventing packages to build reproducibly and who has been fixed or worked around in Debian. This is mainly intended as a reference for developers and packagers working on other projects.

Files in data.tar.gz contains build paths

The build path is embedded in DWARF sections of ELF files among other types of file generated during builds. This has proven a real headache to fix after the path have been captured.

We are thus making the build path part of the build environment, and record it in .buildinfo files.

Generation of files in data.tar depends on (pseudo-)randomness

Now fixed:

{data,control}.tar.{gz,xz,bz2} will store files in filesystem order

Changes to dpkg are discussed in 719845. Test case patch for pkg-tests. Patches that fork `sort` to get a stable order for files in control and data archives.

Files generated by debhelper depend on filesystem order

Several components of debhelper generate output which depends on the filesystem order. See 774100, 774102, and 775020.

Randomness in control file

Now fixed:

.deb ar-archive header contains a timestamp

.deb are ar-archives. The header currently contains the “current time”.

759999 contains patches against dpkg that will preset the timestamp to the time of the latest entry of debian/changelog when a package is built using dpkg-buildpackage.

XSLT generate-id() is non-deterministic

XSLT's generate-id() function is explicitly allowed by the XSLT spec to be non-deterministic, and is frequently implemented using memory addresses of XML nodes, which are of course non-deterministic thanks to ASLR. Consequentially, files that are generated by XSLT (typically documentation) that include the result of generate-id() in their output do not build deterministically.

Tentative fix is in the pu/reproducible_builds branch.

Files in data.tar contain timestamps