18472
Comment: Reported libxmlb issue upstream
|
18767
Added information about the required size and requirements for the mount point
|
Deletions are marked like this. | Additions are marked like this. |
Line 28: | Line 28: |
NB: The GNOME image requires about 15GB, so you'll need sufficient memory. If that is not available, use a fast disc (e.g. SSD) instead. * The mount point for the image needs to support `mknod` and must allow the execution within the `chroot`, so it must be mounted with `dev` and `exec`. |
Reproducible Live images
This page tracks the progress on creating reproducible live images, using live-build from DebianLive
Contents
Configuration
The computer that is used for generating live images:
- Debian/sid
apt-cacher-ng running on http://localhost:3142
The most recent checkout of the git repository of live-build (https://salsa.debian.org/live-team/live-build)
An additional line in /etc/apt-cacher-ng/acnf.conf: VfilePatternEx: /project/trace/ftp-master\.debian\.org$
Snapshot
In order to get a more reproducible image, a snapshot is used. At the time of focusing on Bullseye, the timestamp 20210101T083123Z (1609489883) was chosen.
Considerations
Building an image must be fast, because a lot of rebuilds will be required -> use /dev/shm
- NB: The GNOME image requires about 15GB, so you'll need sufficient memory. If that is not available, use a fast disc (e.g. SSD) instead.
The mount point for the image needs to support mknod and must allow the execution within the chroot, so it must be mounted with dev and exec.
Reduce network traffic, don't download the same files over-and-over -> use apt-cacher-ng
While looking for (non-)reproducibility, the command line option --parent-mirror-binary is used. Official live images should not use this command line option (and will therefore use deb.debian.org in /var/lib/apt/lists)
Preparing the build environment
All command lines must be executed as root.
export LIVE_BUILD=/home/roland/git/live-build export SOURCE_DATE_EPOCH=1609489883 cd /dev/shm mount /dev/shm -odev,exec,remount mkdir live cd live
Building images for Debian Buster
Timestamp: 20210210T031935Z
export LIVE_BUILD=/home/roland/git/live-build export SOURCE_DATE_EPOCH=1612927175 cd /dev/shm mount /dev/shm -odev,exec,remount mkdir live cd live
Mini, with live installer
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210210T031935Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/202102101031935Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution buster --debian-installer live --cache-packages false
Building images for Debian Bullseye
Image |
Installer |
Reproducible |
Notes |
mini |
none |
yes |
|
mini |
live/text |
yes |
1 |
mini |
live/gui |
yes |
1 |
mini |
daily |
yes |
2 |
standard |
none |
yes |
3, 4 |
standard |
live |
yes |
1, 3, 4 |
GNOME |
none |
yes |
3, 4, 5 |
GNOME |
live |
yes |
1, 3, 4, 5 |
KDE |
live |
yes |
1, 3, 4, 5 |
Cinnamon |
live |
yes |
3, 4, 5, 6 |
Note 1: The installer fails, because the installer uses an older kernel. Use the daily installer instead, until the installer is updated -> this is resolved
Note 2: Reproducibility tested by setting SOURCE_DATE_EPOCH during the first lb build and running the second build a few minutes after finishing the first build
- Note 3: fontconfig uses random GUIDs, which can be circumvented. Hooks 1000 and 1001 implement a work around
Note 4: mdadm adds a timestamp to the configuration file, which can be circumvented. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=982607. Until mdadm is updated, hook 1002 implements a work around. This will be fixed in bookworm, 4.2~rc2-4
Note 5: During update-initramfs, the hook of plymouth adds fonts to the initrds. Hooks 1000 and 1003 implement a work around
Note 6: libxml-sax-perl has a random order in its generated files, which can be circumvented. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993444. Hooks 1004 and 9000 implement a work around
Non-reproducible issue in Debian sid
The package dictionaries-common produced unstable content: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1000674 (fixed in 1.28.14) and https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1000685 (fixed in 1.28.13)
Seen in the Cinnamon, KDE and GNOME images: Differences at the first bytes of /var/cache/app-info/cache/C-local-metainfo.xb and /var/cache/app-info/cache/C-os-catalog.xb.
Caused by appstreamcli refresh-cache in package appstream (the apt-conf/50appstream file source)
A possible source is #include <xmlb.h> in as-cache.c -> Feature request for diffoscope support
Reported upstream in libxmlb
- Seen in the Cinnamon image:
Ordering differences in /var/lib/texmf/ls-R, /var/lib/texmf/web2c/luahbtex/luahbtex.fmt, /var/lib/texmf/web2c/luatex/dviluatex.fmt, /var/lib/texmf/web2c/luatex/luatex.fmt
Embedded timestamps in /var/lib/texmf/web2c/metafont/mf.log, /var/lib/texmf/web2c/pdftex/etex.log, /var/lib/texmf/web2c/pdftex/pdfetex.log, /var/lib/texmf/web2c/pdftex/pdftex.log, /var/lib/texmf/web2c/tex/tex.log, /var/lib/texmf/web2c/updmap.log
Small changes in /var/lib/texmf/web2c/metafont/mf.base, /var/lib/texmf/web2c/pdftex/etex.fmt, /var/lib/texmf/web2c/pdftex/pdfetex.fmt, /var/lib/texmf/web2c/pdftex/pdftex.fmt, /var/lib/texmf/web2c/tex/tex.fmt
Updated preparation
With the new snapshot service, the images can be generated without time-outs
export LIVE_BUILD=/home/roland/git.nobackup/live-build cd /dev/shm mount /dev/shm -odev,exec,remount mkdir live cd live # Use the timestamp of the latest mirror snapshot wget http://snapshot.notset.fr/mr/timestamp/debian/latest # # Extract the timestamp from the JSON file # # Input: # { # "_api": "0.3", # "_comment": "notset", # "result": "20210828T083909Z" # } # Output: # 20210828T083909Z # export SNAPSHOT_TIMESTAMP=$(cat latest | awk '/"result":/ { split($0, a, "\""); print a[4] }') # Convert SNAPSHOT_TIMESTAMP to Unix time (insert suitable formatting first) export SOURCE_DATE_EPOCH=$(date -d $(echo $SNAPSHOT_TIMESTAMP | awk '{ printf "%s-%s-%sT%s:%s:%sZ", substr($0,1,4), substr($0,5,2), substr($0,7,2), substr($0,10,2), substr($0,12,2), substr($0,14,2) }') +%s) export http_proxy=http://localhost:3142 export MIRROR=http://snapshot.notset.fr/archive/debian/${SNAPSHOT_TIMESTAMP} # The basic configuration line: lb config --apt-http-proxy ${http_proxy} --parent-mirror-bootstrap ${MIRROR} --parent-mirror-binary ${MIRROR} --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer live --cache-packages false
Command lines
Mini, without installer
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer none --cache-packages false
Mini, with live/text-only installer
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer live --cache-packages false --debian-installer-gui false
Mini, with latest installer
unset SOURCE_DATE_EPOCH lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/deb.debian.org/debian --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --distribution bullseye --debian-installer live --cache-packages false --debian-installer-gui false --parent-debian-installer-distribution daily # When doing comparisons, use the following line for the first run: # export SOURCE_DATE_EPOCH=`date +%s`; lb build
Mini, with live installer
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer live --cache-packages false
Standard, without installer
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer none --cache-packages false echo "live-task-standard" > config/package-lists/desktop.list.chroot
Standard, with live installer
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer live --cache-packages false echo "live-task-standard" > config/package-lists/desktop.list.chroot
GNOME desktop, without installer
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer none --cache-packages false echo "live-task-gnome" > config/package-lists/desktop.list.chroot
GNOME desktop, with live installer
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer live --cache-packages false echo "live-task-gnome" > config/package-lists/desktop.list.chroot
KDE desktop, with live installer
lb config --apt-http-proxy http://localhost:3142 --parent-mirror-bootstrap http://localhost:3142/snapshot.debian.org/archive/debian/20210101T083123Z --parent-mirror-binary http://snapshot.debian.org/archive/debian/20210101T083123Z --security false --updates false --apt-options "--yes -o Acquire::Check-Valid-Until=false" --distribution bullseye --debian-installer live --cache-packages false echo "live-task-kde" > config/package-lists/desktop.list.chroot
The hook scripts that should be applied after any of the command lines above, to fix non-reproducible packages
cat > config/hooks/normal/1000-reproducible-function-uuid_generate_random.hook.chroot << EOF #!/bin/sh set -e # util-linux creates random UUIDs when uuid_generate_random is called # Use LD_PRELOAD to replace uuid_generate_random with a less random version # Don't run if gcc is not installed if [ ! -e /usr/bin/cc ]; then exit 0 fi cat > unrandomize_uuid_generate_random.c << END_OF_SOURCE #include <stdlib.h> #include <stdio.h> #define SEQUENCE_FILENAME "/var/cache/unrandomize_uuid_generate_random.sequence_number" /* https://tools.ietf.org/html/rfc4122 */ typedef unsigned char uuid_t[16]; /* Our pseudo-random version */ void uuid_generate_random(uuid_t out) { /* Nil UUID */ for (int i=0;i<16;i++) { out[i] = 0x00; } out[6]=0x40; /* UUID version 4 means randomly generated */ out[8]=0x80; /* bit7=1,bit6=0 */ /* The file doesn't need to exist yet */ FILE *f = fopen(SEQUENCE_FILENAME, "rb"); if (f) { fread(out+12, 4, 1, f); fclose(f); } /* Use the next number. Endianness is not important */ (*(unsigned long*)(out+12))++; unsigned long long epoch; /* Use SOURCE_DATE_EPOCH when provided */ char *date = getenv("SOURCE_DATE_EPOCH"); if (date) { epoch = strtoll(date, NULL, 10); } else { epoch = 0ll; } out[0] = (epoch & 0xFF000000) >> 24; out[1] = (epoch & 0x00FF0000) >> 16; out[2] = (epoch & 0x0000FF00) >> 8; out[3] = (epoch & 0x000000FF); /* Write the sequence number */ f = fopen(SEQUENCE_FILENAME, "wb"); if (f) { fwrite(out+12, 4, 1, f); fclose(f); } } END_OF_SOURCE /usr/bin/cc -shared -fPIC unrandomize_uuid_generate_random.c -Wall --pedantic -o /usr/lib/unrandomize_uuid_generate_random.so rm -f unrandomize_uuid_generate_random.c EOF cat > config/hooks/normal/1001-reproducible-fontconfig.hook.chroot << EOF #!/bin/sh set -e # fontconfig creates non-reproducible files with UUIDs # See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=864082 # # Because the UUIDs should not be deleted, the proposed work-around is: # * Use LD_PRELOAD to replace uuid_generate_random with a less random version # Don't run if fontconfig is not installed if [ ! -e /usr/bin/fc-cache ]; then exit 0 fi # Don't run if the LD_PRELOAD module is not compiled if [ ! -e /usr/lib/unrandomize_uuid_generate_random.so ]; then exit 0 fi LD_PRELOAD=/usr/lib/unrandomize_uuid_generate_random.so /usr/bin/fc-cache --force --really-force --system-only --verbose EOF cat > config/hooks/normal/1002-reproducible-mdadm.hook.chroot << EOF #!/bin/sh set -e # mkconf of mdadm creates a file with a timestamp # A bug report with patch is available at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=982607 # This script duplicates that patch # Don't run if mdadm is not installed if [ ! -e /usr/share/mdadm/mkconf ]; then exit 0 fi # If mkconf already contains references to SOURCE_DATE_EPOCH, there is no need to patch the file if grep -q SOURCE_DATE_EPOCH /usr/share/mdadm/mkconf; then exit 0 fi sed -i -e '/# This configuration was auto-generated on/cif [ -z \$SOURCE_DATE_EPOCH ]; then\n echo "# This configuration was auto-generated on \$(date -R) by mkconf"\nelse\n echo "# This configuration was auto-generated on \$(date -R --utc -d@\$SOURCE_DATE_EPOCH) by mkconf"\nfi' /usr/share/mdadm/mkconf EOF cat > config/hooks/normal/1003-reproducible-plymouth.hook.chroot << EOF #!/bin/sh set -e # The hook of plymouth in update-initramfs calls fc-cache # Don't run if plymouth is not installed if [ ! -e /usr/share/initramfs-tools/hooks/plymouth ]; then exit 0 fi # Don't patch if the LD_PRELOAD module is not compiled if [ ! -e /usr/lib/unrandomize_uuid_generate_random.so ]; then exit 0 fi # If the hook already contains references to LD_PRELOAD, there is no need to patch the file if grep -q LD_PRELOAD /usr/share/initramfs-tools/hooks/plymouth; then exit 0 fi sed -i -e 's|fc-cache -s|LD_PRELOAD=/usr/lib/unrandomize_uuid_generate_random.so fc-cache|' /usr/share/initramfs-tools/hooks/plymouth EOF cat > config/hooks/normal/1004-reproducible-libxml-sax-perl.hook.chroot << EOF #!/bin/sh set -e # update-perl-sax-parsers of libxml-sax-perl creates a file with a random order of its lines # A bug report with patch is available at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993444 # This script duplicates that patch # Don't run if libxml-sax-perl is not installed if [ ! -e /usr/bin/update-perl-sax-parsers ]; then exit 0 fi # If Debian.pm already contains a sort line, there is no need to patch the file if grep -q sort /usr/share/perl5/XML/SAX/Debian.pm; then exit 0 fi sed -i -e '/foreach my \$key/s/keys/sort keys/' /usr/share/perl5/XML/SAX/Debian.pm # Regenerate the file that has more than one key-value pair update-perl-sax-parsers --remove XML::SAX::Expat update-perl-sax-parsers --add XML::SAX::Expat --priority 50 update-perl-sax-parsers --update EOF cat > config/hooks/normal/9000-cleanup-ucf-backup-files.hook.chroot << EOF #!/bin/sh set -e # Delete all older backups of ucf files # The current files are /var/lib/ucf/hashfile and /var/lib/ucf/registry rm -f /var/lib/ucf/hashfile.* rm -f /var/lib/ucf/registry.* EOF
Checking reproducibility
Ideally reprotest will be used to check reproducibility. For now, 2 images are built and then compared with diffoscope.
Using diffoscope
# 1) Prepare the build environment # 2) Run the command line to create the configuration (see above) lb build mv live-image-amd64.hybrid.iso run01.iso lb clean --purge lb config lb build mv live-image-amd64.hybrid.iso run02.iso diffoscope run01.iso run02.iso --html-dir html_run01_02
Using reprotest
reprotest requires full access on /tmp, so it must be remounted (when applicable)
# 1) Prepare the build environment # 2) Run the command line to create the configuration (see above) mount /tmp -odev,exec,suid,remount reprotest --variations=-all,+environment,+build_path,-kernel,+aslr,+num_cpus,+time,-user_group,-fileordering,-domain_host,+home,+locales,+exec_path,+timezone,-umask "lb clean --purge&&lb config&&lb build" "*.iso" # Disabled tests: # kernel The variant with kernel 2.6 is too old for debootstrap # user_group Live build only works for root, the variation needs a list # fileordering Disorderfs needs to be mounted with dev,exec,suid # domain_host Needs additional rights during debootstrap # umask Some files get a different umask, further investigation is required
See also
Older work: https://rclobus.nl/blog/?p=190
Page maintainer: Roland Clobus