Remastering a Debian installation ISO or a CD/DVD image.

Introduction

The Debian installer is based on udebs. Many of those udebs are in the installation image but some of them are in the image's initrd. If you want to modify the udeb's in the initrd then you should follow the instructions on how to build a full image with a custom kernel. However, when you simply want to substitute, amend or add to Debian packages or non-initrd udeb's, you can do this without building a full image.

An already built installation image is required; a netinst or CD/DVD/BD image would be suitable. The remastering of such an image will be illustrated here using the first two DVD images for the Debian 9 (stretch) i386 architecture. The first image is the installation image and the second contains packages we want to add to the first image.

Almost all the remastering process can be carried out as an ordinary user. There is no need for root privileges until it comes to writing the new image to a USB device with dd (or using the genisoimage command).

Creating a Copy of the Installation Image

This is the image (DVD-1) that gets modified; it forms the basis for the remastered image. Because an ISO9660 image cannot be modified directly (it is a read-only image) it must first be extracted. There is a number of techniques to do this but bsdtar will be used here.

First make a directory in which to put the extracted files and then extract them:

$ mkdir cd
$ bsdtar -C cd/ -xf debian-9.3.0-i386-DVD-1.iso

Make the files in cd writable:

chmod -R +w cd

Transferring Packages to the Installation Image

Looking at the contents of DVD-1 when extracted to cd the following two directories will be seen:

~/cd/pool/main 
~/cd/dists/stretch/main/binary-i386

pool contains the Debian packages and it is where any additional packages (from DVD-2, say) would be put. dists is for storing Packages lists and Release files - the index files.

Now put any packages that you want in your image into pool/main or pool/contrib. As an example, suppose you want to copy all of pool/main on DVD-2 to pool/main on DVD-1, the installation image. This could be done either by loopmounting DVD-2 or directly extracting its files to a directory.

Choosing the second method:

mkdir dvd-2
bsdtar -C dvd-2/ -xf debian-9.3.0-i386-DVD-2.iso
chmod -R +w dvd-2
cp -a dvd-2/pool/main/* cd/pool/main/

Creating New Packages and Release Files and Fixing md5sum.txt

Putting packages in pool is not sufficient to produce a valid new image; all the index files (Packages, Packages.gz and Release) and md5sums.txt have to be updated to reflect this change in the image.

It is convenient to update package information using apt-ftparchive and a configuration file.

Create the file config-deb and run:

apt-ftparchive generate config-deb

to generate Packages and Packages.gz files.

# A config-deb file.

# Points to where the unpacked DVD-1 is.
Dir {
    ArchiveDir "cd";
};

# Sets the top of the .deb directory tree.
TreeDefault {
   Directory "pool/";
};

# The location for a Packages file.                
BinDirectory "pool/main" {
   Packages "dists/stretch/main/binary-i386/Packages";
};

# We are only interested in .deb files (.udeb for ubeb files).                                
Default {
   Packages {
       Extensions ".deb";
    };
};

One way of updating the Release file in cd/dists/stretch/ is to delete everything in the file from the MD5Sum: line to the end of the file

sed -i '/MD5Sum:/,$d' cd/dists/stretch/Release

followed by (editing the file, if desired)

apt-ftparchive release cd/dists/stretch >> cd/dists/stretch/Release

The md5sum of any image file has to accord with what is in cd/md5sum.txt. Regenerate it with

cd cd; md5sum `find ! -name "md5sum.txt" ! -path "./isolinux/*" -follow -type f` > md5sum.txt; cd ..

It is likely there will be warning message, which appears not to affect the production of an md5sum.txt:

find: File system loop detected; ‘./debian’ is part of the same file system loop as ‘.’.

Including Override Information

The new image to be created will differ from the image it is based on, if only because of the extra packages it contains. As it stands at this stage, another difference between the new image and DVD-1 (say) lies in the Packages file content. In the course of producing this file no account was taken of the centrally managed override file. Although not critical, the absence of override information could be seen as a defect; fortunately, it is an easy one to rectify.

The purpose of an override file is explained here. To illustrate the intention: the package description for less has a Priority field value of important. This is considered by the Debian archive maintainers to be too high, so it is adjusted via an override file to standard, which is what is used in constructing the installer to determine what is installed. As it happens, a user argued against this downgrading, with the result that less has returned to Priority: important for Debian 10 (buster).

In the config-deb file change

BinDirectory "pool/main" {
   Packages "dists/stretch/main/binary-i386/Packages";
};

to

BinDirectory "pool/main" {
   Packages "dists/stretch/main/binary-i386/Packages";
   BinOverride "override";
   ExtraOverride "override.extra";
 };

and add this snippet:

Dir {
   ArchiveDir "cd";
   OverrideDir "indices";
   CacheDir "indices";
 };

Obtain override files from a mirror, put uncompressed versions (use gunzip) in the created directory cd/indices/override and carry out the previous procedures.

Creating a New Image

The original installation image (DVD-1) was designed to be bootable after being burned to a CD, DVD or BD medium or written to a USB device such as a USB stick. The next three commands will achieve this versatility for Legacy BIOS and for EFI in Legacy/CSM mode.

xorriso is the package required to create a hybrid image, one which can be booted both from CD/DVD/BD media as well as from a usb key drive. isolinux will also need to be on the system.

chmod -R -w cd

dd if=debian-9.3.0-i386-DVD-1.iso bs=1 count=432 of=isohdpfx.bin

xorriso -as mkisofs -o test.iso \
-isohybrid-mbr isohdpfx.bin \
-c isolinux/boot.cat -b isolinux/isolinux.bin \
-no-emul-boot -boot-load-size 4 -boot-info-table ./cd

The dd command is advised so that isohdpfx.bin and isolinux.bin stem from the same version of SYSLINUX. It is safer but, without it, you could still find the xorriso command to be successful with /usr/lib/ISOLINUX/isohdpfx.bin in its place.

Insert the key into a USB port and use lsblk to find its block device, /dev/sdX. Then, as root (using su or sudo), write the image to a usb key drive:

dd if=test.iso bs=4M of=/dev/sdX; sync

or, as an ordinary user, burn the image to a CD, DVD, or BD medium in drive /dev/sr0:

xorriso -as cdrecord -v dev=/dev/sr0 -eject test.iso

For a medium which is bootable from a cdrom only, genisoimage can be used in place of xorriso. To make the cdrom bootable, you need to run genisoimage with appropriate parameters. Here is how to do it for x86/amd64, using isolinux for Legacy BIOS. For instructions on how to re-use the full boot equipment of installation ISOs for i386, amd64, arm64 and other architectures, see RepackBootableISO.

cd cd
genisoimage -o test.iso -r -J -no-emul-boot -boot-load-size 4 \
-boot-info-table -b isolinux/isolinux.bin -c isolinux/boot.cat ./cd 

Installing From the New Image

Installing Debian from the new i386 ISO is just a matter of providing a computer's input device with it, booting from it and choosing the installation method. Perhaps, however, there is one thing to be aware of, depending on how it is intended to use the archive after installation.

With a connection to the internet any package updates that are available (security updates, for example) would normally be pulled in during the installation of the image. Archive information would now become outdated and some packages from an archive would be uninstallable. If this is unwanted, use one of these two solutions:

On the other hand, if the intention is to have access to stable updates and security fixes on the installed system, add

after the package archive entry.

Packages not available in the package archive will come from deb.debian.org.

Accessing the New Image's Archive

At first boot of the new operating system /etc/apt/sources.list will have a line something like this:

deb cdrom:[Debian GNU/Linux 9.3.0 _Stretch_ - Official i386 NETINST 20171209-13:03]/ stretch contrib main non-free

Change that to

deb [trusted=yes] cdrom:[Debian GNU/Linux 9.3.0 _Stretch_ - Official i386 NETINST 20171209-13:03]/ stretch contrib main non-free

apt update

should now produce a positive outcome.

Assuming the installation image and the archive is held on a a USB device, its block device (found from lsblk), can be mounted (as root) with

mount /dev/sdX /media/cdrom

and packages installed with

apt install <package>

Alternatively, a USB devices's LABELs can be found from

  • lsblk -f

and used for automatic mounting as described below.

Automatically mounting a partition on the device when it is accessed and automatically unmounting it when access to it is not wanted can be done with an entry in /etc/fstab. All the parameters necessary for this to happen are available from systemd 220 onwards and explained in the systemd.mount manual. Comment out any /dev/srX entry in /etc/fstab.

  • LABEL=ISOIMAGE /media/cdrom0 iso9660 defaults, noauto,x-systemd.automount,x-systemd.idle-timeout=5,x-systemd.device-timeout=1 0 0

  • x-systemd.automount mounts /media/cdrom when any command wants to use it.

  • systemd.idle-timeout leads to unmounting the partition a specified time period after the calling program ceases to access it.

  • noauto causes /media/cdrom not to be mounted while the machine is booting.

  • x-systemd.device-timeout configures how long systemd should wait when no device is plugged in or an incorrect device is found.

On a running system, do

  • systemctl daemon reload

followed by

  • systemctl restart media-cdrom0.automount

after changing /etc/fstab.

Miscellaneous

The account in the previous sections should be sufficient to obtain a bootable, remastered ISO image. The following material was part of the original page and is left here in the event that some of it could prove useful.

Files and hooks

A list of hooks available for debian-installer: https://d-i.debian.org/doc/internals/apb.html

 (filepaths as seen from the installer)

List of packages (udebs) to install or not install in into the d-i ramdisk.

  ''cdrom''.disk/udeb_include ["anna"]
  ''cdrom''.disk/udeb_exclude

The content of these files are taken as lists of packages to install or not into the target area.

  ''cdrom''.disk/base_include [base-installer]
  ''cdrom''.disk/base_exclude

If the file ''cdrom''.disk/base_installable exists, the base-installer udeb selects the cdrom as source for the installation.

Alternate Method

If you use apt-move to store the packages apt downloads when updating your system, you can use the mirror it creates to fill the cdrom with all the packages on your current system. Remember to call "apt-move sync" before mastering the image, to make sure, all dependencies are met.

Helper scripts: diunpk, dipk

Here are example scripts for unpacking and packing d-i CD image file. You need to have aufs modules available.

* diunpk: execute this with d-i.iso file as its argument. This generates tree under dated directory and cd into it.

#!/bin/sh -e

# Local customization to match your id (check with 'id' command).
uid=1000
gid=1000

# Default values
# $1 d-i iso image file
# $2 d-i ro mount point
# $3 d-i rw tree

di_iso=${1:-d-i.iso}
di_ro=${2:-d-i.ro}
di_rw=${3:-d-i.rw}

pwd=$(pwd)
timestamp=$(date -u +%Y%m%d%H%M%S)
mkdir $timestamp

mkdir $timestamp/$di_ro
mkdir $timestamp/$di_rw

sudo mount ${di_iso} $timestamp/${di_ro} -t iso9660 -o loop,uid=${uid},gid=${gid}
sudo mount -t aufs -o br:$timestamp/${di_rw}:$timestamp/${di_ro} none $timestamp/${di_rw}
sudo chmod u+w $timestamp/${di_rw}
sudo chmod u+w $timestamp/${di_rw}/md5sum.txt
sudo find $timestamp/${di_rw}/dists -exec chmod u+w {} \;
sudo find $timestamp/${di_rw}/pool  -type d -exec chmod u+w {} \;
cd $timestamp

* dipk: execute this in dated directory.

#!/bin/sh -e
set -x
# Local customization
uid=1000
gid=1000
arch=amd64
release=squeeze

# Default values
# $1 d-i iso image file
# $2 d-i ro mount point
# $3 d-i rw tree

di_iso=${1:-d-i.iso}
di_ro=${2:-d-i.ro}
di_rw=${3:-d-i.rw}

cat > config << EOF
Dir {
    ArchiveDir ".";
    OverrideDir ".";
    CacheDir ".";
 };
            
 TreeDefault {
    Directory "pool/";
 };
                    
 BinDirectory "pool/main" {
    Packages "dists/${release}/main/debian-installer/binary-${arch}/Packages";
 };
                                   
 Default {
    Packages {
        Extensions ".udeb";
    };
 };
EOF

cd $di_rw
sudo apt-ftparchive generate ../config
sudo md5sum $(find ! -name "md5sum.txt" ! -path "./isolinux/*" -follow -type f) > md5sum.txt
cd -

#genisoimage ...
sudo genisoimage -r -o $di_iso -V di$(date -u +%m%d%H%M%S) \
   -b isolinux/isolinux.bin -c isolinux/boot.cat \
   -no-emul-boot -boot-load-size 4 -boot-info-table $di_rw

# check mounted by "mount"
#sudo umount ${di_rw}
#sudo umount ${di_ro}
#rm -rf $di_rw

Create a udeb packages file

Write a config-udeb file with something like this :

 Dir {
    ArchiveDir "cd";
    OverrideDir "indices";
    CacheDir "indices";
 };
            
 TreeDefault {
    Directory "pool/";
 };
                    
 BinDirectory "pool/main" {
    Packages "dists/squeeze/main/binary-i386/Packages";
    BinOverride "override";
    ExtraOverride "override.extra";
 };
                                   
 Default {
    Packages {
        Extensions ".udeb";
    };
 };

Because of the BinOverride line you also need the override file, you can get it from any debian mirror, e.g. get http://ftp.de.debian.org/debian/indices/override.squeeze.main.gz and extract it to the indices/override.

Now go into the directory with your config file and run

apt-ftparchive generate config-udeb

to generate the Packages and Releases files.

To update the cd/dists/squeeze/Release file, make a new text file called config-rel with something like this:

APT::FTPArchive::Release::Codename "squeeze";
APT::FTPArchive::Release::Origin "Debian";
APT::FTPArchive::Release::Components "main";
APT::FTPArchive::Release::Label "Debian";
APT::FTPArchive::Release::Architectures "i386";
APT::FTPArchive::Release::Suite "testing";

To generate the Release file, run

 # apt-ftparchive -c config-rel release cd/dists/squeeze > cd/dists/squeeze/Release

See Also