>>> THIS IS WORK IN PROGRESS BASED ON MY MESSAGE https://lists.debian.org/debian-live/2018/01/msg00010.html

Downloading Debian Jigdo ISOs by a Debian Live system

This page describes how to use a Debian Live system for downloading large Debian installation ISOs or the full set of DVD sized installation ISOs.

Contents

  1. Downloading Debian Jigdo ISOs by a Debian Live system
    1. What is Jigdo ?
    2. Obtain a Debian Live medium
      1. Download Debian Live
      2. Verify the Debian Live download
      3. Put Debian Live onto medium
    3. Boot Debian Live
    4. Get access to a writable disk filesystem
    5. Install package jigdo-file
    6. If needed, work around a shortcoming of older jigdo-lite
    7. Download one or more Jigdo ISOs
    8. Verify the downloaded ISOs
    9. Put the resulting ISOs onto medium
    10. Brief description of the lines in .jigdo files
  2. See also:
  3. External Links

What is Jigdo ?

Jigdo is used by Debian to provide the framework of an ISO 9660 image and a set of Debian package URLs which can be used to fill the gaps in the framework. Result of this filling process is the complete ISO image.

Each ISO image is defined by two Jigdo specific files: *.template contains the compressed framework and checksums of the gap content. *.jigdo contains a compressed translation table from checksum to file path, overall image checksums, and a list of fallback mirror server URLs. The package URLs are composed from a mirror URL and the file path from the list in the *.jigdo file.

The downloading user provides the URL of a mirror server which is tried first for package downloading. Packages which remain missing afterwards get downloaded from the fallback mirror servers.

Link lists to the jigdo files of the current Debian "stable" release ISO images are at the end of the web pages which are linked to by

E.g. at

Obtain a Debian Live medium

If you already have a Debian-based system installed and running, then you do not need the Live system, but may rather hop to paragraph Install package jigdo-file.

Debian Live is a way to get such a Debian-based system without the need to install it on your non-Debian PC. It is available for 32-bit Intel-or-AMD CPUs and for 64-bit Intel-or-AMD CPUs.

Download Debian Live

A brief overview of Debian Live is given at https://www.debian.org/CD/live/ . If you have a 64-bit Intel or AMD CPU then go to the "amd64" list:

If you have a 32-bit CPU go to "i386" (even if it is made by AMD):

There you see various *.iso files with names of graphical GNU/Linux desktops like "gnome", "kde", "xfce". Let's assume you decide for "xfce" and download by help of your normal operating system: 

https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/debian-live-9.3.0-amd64-xfce.iso

Verify the Debian Live download

You should also download the checksum files from the same directory as the Debian Live ISO. In our example: 

https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/SHA512SUMS
https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/SHA512SUMS.sign

In file SHA512SUMS is listed 

80080716db2fe5e808c7ae771fa560e1306465abfb8d92f1290ec80377150f0aed38d05a9abe425b4548a5d30b03a5f5135b0d158ae819855c6d6ef89b689e44  debian-live-9.3.0-amd64-xfce.iso

This should be compared with the result of a SHA512 checksumming program, like this shell command 

sha512sum debian-live-9.3.0-amd64-xfce.iso

which should give the same long string "8008...9e44".

A perfect match of that string outrules any incidental damage. The authenticity of file SHA512SUMS can be verified by OpenPGP implementations like in this shell command using GnuPG: 

gpg --keyserver keyring.debian.org --recv-keys 64E6EA7D 6294BE9B 09EA8AC3
gpg --with-fingerprint --verify SHA512SUMS.sign SHA512SUMS

It must say among other info: 

gpg: Good signature from "Debian CD signing key <debian-cd@lists.debian.org>"
...
Primary key fingerprint: DF9B 9C49 EAA9 2984 3258  9D76 DA87 E80D 6294 BE9B

The reported "key fingerprint" must match one of those listed at https://www.debian.org/CD/verify .

GPG binaries for other operating systems can be found on https://www.gnupg.org/download/index.html

Put Debian Live onto medium

Hop to the next paragraph if you plan to boot a virtual machine with the Debian Live ISO as virtual CD-ROM and write access to the filesystem to which you want to download the Debian ISOs by Jigdo.

If you prefer to boot a real machine:
The Debian Live image file may either be burned onto a DVD medium, or copied plainly onto a USB stick. For instructions see these four paragraphs of the FAQ:

Be extremely careful not to overwrite your hard disk instead of the USB stick !

Consider to first read from the intended device address and watch the USB stick blinking, before daring to write to that address.

Boot Debian Live

You may now boot a virtual machine with debian-live-9.3.0-amd64-xfce.iso as virtual CD-ROM.

If you prefer to boot a real machine:
Put the DVD into the drive of your computer or plug the USB stick into a USB socket of your computer. Shut down its operating systen, if running, and power on the computer.

After the machine's firmware issued its usual messages you should soon get to see a menu screen with the first item highlighted. Press the Return key to start it.

For a while you will see a black screen or obscure messages. After 1 or 2 minutes you should see a sparsely populated graphical desktop screen with icons representing your computer's reconizable storage media and disk partitions.

Open a shell terminal from the desktop icon at the bottom of the screen.

Open a web browser from desktop icon and go to the URL of this wiki page. If you cannot connect to the internet, then you will first have to solve this problem.

>>> How to get firmware and network helper software ?

Get access to a writable disk filesystem

Open the File Manager from desktop icon or by double-clicking any of the storage media icons. Its window will show a list of storage media in a column at the left side. Double-click one to get acces to its directory tree. By try-and-error it should be possible to navigate to the directory on your hard disk where you want to build the ISO image.

Then read the filesystem path of the reached directory from the text input line at the top of the File Manager window. This path can be long and ugly like 

/media/user/e3378181-54dc-4fe3-97e2-d7f2b6c1d296/MyLinuxDownloads/Isos

Mark this text line by pressing the left mouse button and moving the cursor over the whole text.

>>> ??? any peculiarities of MS-Windows or MacOS disks

Now click at the shell terminal window, write "cd " and click the rightmost mouse button to paste in the path. You should see 

cd /media/user/e3378181-54dc-4fe3-97e2-d7f2b6c1d296/MyLinuxDownloads/Isos

Execute this shell command by pressing the Return (or Enter) key.

Install package jigdo-file

At least up to the end of 2017, Debian Live systems do not bring the Jigdo download script out of the box. So shell terminal command jigdo-lite will fail with the message 

bash: jigdo-lite: command not found

If it instead says "Jigsaw Download", then go on to the next paragraph.

To get script "jigdo-lite" and its helper programs execute these two commands: 

sudo apt-get update
sudo apt-get install jigdo-file

The final stage of this first package installation may last 1 or 2 minutes.

If needed, work around a shortcoming of older jigdo-lite

Older versions of jigdo-lite cannot cope with the "https:" prefix of the current *.jigdo URLs. So if the download procedure shown in the next paragraph fails immediately after entering the *jigdo URL with an error message like 

File `https://cdimage.debian.org/debian-cd/current/amd64/jigdo-bd/debian-9.4.0-amd64-BD-1.jigdo' does not exist!

then you need to manually download each *.jigdo and each *.template file of the ISO images which you want to get. E.g. 

wget https://cdimage.debian.org/debian-cd/current/amd64/jigdo-bd/debian-9.4.0-amd64-BD-1.jigdo
wget https://cdimage.debian.org/debian-cd/current/amd64/jigdo-bd/debian-9.4.0-amd64-BD-1.template

When jigdo-lite sees the desired files already in the download directory it will go on with asking for other info.

The *.jigdo and *.template files which were built after february 12 2018 are listed in the SHA512SUMS files which are offered in the same directories. If new enough, one should already now download and verify SHA512SUMS, in order to then verify *.jigdo and *.template: 

wget https://cdimage.debian.org/debian-cd/current/amd64/jigdo-bd/SHA512SUMS
wget https://cdimage.debian.org/debian-cd/current/amd64/jigdo-bd/SHA512SUMS.sign
gpg --keyserver keyring.debian.org --recv-keys 64E6EA7D 6294BE9B 09EA8AC3
gpg --with-fingerprint --verify SHA512SUMS.sign SHA512SUMS
sha512sum debian-9.4.0-amd64-BD-1.jigdo
sha512sum debian-9.4.0-amd64-BD-1.template

See above paragraph Verify the Debian Live download for details.

Download one or more Jigdo ISOs

Execute in the shell terminal: 

jigdo-lite

The script will ask three questions: *.jigdo URL, local package repository to scan, and user-chosen mirror URL.

asks for the URL of a *.jigdo file. Like 

https://cdimage.debian.org/debian-cd/current/amd64/jigdo-bd/debian-9.4.0-amd64-BD-1.jigdo

This URL will be downloaded if not the file already exists in the current directory.

should simply get an empty line as input. It asks for a directory where a Debian-style repository of packages can be found under the name "pool".

asks for the user-chosen URL of a Debian server repository. It should lead to a directory where a repository of packages can be found under the name "pool". The repositories are mirrored worldwide. See for country specific proposals:

In germany, for example, you could pick 

http://ftp.de.debian.org/debian/

If the desired ISO image is old, e.g. from https://cdimage.debian.org/cdimage/archive, then the repository for old Debian releases is a good choice: 

http://archive.debian.org/debian/

Now begins automatic package downloading for a time span which depends on the size of the resulting ISO, the bandwidth of the mirror server, and its latency.

If a screensaver with password prompt appears after a while, you can get back to the desktop by

Verify the downloaded ISOs

When the ISO is complete and jigdo-lite confirms matching checksum, you should nevertheless verify its checksum in the file SHA512SUMS and the authenticity of this file by its signature SHA512SUMS.sign: 

wget https://cdimage.debian.org/debian-cd/current/amd64/jigdo-bd/SHA512SUMS
wget https://cdimage.debian.org/debian-cd/current/amd64/jigdo-bd/SHA512SUMS.sign
gpg --keyserver keyring.debian.org --recv-keys 64E6EA7D 6294BE9B 09EA8AC3
gpg --with-fingerprint --verify SHA512SUMS.sign SHA512SUMS
sha512sum debian-9.4.0-amd64-BD-1.iso

See above paragraph Verify the Debian Live download for details. The command gpg --recv-keys is needed only if you did not perform it already when verifying .jigdo and .template or if you have shutdown the system since then. It does not harm to run it more often than needed, though.

Put the resulting ISOs onto medium

If you are more comfortable with non-GNU/Linux tools, you may instead of the following proposals use your existing operating system and follow the advise at

Here is how to do it directly by the Debian Live system:

For burning the ISO onto a CD, DVD, or BD medium you may use the pre-installed program xfburn with its task "Burn Image".

Or you may install xorriso by 

sudo apt-get install xorriso

and burn the ISO by e.g. this shell command: 

xorriso -as cdrecord -v dev=/dev/sr0 -eject stream_recording=on  debian-9.4.0-amd64-BD-1.iso

See output of xorriso -devices for a list of detected /dev/sr devices.

For writing to USB stick do e.g. 

sudo cp debian-9.4.0-amd64-BD-1.iso /dev/sdd

where /dev/sdd is the overall device address of the USB stick. Do not pull the stick before the "cp" run has ended.

Be extremely careful not to overwrite your hard disk instead of the USB stick !

Consider to first read 200 MB from the intended device address by e.g. 

sudo dd if=/dev/sdd of=/dev/null bs=1M count=200

and watch the USB stick blinking, before daring to write to that address.

About verifying the correctness of the copy-to-medium procedure, see

Afterwards you may delete the *.jigdo, *.template, and *.iso file from the download directory to make room for the next download.

Brief description of the lines in .jigdo files

While the .template file is strictly binary, the .jigdo file is human readable after decompression by gunzip. 

gunzip <debian-9.4.0-amd64-BD-1.jigdo | less

The lines in sections "[Jigdo]", "[Image]", and "[Servers]" are quite self-explanatory. The lines in section "[Parts]" consist of a base64 encoded checksum, the text snippet "=Debian:", and a relative file path of a "pool/" package or some other file which is supposed to be on the Debian mirror servers.

Decoding the base64 string to the usual hex string format needs some shell gymnastics.

With a MD5 checksum (22 base64 characters) add "==" as padding: 

checksum='CI4y7_tlTWg-t2vmR8S8bQ'==

echo -n "$checksum" | sed -e 's/-/+/g' -e 's/_/\//g' | \
  base64 -d - | \
  od -t x1 | head -n 1 | sed -e 's/^.......//' -e 's/ //g'

yields on standard output: 

088e32effb654d683eb76be647c4bc6d

With the longer SHA256 checksums only one padding "=" is needed, and two hex lines have to be united: 

checksum='3VpfoS-0at8kB1NJbzZgIg-ePBTDxsrxR9ISBkRRkTs'=

echo -n "$checksum" | sed -e 's/-/+/g' -e 's/_/\//g' | \
  base64 -d - | \
  od -t x1 | head -n 2 | sed -e 's/^.......//' -e 's/ //g' | \
  tr -d '\n' ; echo

yields 

dd5a5fa12fb46adf240753496f3660220f9e3c14c3c6caf147d212064451913b

See also: