16758
Comment:
|
16803
|
Deletions are marked like this. | Additions are marked like this. |
Line 8: | Line 8: |
The path to creating the procedure began when information on how create a debian partial mirror without having a full mirror in the first place couldn't be found. deb-mirror and friends all required that one start with a full mirror which isn't an option for for everyone. The decision was made to try creating a custom partial mirror using a network install cd and what was downloaded using aptitude (apt-get with a nice frontend) on an older system that was being refurbished. It took more work than expected, but it now works. | This article was begun it was found that information on how create a debian partial mirror without having a full mirror in the first place did not appear to be available. deb-mirror and friends all required that one start with a full mirror which doesn't work if you don't have dsl, or have bandwidth caps. The decision was made to try creating a custom partial mirror using a network install cd and what was downloaded using aptitude (apt-get with a nice frontend) on an older system that was being refurbished. It took more work than expected, but it now functional. |
Line 340: | Line 340: |
=== 3. Get overrides.<dist>.* === | === 3. Get overrides.<dist>.* === |
Line 344: | Line 344: |
If you want to includes packages from non-US as well you ought to create another indices directory (e.g. indices-non-US), and get the overrides from a non-US mirror as well (from /debian-non-US/indices-non-US). | If you want to includes packages from non-US you ought to create another indices directory (e.g. {{{indices-non-US}}}), and get the overrides from a non-US mirror (from {{{/debian-non-US/indices-non-US}}}). |
Line 399: | Line 399: |
* {{{OverrideDir}}} is the directory which contains your overrides ''Note that this means that if you want non-US, you need another {{{.conf}}} with your non-US directories in it.'' | * {{{OverrideDir}}} is the directory which contains your overrides ''Note that this means that if you want non-US, you need another {{{.conf}}} with your non-US directories.'' |
Line 402: | Line 402: |
* The path in quotes after {{{BinDirectory}}} (and before the curly brace), is the location, relative to {{{ArchiveDir}}} of the binary packages (well actually the alphabetic directories, under which are the package directories). * Under {{{BinDirectory}}}, {{{Packages}}} is the full pathname of the Packages (without .gz even if you are creating compressed files) file you are creating (in this case the debian-installer Packages.gz). |
* The path in quotes after {{{BinDirectory}}} (and before the curly brace), is the location, relative to {{{ArchiveDir}}} of the binary packages (well actually the alphabetic directories, which have the package directories under them). * Under {{{BinDirectory}}}, {{{Packages}}} is the full pathname of the Packages (without .gz even if you are creating compressed files) file you are creating (in this case the debian-installer {{{Packages.gz}}}). |
Building a Custom Debian CD Set
2004-11-22 DanielDickinson
Introduction
This page will describe the steps used to create a custom debian install cd using a partial mirror mostly gathered from a prototype system's apt cache. The intructions here assume i386 but should work for any architecture simply by replacing i386 in the instructions with the architecture of your choice.
This article was begun it was found that information on how create a debian partial mirror without having a full mirror in the first place did not appear to be available. deb-mirror and friends all required that one start with a full mirror which doesn't work if you don't have dsl, or have bandwidth caps. The decision was made to try creating a custom partial mirror using a network install cd and what was downloaded using aptitude (apt-get with a nice frontend) on an older system that was being refurbished. It took more work than expected, but it now functional.
Quick Start
Download a DebianInstaller network install cd (netinst.iso).
Install Debian on a prototype system (this is optional)
?AptGet packages (or aptitude if you prefer, as I do) -> hint: you don't have to install, just download to the apt cache, as long apt-move works.
AptMove to a directory of your choice.
create a pool directory on the filesystem where you will build your new cd.
- copy the pool directory from network install cd to the directory you just created.
copy from the directory you apt-move'd to to the pool directory (you could just do this first, but if you realize you missed something the first time around, you'll need to do this).
get installer-i386 from a debian mirror which is from the same release of DebianInstaller as your network install cd (unless you also build the installer and associated packages). get overrides.<dist>.''' files from http://<mirror>/debian/indices
- place in your debian-dir (i.e. the same directory as the pool directory you created above).
gunzip the overrides in-place the indices directory (you should probably keep the original gzipped versions too)
- generate packages files.
generate DebianInstaller packages files:
DebianInstaller lives in <debian-dir>''dists''<dist>/main/debian-installer/binary-i386
so create the appropriate subdirectories (e.g. mkdir -p /debian/dists/sarge/main/debian-installer/binary-i386).
in a directory for your scripts and config files for this project place the following apt.conf:
APT { ["FTPArchive"] { Release { Origin "debian-cd"; Label "yousarge"; Suite "testing"; Version "0.1"; Codename "sarge"; Architectures "i386"; Components "main contrib"; Description "Your Sarge CD Set"; }; }; };
And the following yourcdname-di.conf (creating the directories described therein, as necessary):
Dir { ArchiveDir "/debian"; OverrideDir "indices"; CacheDir "/tmp"; };
TreeDefault { Directory "pool/"; };
BinDirectory "pool/main" { Packages "dists/sarge/main/debian-installer/binary-i386/Packages"; BinOverride "override.sarge.main"; ExtraOverride "override.sarge.extra.main"; };
Default { Packages { Extensions ".udeb"; Compress ". gzip"; };
Contents { Compress "gzip"; }; };
execute the command
apt-ftparchive -c apt.conf generate yourcdname-di.conf
- repeat for binary-i386:
using the following yourcdname.conf instead of yourcdname-di.conf:
Dir { ArchiveDir "/debian"; OverrideDir "indices"; CacheDir "/tmp"; };
TreeDefault { Directory "pool/"; };
BinDirectory "pool/main" { Packages "dists/sarge/main/binary-i386/Packages"; BinOverride "override.sarge.main"; ExtraOverride "override.sarge.extra.main"; };
BinDirectory "pool/contrib" { Packages "dists/sarge/contrib/binary-i386/Packages"; BinOverride "override.sarge.contrib"; ExtraOverride "override.sarge.extra.contrib"; };
BinDirectory "pool/non-free" { Packages "dists/sarge/non-free/binary-i386/Packages"; BinOverride "override.sarge.non-free"; ExtraOverride "override.sarge.extra.non-free"; };
Default { Packages { Extensions ".deb .udeb"; Compress ". gzip"; };
Contents { Compress "gzip"; }; };
- create a list of packages you want on the cd (probably just a list of files in your pool dir).
- Assuming you have debian-cd installed, follow the directions in /usr/share/debian-cd.
- In the mirrorcheck step, if any missing Depends: are listed, download the packages into your pool dir to correct that, regenerate your packages files, and start the debian-cd instructions over again (you may need to exit the shell you started the procedure in because debian-cd alters the environment while it builds the cd ["set"]).
- In the make list step wach for debconf_required errors, and as above download the needed packages, regenerate package lists, and restart the debian-cd procedure.
- assuming sucessful completion of the debian-cd cd-building procedure burn your cd and test.
Detailed Instructions
1. Getting Your Packages
The Hard Way
- Figure out what packages you'll need (this is the hard part because of dependencies).
Download them individually (if you make a list and use wget on the list, it wouldn't be as painful, but keeping up with changing versions would rapidly become painful) into the pool directory in the filesystem where you'll be doing your work (e.g. /home/debian/pool).
- Don't forget any dependencies (by hand! ugh!).
The Easy Way, Part I: Get Your Base System
Download a ?DebiainInstaller network install cd.
- Use the install cd and install your base system on some computer.
Copy the network install cd to the filesystem where you'll be doing your work (e.g. /home/debian).
The Easy Way, Part II: Getting Additional Packages
Use aptitude or ?AptGet to download the packages you want on the system and cd.
Get AptMove (e.g. apt-get apt-move)
Modify apt-move.conf to use your pool directory as its local mirror (LOCALDIR)
AptMove your other packages to your pool directory (e.g. /home/debian/pool) by typing: apt-move update
NB: I haven't actually tried using AptMove directly onto the partial mirror because of the way in which I learned what I needed to do, but it should work.
Notes on the use of AptMove
You will need to change apt-move.conf to point to your mirror. See AptMove for details (or man apt-move).
The Slightly Harder Way, Part II: Getting Additional Packages
Use Aptitude or ?AptGet to download the packages you want on the system and cd.
Get AptMove (e.g. apt-get apt-move)
Modify apt-move.conf to use a temporary directory as its local mirror (LOCALDIR)
AptMove your other packages to a temporary directory (e.g. /home/debian/tmp) by typing: apt-move update
Use the following script to copy packages from the temporary directory to your pool directory (e.g. /home/debian/pool). This is my preferred mode of operation because I have more control over what happens to what apt-move calls the 'mirror' (the pool directory), especially when building cd's for testing or unstable, which change quickly.
#!/bin/bash
function print_usage { echo echo "copy-new-debs-to-pool [--move] [--keep-old] dir-to-copy add-to-dir" echo echo "paths must be absolute" echo "and must point to the directory in which the pool subdirectory resides" echo "The pool directory must have main contrib and non-free subdirectories" echo }
if [ "$1" = "--move" ] ; then MOVE=1 if [ "$2" = "--keep-old" ] ; then KEEP=1 else KEEP=0 fi else MOVE=0 if [ "$1" = "--keep-old" ] ; then KEEP=1 else KEEP=0 fi fi
if [ -z "$1" ]; then print_usage exit fi
if [ -z "$2" ]; then print_usage exit fi
if [ $MOVE -eq 1 -a -z "$3" ]; then print_usage exit else if [ $KEEP -eq 1 -a -z "$3" ]; then print_usage exit fi fi
if [ $MOVE -eq 1 -a $KEEP -eq 1 -a -z "$4" ]; then print_usage exit fi
if [ $MOVE -eq 0 -a $KEEP -eq 0 ]; then ["RC1DIR"]=$1 ["RC2DIR"]=$2 else if [ $MOVE -eq 1 -a $KEEP -eq 1 ]; then ["RC1DIR"]=$3 ["RC2DIR"]=$4 else ["RC1DIR"]=$2 ["RC2DIR"]=$3 fi fi
echo "Copying from $["RC1DIR"] to $["RC2DIR"]"
TMPFILE=`tempfile`
cd $["RC1DIR"] find pool -type f || sort >$TMPFILE rc1base=$["RC1DIR"] rc2base=$["RC2DIR"] cd - for rc1file in `cat $TMPFILE`; do pkgfullname=`basename $rc1file` pkgname=`echo $pkgfullname || cut -f1 -d_` rc1ver=`echo $pkgfullname || cut -f2 -d_` pooldir=`dirname $rc1file` pkgend=`echo $pkgfullname || cut -f3 -d_` rc2file=`ls $rc2base/$pooldir/$pkgname\__*\__$pkgend 2>/dev/null` if [ "$rc2file" ]; then rc2maxver="!!!!!!!!!!!!!!!!!" rc2maxfile="!!!!!!!!!!!!!!!!" for rc2match in $rc2file; do rc2pkgfullname=`basename $rc2match` rc2ver=`echo $rc2pkgfullname || cut -f2 -d_` if [ "$rc2ver" ">" "$rc2maxver" ]; then rc2maxver=$rc2ver rc2maxfile=$rc2match fi done rc2file=$rc2maxfile if [ $rc1ver '>' $rc2maxver ] ; then echo "Copying $rc1file newer than $rc2file" if [ $MOVE -eq 1 ] ; then mv -f $rc1base/$rc1file `dirname $rc2base/$rc1file` if [ $KEEP -eq 1 ]; then echo "Not removing obsolete $rc2file" else echo "Removing obsolete $rc2file" rm -f $rc2base/$rc2file fi else cp $rc1base/$rc1file `dirname $rc1base/$rc1file` if [ $KEEP -eq 1 ]; then echo "Not removing obsolete $rc2file" else rm -f $rc2base/$rc2file fi fi continue fi else echo "Copying $pkgname not in $["RC2DIR"]" install -d $["RC2DIR"]/`dirname $rc1file` if [ $MOVE -eq 1 ]; then mv -f $["RC1DIR"]/$rc1file `dirname $["RC2DIR"]/$rc1file` else cp $["RC1DIR"]/$rc1file `dirname $["RC2DIR"]/$rc1file` fi fi done
rm -f $TMPFILE
Alternate Method
Instead of copying the network install cd first, copy your Apt cache to your desired pool directory first (as above), then use the above script to copy from the network install pool directory to your pool directory.
2. Get DebianInstaller and udebs
Option 1
Should only be used when using a stable distribution's network install cd and stable packages, for sarge or later because the udebs must match the installer for a successful install. If you insist on using it with testing or unstable, make sure you're using the right versions of the udebs, especially kernel, for the installer. You have been warned.
Download installer-i386 from a ["Debian"] mirror (e.g. ftp://ftp.debian.org/debian/dists/sarge/main/current)
- You should have the correct udebs in your tree if you have downloaded the installer-i386 from the same distribution as your network install cd
Option 2
Build DebianInstaller and udebs. (See ?DebianInstallerBuild).
- Copy udebs to your pool directory, using something like the following script:
IFS=$'\n' pushd /debian-installer/installer/build/localudebs for file in `ls`; do pkg=`echo $file || cut -f1 -d_` firstletter=`expr substr $pkg 1 1` if [ $firstletter = l ] then libletters=`expr substr $pkg 1 3` if [ $libletters = "lib" ]; then dirletters=`expr substr $pkg 1 4` mkdir -p /debian/pool/main/$dirletters/$pkg cp $file /debian/pool/main/$dirletters/$pkg else mkdir -p /debian/pool/main/$firstletter/$pkg cp $file /debian/pool/main/$firstletter/$pkg fi else mkdir -p /debian/pool/main/$firstletter/$pkg cp $file /debian/pool/main/$firstletter/$pkg fi done popd
You obviously need to change the paths. This script is not really finished, but it's good enough for now.
3. Get overrides.<dist>.*
Download /debian/indices/overrides.<dists>.'''.gz for your distro (e.g. for sarge, /debian/indices/overrides.sarge.'''.gz) into the directory indices which is in the same root directory as your pool (mirror) directory (e.g. /debian/pool, so /debian/indices).
- Gunzip the overrides, keeping the compressed files.
If you want to includes packages from non-US you ought to create another indices directory (e.g. indices-non-US), and get the overrides from a non-US mirror (from /debian-non-US/indices-non-US).
4. Generate Packages.gz and Release files for your 'distribution'
generate DebianInstaller packages files:
DebianInstaller lives in <debian-dir>''dists''<dist>/main/debian-installer/binary-i386
so create the appropriate subdirectories (e.g. mkdir -p /debian/dists/sarge/main/debian-installer/binary-i386).
in a directory (e.g. custom-cd-scripts) for your scripts and config files for this project place the following apt.conf:
APT { ["FTPArchive"] { Release { Origin "debian-cd"; Label "yousarge"; Suite "testing"; Version "0.1"; Codename "sarge"; Architectures "i386"; Components "main contrib"; Description "Your Sarge CD Set"; }; }; };
And the following customcd-di.conf (creating the directories described therein, as necessary):
Dir { ArchiveDir "/debian"; OverrideDir "indices"; CacheDir "/tmp"; };
TreeDefault { Directory "pool/"; };
BinDirectory "pool/main" { Packages "dists/sarge/main/debian-installer/binary-i386/Packages"; BinOverride "override.sarge.main"; ExtraOverride "override.sarge.extra.main"; };
Default { Packages { Extensions ".udeb"; Compress ". gzip"; };
Contents { Compress "gzip"; }; };
ArchiveDir is the parent directory of your pool and dists subdirectories
OverrideDir is the directory which contains your overrides Note that this means that if you want non-US, you need another .conf with your non-US directories.
CacheDir is your temporary (cache) directory.
Under TreeDefault, Directory indicates the location of your pool (archive) directory
The path in quotes after BinDirectory (and before the curly brace), is the location, relative to ArchiveDir of the binary packages (well actually the alphabetic directories, which have the package directories under them).
Under BinDirectory, Packages is the full pathname of the Packages (without .gz even if you are creating compressed files) file you are creating (in this case the debian-installer Packages.gz).
execute the command
apt-ftparchive -c apt.conf generate yourcdname-di.conf
repeat for the binary-i386 (main) directory:
using the following customcd.conf instead of customcd-di.conf:
Dir { ArchiveDir "/debian"; OverrideDir "indices"; CacheDir "/tmp"; };
TreeDefault { Directory "pool/"; };
BinDirectory "pool/main" { Packages "dists/sarge/main/binary-i386/Packages"; BinOverride "override.sarge.main"; ExtraOverride "override.sarge.extra.main"; };
BinDirectory "pool/contrib" { Packages "dists/sarge/contrib/binary-i386/Packages"; BinOverride "override.sarge.contrib"; ExtraOverride "override.sarge.extra.contrib"; };
BinDirectory "pool/non-free" { Packages "dists/sarge/non-free/binary-i386/Packages"; BinOverride "override.sarge.non-free"; ExtraOverride "override.sarge.extra.non-free"; };
Default { Packages { Extensions ".deb .udeb"; Compress ". gzip"; };
Contents { Compress "gzip"; }; };
If you are using any non-US packages you will need to repeat the above Packages file creation in each of the appropriate subdirectories.
... to be continued ...