Differences between revisions 2 and 94 (spanning 92 versions)
Revision 2 as of 2007-05-10 09:13:16
Size: 5470
Editor: ?HvWoerkom
Comment:
Revision 94 as of 2014-07-03 08:17:10
Size: 9221
Editor: ?Beuc
Comment: apt-get update for good measure
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= This Page is about Rebuilding an official Debian kernel package =
There is the Kernel Handbook about this:
http://kernel-handbook.alioth.debian.org/
#language en
||<tablestyle="width: 100%;" style="border: 0px hidden">~-[[DebianWiki/EditorGuide#translation|Translation(s)]]: English - [[it/HowToRebuildAnOfficialDebianKernelPackage|Italiano]]-~||<style="text-align: right;border: 0px hidden"> (!) [[/Discussion|Discussion]]||

= Building custom kernel packages =

This page is about rebuilding an official Debian kernel package with custom changes. There is the Kernel Handbook about this: http://kernel-handbook.alioth.debian.org/ (which explains how to "Rebuild an official Debian linux kernel package", hopefully : http://kernel-handbook.alioth.debian.org/ch-common-tasks.html ch 4.2 : "Rebuilding an official Debian kernel package").
Line 7: Line 10:
== Reasons to do this ==
 1. Modifying the Debian configuration to enable the Nvidia closed source driver creation. Specifically in 2.6.20 if '''CONFIG_PARAVIRT''' is set, the Nvidia driver won't build, you might want to change that setting.
 1.#2 Modifying the Debian configuration to change the '''preemption model'''.
 1.#3 Modifying the Debian configuration to change the '''timer frequency'''.
 1.#4 Modifying the Debian configuration to change the '''Logo'''.
 1.#5 Modifying the Debian configuration to add '''kernel timings'''.
<<Anchor(reasons)>> '''Reasons to do this:'''
Line 14: Line 12:
What you'll get is exactly the kernel that Debian releases but with those added changes.  1. Modifying the Debian configuration to change the '''preemption model'''.
 1. Modifying the Debian configuration to change the '''timer frequency'''.
 1. Add a patch fix.
 1. Add your favorite patchset.
What you'll get is exactly the kernel that Debian releases but with those added changes. The new packages will not step on the official kernel images' toes.
Line 16: Line 18:
== Presumption ==
You are just interested in building one flavor of a kernel-image.
Line 19: Line 19:
== 10 easy (albeit time consuming) steps ==
 1. Create the directory where you will build the kernel package as a normal user.
 1.#2 Get the kernel source. You could install the kernel source package:
== For the impatient ==

Example with [[http://www.frankvm.com/nfs-ngroups/|NFS ngroups patch]] and the [[https://packages.debian.org/source/stable/linux|latest stable kernel]]:
Line 23: Line 23:
apt-get install linux-source-2.6 # Run this in the same distro version than the recompiled kernel, possibly in a chroot
apt-get install devscripts equivs wget
wget http://www.frankvm.com/nfs-ngroups/3.2.35-nfs-ngroups-4.59.patch
apt-get update
apt-get source linux
dpkg-source -x linux_3.2.57-3+deb7u2.dsc
cd linux-3.2.57/
mk-build-deps -i
sed -i -e 's/~test/~nfs/g' debian/bin/test-patches # version suffix
sed -i -e 's/^abiname:.*/&nfs/' debian/config/defines # package suffix
time debian/bin/test-patches -j$(nproc) ../3.2.35-nfs-ngroups-4.59.patch # maybe use --fuzz 3, but be careful
fakeroot make -f debian/rules.gen binary-arch_amd64_none_real # if you need the linux-headers-XXX-common package e.g. for DMKS modules
# As for 2014-06, after ~45mn using -j8, and 10Go disk space, you get your .deb's :)
Line 25: Line 37:
But that will get the entire kernel source. Very large. A better approach is to keep up to date on the latest stable kernels from http://www.kernel.org and download the patch files when they become available. Then you just run a script to install the source into your directory:
<<Anchor(add-a-patch)>>
== Add a patch to linux and build new kernel packages ==
 1. Create the directory where you will build the kernel package as a normal user and make sure you have all necessary packages installed: {{{
apt-get install fakeroot build-essential devscripts
apt-get build-dep linux
}}}

 1. Get the package source. For this to work, you need to have {{{deb-src}}} lines for the official archive in {{{/etc/apt/sources.list}}}: {{{
apt-get source linux
cd linux-*
}}}
 Now you are in the tree that holds the source and all changes Debian made to it. As an alternative, you could get a kernel tarball, e.g. from buildserver (see DebianKernel), unpack it, and get the latest Debian patches + config settings: {{{
tar xzf linux_3.0.0~rc6.orig.tar.gz
cd linux-*
svn export svn://svn.debian.org/svn/kernel/dists/trunk/linux/debian
}}}

 1. Look at the top of {{{debian/changelog}}} and think of an appropriate version number. You cannot {{{dch}}} yet, though. If you are adding a single patch "foo", you might want to append "+foo.1". If you are adding multiple patches for your company or yourself, use "+somename.1".

 1. Append the same string to the {{{abi.abiname}}} field in {{{debian/config/defines}}} to ensure that your custom kernel package name doesn't bite with the official ones. In our case, we're now creating {{{linux-image-3.x.0-y+foo.1-zzz}}}.

 1. Use quilt to add your patch

 1. Optional : If it's not the first build attempt, and should you need to clean things in case of build failures : {{{
make -f debian/rules clean
}}}

 1. Now check that your patchset still applies and fix any conflicts:
 Note: on the first run, the target fails. Make sure you run it twice.
 {{{
This target is made to fail intentionally, to make sure
that it is NEVER run during the automated build. Please
ignore the following error, the debian/control file has
been generated SUCCESSFULLY.
}}}

 {{{
make -f debian/rules source-all
}}}

 1. Now that the above error message has occured after the first “debian/rules clean”, {{{dch}}} will work because {{{debian/control}}} has been (re-)generated. (Note that, after editing {{{debian/changelog}}}, “debian/rules clean” must again be run because its hash changes.) Start a new stanza at the top of {{{debian/changelog}}} and use an appropriate version number. Ensure that the distribution is '''not''' set to the regular release names. Change it to something like '''UNRELEASED.''' You could do this with the {{{dch}}} tool from the {{{devscripts}}} package like so: {{{
dch --local +foo.
}}}

 1. Document the addition of this patch in the new stanza in {{{debian/changelog}}}.

<<Anchor(build-all)>>
=== Building all kernel-related packages ===
 0. Make sure you have at least 40 GB of disk space free (version 2.6.32-17 took 30 GB).

 1. Assuming you want to build '''all''' kernel-related packages, build with the appropriate {{{DEBIAN_KERNEL_JOBS}}} parameter (the number of processors you can use for the build): {{{
~/src/linux-3.2.31$ debuild -e DEBIAN_KERNEL_JOBS=$(nproc)
}}}

If you plan to do multiple rebuilds you might want to use ccache by adding the --prepend-path=/usr/lib/ccache option to the above debuild command.

<<Anchor(build-one)>>
=== Building only a single kernel variant ===
 0. Make sure you have at least 10 GB of disk space free (setup_amd64_none_amd64 took 7.3 GB in version 2.6.32-17).

 1. Prepare the rules file for the one build you want to make, e.g.: {{{
fakeroot make -f debian/rules.gen setup_i386_none_686-pae
}}}
 This will build a tree in the {{{debian/build/build_i386_none_686-pae}}} directory. Go to that directory.

 1. Change the {{{.config}}} file, using e.g. {{{
make menuconfig
cd ../../..
}}}

 1. Compile the kernel and generate the kernel packages. Replace {{{$NR_CPUS}}} with the number of CPUs of the build machine (keeping it all on the same line) and run e.g. {{{
fakeroot make -f debian/rules.gen binary-arch_i386_none_686-pae binary-indep \
DEBIAN_KERNEL_JOBS=${NR_CPUS}
}}}
 Some time later the package files will be generated, e.g.: {{{
linux-image-3.2.0-4+foo.1-686-pae_3.2.31-1+foo.1_i386.deb
linux-image-3.2.0-4+foo.1-686-pae-dbg_3.2.31-1+foo.1_i386.deb
linux-headers-3.2.0-4+foo.1-686-pae_3.2.31-1+foo.1_i386.deb
xen-linux-system-3.2.0-4+foo.1-686-pae_3.2.31-1+foo.1_i386.deb
}}}
 '''Problem''': In this case, {{{linux-headers-3.2.0-4+foo.1-common}}} will be missing. One needs to invoke the {{{binary-arch_i386}}} target, which will yield all feature sets (RT) and flavours to be generated, and obviously takes a lot longer. See [[http://lists.debian.org/debian-kernel/2008/04/msg00190.html|this thread]].
 Solution: {{{
fakeroot make -f debian/rules.gen binary-arch_i386_none_real
}}}

 1. If you apply further changes to the source package and need to rebuild the binary packages without rebuilding each and every binary objects, you can remove the build stamp before running the above mentioned building command. e.g. {{{
rm debian/stamps/build_i386_none_686-pae_plain
}}}

== The story of linux-tools ==

The {{{linux-headers-*}}} packages created with the above method depend on {{{linux-kbuild-*}}}, which is a '''not''' built from the {{{linux}}} source package, but from {{{linux-tools}}}.

Often, when a new kernel comes around, {{{linux-kbuild-3.xx}}} isn't yet available in the archive, so you either have to build it yourself, or wait.

A [[http://lists.debian.org/debian-kernel/2012/08/msg00390.html|Mini-Howto]] from Sedat (dileks) gives detailed instructions how to build {{{linux-kbuild-3.5}}}. An integration here would blow-up this wiki unnecessarily.

=== How to build linux-tools yourself ===

Checkout the source from SVN repository: {{{
svn co svn://svn.debian.org/kernel/dists/trunk/linux-tools
}}}
Then, download the vanilla kernel tarball (usually the 3.x version, no 3.x.y version): {{{
VERSION="3.6"
wget http://ftp.kernel.org/pub/linux/kernel/v3.x/linux-${VERSION}.tar.bz2
}}}
Now, you prepare the Debian package:
Line 27: Line 146:
#!/bin/bash
set -x
PLACE=/home/hugo/linux-2.6.20-3-build
cd linux-tools
./debian/bin/genorig.py ../linux-${VERSION}.tar.bz2
cd ..
tar -xzf orig/linux-tools_${VERSION}.orig.tar.gz
cd linux-tools-${VERSION}
cp -a ../linux-tools/* ./
./debian/bin/gencontrol.py
}}}
Line 31: Line 155:
cd $PLACE
tar -xjvf /hda10/backup.files/linux-2.6.9.tar.bz2
cd $PLACE/linux-2.6.9
patch -p1 < /hda10/backup.files/patch-2.6.10
cd $PLACE
mv $PLACE/linux-2.6.9 $PLACE/linux-2.6.10
cd $PLACE/linux-2.6.10
patch -p1 < /hda10/backup.files/patch-2.6.11
cd $PLACE
mv $PLACE/linux-2.6.10 $PLACE/linux-2.6.11
cd $PLACE/linux-2.6.11
patch -p1 < /hda10/backup.files/patch-2.6.12
cd $PLACE
mv $PLACE/linux-2.6.11 $PLACE/linux-2.6.12
cd $PLACE/linux-2.6.12
patch -p1 < /hda10/backup.files/patch-2.6.13
cd $PLACE
mv $PLACE/linux-2.6.12 $PLACE/linux-2.6.13
cd $PLACE/linux-2.6.13
patch -p1 < /hda10/backup.files/patch-2.6.14
cd $PLACE
mv $PLACE/linux-2.6.13 $PLACE/linux-2.6.14
cd $PLACE/linux-2.6.14
patch -p1 < /hda10/backup.files/patch-2.6.15
cd $PLACE
mv $PLACE/linux-2.6.14 $PLACE/linux-2.6.15
cd $PLACE/linux-2.6.15
patch -p1 < /hda10/backup.files/patch-2.6.16
cd $PLACE
mv $PLACE/linux-2.6.15 $PLACE/linux-2.6.16
cd $PLACE/linux-2.6.16
patch -p1 < /hda10/backup.files/patch-2.6.17
cd $PLACE
patch -p1 < /hda10/backup.files/patch-2.6.17
cd $PLACE
mv $PLACE/linux-2.6.16 $PLACE/linux-2.6.17
cd $PLACE/linux-2.6.17
patch -p1 < /hda10/backup.files/patch-2.6.18
cd $PLACE
mv $PLACE/linux-2.6.17 $PLACE/linux-2.6.18
cd $PLACE/linux-2.6.18
patch -p1 < /hda10/backup.files/patch-2.6.19
cd $PLACE
mv $PLACE/linux-2.6.18 $PLACE/linux-2.6.19
cd $PLACE/linux-2.6.19
patch -p1 < /hda10/backup.files/patch-2.6.20
cd $PLACE
mv $PLACE/linux-2.6.19 $PLACE/linux-2.6.20
cd $PLACE/linux-2.6.20
exit
NOTE: Optionally, you have to change Debian-version in debian/changelog if $VERSION of Linux-kernel is higher, think of 3.5 (latest version in linux-tools SVN-branch) is smaller than 3.6-rcX (version of tarball), otherwise the genorig.py line fails.

Finally, adjust the Debian version and add comments like "Non-maintainer upload" plus "New upstream version" via dch command. Last but not least, build the package itself after you installed eventually missing build-dependencies: {{{
dch -i
make -f debian/rules clean
dpkg-checkbuilddeps
cp -av ../orig/linux-tools_${VERSION}.orig.tar.gz ../ <--- Necessary to get a diff file.
dpkg-buildpackage -us -uc (or debuild)
Line 82: Line 164:
That gets the '''2.6.9 source''' from the HDD and patches it for every version up to 2.6.20. and you are done.
Line 84: Line 166:
 1.#3 Download the Debian diff.gz file. That is the file that contains all the Debian changes for this kernel package. E.g http://ftp.debian.org/debian/pool/main/l/linux-2.6/linux-2.6_2.6.20-3.diff.gz
 1.#4 Patch the source with that. Cd into the tree of where you put the source, e.g. /home/hugo/linux-2.6.20-3-build/linux-2.6.20. You created linux-2.6.20-3-build and your script put linux-2.6.20 into it, cd into the latter and then:
{{{
zcat <path>linux-2.6_2.6.20-3.diff.gz| patch -p1
}}}
Now your tree holds the source and all changes Debian made to it.
 1.#5 '''Make debian/rules executable'''. In your tree there is a directory named '''debian''', in there is a file called '''rules''', that has to be made executable, because it is executed.
 1.#6 '''Change the file debian/bin/abicheck.py'''. After line 18 that contains '''ret = 0''' add a line that says '''return 0'''. This will stop you from doing the abicheck.
 1.#7 Prepare the tree for build. Still in the tree directory where you patched, do:
{{{
fakeroot debian/rules debian/build debian/stamps
fakeroot make -f debian/rules.gen setup-i386-none-k7
}}}
This presumes you are only interested in building a k7 AMD processor image! If you where interested in a 686 processor image do:
{{{
fakeroot make -f debian/rules.gen setup-i386-none-686
}}}
This will build a tree in the debian/build/build-i386-none-k7 or debian/build/build-i386-none-686 directory. Cd to that directory.
 1.#8 Change the .config file. Do:
{{{
make menuconfig
}}}
and make any changes you had in mind. E.g the changes mentioned above in '''Reasons for doing this'''. Also this is the time to add additional patches, e.g. the Debian Logo patch.
 1.#9 Compile the kernel and generate the image and headers debs. Do:
{{{
fakeroot make -f debian/rules.gen binary-arch-i386-none-k7
}}}
or:
{{{
fakeroot make -f debian/rules.gen binary-arch-i386-none-686
}}}
And on a 2.0GHz CPU the compile will take 1 hour and 20 minutes and 2 deb files will be generated: the image file and the headers file, e.g.
{{{
linux-image-2.6.20-1-k7_2.6.20-3_i386.deb
linux-headers-2.6.20-1-k7_2.6.20-3_i386.deb
}}}
 1.#10 Install them with dpkg and now you have installed the stock Debian kernel with your changes in it. Congratulations ;-)
-----
 . CategoryKernel

Translation(s): English - Italiano

(!) ?Discussion

Building custom kernel packages

This page is about rebuilding an official Debian kernel package with custom changes. There is the Kernel Handbook about this: http://kernel-handbook.alioth.debian.org/ (which explains how to "Rebuild an official Debian linux kernel package", hopefully : http://kernel-handbook.alioth.debian.org/ch-common-tasks.html ch 4.2 : "Rebuilding an official Debian kernel package").

We are following section 4.2: Rebuilding an official Debian kernel package

Reasons to do this:

  1. Modifying the Debian configuration to change the preemption model.

  2. Modifying the Debian configuration to change the timer frequency.

  3. Add a patch fix.
  4. Add your favorite patchset.

What you'll get is exactly the kernel that Debian releases but with those added changes. The new packages will not step on the official kernel images' toes.

For the impatient

Example with NFS ngroups patch and the latest stable kernel:

# Run this in the same distro version than the recompiled kernel, possibly in a chroot
apt-get install devscripts equivs wget
wget http://www.frankvm.com/nfs-ngroups/3.2.35-nfs-ngroups-4.59.patch
apt-get update
apt-get source linux
dpkg-source -x linux_3.2.57-3+deb7u2.dsc
cd linux-3.2.57/
mk-build-deps -i
sed -i -e 's/~test/~nfs/g' debian/bin/test-patches  # version suffix
sed -i -e 's/^abiname:.*/&nfs/' debian/config/defines  # package suffix
time debian/bin/test-patches -j$(nproc) ../3.2.35-nfs-ngroups-4.59.patch  # maybe use --fuzz 3, but be careful
fakeroot make -f debian/rules.gen binary-arch_amd64_none_real  # if you need the linux-headers-XXX-common package e.g. for DMKS modules
# As for 2014-06, after ~45mn using -j8, and 10Go disk space, you get your .deb's :)

Add a patch to linux and build new kernel packages

  1. Create the directory where you will build the kernel package as a normal user and make sure you have all necessary packages installed:

    apt-get install fakeroot build-essential devscripts
    apt-get build-dep linux
  2. Get the package source. For this to work, you need to have deb-src lines for the official archive in /etc/apt/sources.list:

    apt-get source linux
    cd linux-*

    Now you are in the tree that holds the source and all changes Debian made to it. As an alternative, you could get a kernel tarball, e.g. from buildserver (see DebianKernel), unpack it, and get the latest Debian patches + config settings:

    tar xzf linux_3.0.0~rc6.orig.tar.gz
    cd linux-*
    svn export svn://svn.debian.org/svn/kernel/dists/trunk/linux/debian
  3. Look at the top of debian/changelog and think of an appropriate version number. You cannot dch yet, though. If you are adding a single patch "foo", you might want to append "+foo.1". If you are adding multiple patches for your company or yourself, use "+somename.1".

  4. Append the same string to the abi.abiname field in debian/config/defines to ensure that your custom kernel package name doesn't bite with the official ones. In our case, we're now creating linux-image-3.x.0-y+foo.1-zzz.

  5. Use quilt to add your patch
  6. Optional : If it's not the first build attempt, and should you need to clean things in case of build failures :

    make -f debian/rules clean
  7. Now check that your patchset still applies and fix any conflicts: Note: on the first run, the target fails. Make sure you run it twice.
    This target is made to fail intentionally, to make sure
    that it is NEVER run during the automated build. Please
    ignore the following error, the debian/control file has
    been generated SUCCESSFULLY.
    make -f debian/rules source-all
  8. Now that the above error message has occured after the first “debian/rules clean”, dch will work because debian/control has been (re-)generated. (Note that, after editing debian/changelog, “debian/rules clean” must again be run because its hash changes.) Start a new stanza at the top of debian/changelog and use an appropriate version number. Ensure that the distribution is not set to the regular release names. Change it to something like UNRELEASED. You could do this with the dch tool from the devscripts package like so:

    dch --local +foo.
  9. Document the addition of this patch in the new stanza in debian/changelog.

  1. Make sure you have at least 40 GB of disk space free (version 2.6.32-17 took 30 GB).
  2. Assuming you want to build all kernel-related packages, build with the appropriate DEBIAN_KERNEL_JOBS parameter (the number of processors you can use for the build):

    ~/src/linux-3.2.31$ debuild -e DEBIAN_KERNEL_JOBS=$(nproc)

If you plan to do multiple rebuilds you might want to use ccache by adding the --prepend-path=/usr/lib/ccache option to the above debuild command.

Building only a single kernel variant

  1. Make sure you have at least 10 GB of disk space free (setup_amd64_none_amd64 took 7.3 GB in version 2.6.32-17).
  2. Prepare the rules file for the one build you want to make, e.g.:

    fakeroot make -f debian/rules.gen setup_i386_none_686-pae

    This will build a tree in the debian/build/build_i386_none_686-pae directory. Go to that directory.

  3. Change the .config file, using e.g.

    make menuconfig
    cd ../../..
  4. Compile the kernel and generate the kernel packages. Replace $NR_CPUS with the number of CPUs of the build machine (keeping it all on the same line) and run e.g.

    fakeroot make -f debian/rules.gen binary-arch_i386_none_686-pae binary-indep \
    DEBIAN_KERNEL_JOBS=${NR_CPUS}

    Some time later the package files will be generated, e.g.:

    linux-image-3.2.0-4+foo.1-686-pae_3.2.31-1+foo.1_i386.deb
    linux-image-3.2.0-4+foo.1-686-pae-dbg_3.2.31-1+foo.1_i386.deb
    linux-headers-3.2.0-4+foo.1-686-pae_3.2.31-1+foo.1_i386.deb
    xen-linux-system-3.2.0-4+foo.1-686-pae_3.2.31-1+foo.1_i386.deb

    Problem: In this case, linux-headers-3.2.0-4+foo.1-common will be missing. One needs to invoke the binary-arch_i386 target, which will yield all feature sets (RT) and flavours to be generated, and obviously takes a lot longer. See this thread. Solution:

    fakeroot make -f debian/rules.gen binary-arch_i386_none_real
  5. If you apply further changes to the source package and need to rebuild the binary packages without rebuilding each and every binary objects, you can remove the build stamp before running the above mentioned building command. e.g.

    rm debian/stamps/build_i386_none_686-pae_plain

The story of linux-tools

The linux-headers-* packages created with the above method depend on linux-kbuild-*, which is a not built from the linux source package, but from linux-tools.

Often, when a new kernel comes around, linux-kbuild-3.xx isn't yet available in the archive, so you either have to build it yourself, or wait.

A Mini-Howto from Sedat (dileks) gives detailed instructions how to build linux-kbuild-3.5. An integration here would blow-up this wiki unnecessarily.

How to build linux-tools yourself

Checkout the source from SVN repository:

svn co svn://svn.debian.org/kernel/dists/trunk/linux-tools

Then, download the vanilla kernel tarball (usually the 3.x version, no 3.x.y version):

VERSION="3.6"
wget http://ftp.kernel.org/pub/linux/kernel/v3.x/linux-${VERSION}.tar.bz2

Now, you prepare the Debian package:

cd linux-tools
./debian/bin/genorig.py ../linux-${VERSION}.tar.bz2
cd ..
tar -xzf orig/linux-tools_${VERSION}.orig.tar.gz
cd linux-tools-${VERSION}
cp -a ../linux-tools/* ./
./debian/bin/gencontrol.py

NOTE: Optionally, you have to change Debian-version in debian/changelog if $VERSION of Linux-kernel is higher, think of 3.5 (latest version in linux-tools SVN-branch) is smaller than 3.6-rcX (version of tarball), otherwise the genorig.py line fails.

Finally, adjust the Debian version and add comments like "Non-maintainer upload" plus "New upstream version" via dch command. Last but not least, build the package itself after you installed eventually missing build-dependencies:

dch -i
make -f debian/rules clean
dpkg-checkbuilddeps
cp -av ../orig/linux-tools_${VERSION}.orig.tar.gz ../ <--- Necessary to get a diff file.
dpkg-buildpackage -us -uc (or debuild)

and you are done.