Differences between revisions 23 and 25 (spanning 2 versions)
Revision 23 as of 2008-02-03 11:31:47
Size: 5045
Editor: RonnyAasen
Comment:
Revision 25 as of 2008-10-04 11:03:49
Size: 6219
Editor: HolgerLevsen
Comment:
Deletions are marked like this. Additions are marked like this.
Line 6: Line 6:
 * tasks comes from debian-edu svn  * tasks comes from debian-edu svn (from the tasks dir, but somehow also the changelog version is used. Which h01ger doesnt understand, because the changelog says lenny-test but still, other images are build too)
Line 11: Line 11:
'''someday someone might describe here whats needed to debug failing builds'''
{{{
<h01ger> white, pere, Werner, sep: can you give me hints how to debug the magic cd build?
<h01ger> i'm willing to update the docs :)
<h01ger> why do we have dillo on our cd? its nice but outdated and quite unmaintained..
<white> well i guess becoming the builder user and then checking the Makefile in the ~builder/.../CD-administrator dir and running the build commands should eventually give you the error
<h01ger> how do i run the build commands?
<white> no idea to be honest, it should be in the Makefile, one of the commands there
<pere> h01ger: setting some environment variable should give you a build with output to stdout.
<pere> you have to check the scripts to find it. possibly build.sh.
<white> you can also debug it by just running the cronjob and checking what the script is doing via the process list
<pere> the build log is copied to ~ftp/cd-*/ now, so you could have a look.
}}}

Any sufficiently advanced technology is indistinguishable from magic...

This is in the very early stage of something which should become a howto to understand the build process of the Debian Edu (install) cd - please help to improve it. I will do so too, on my way to eternal wisdom and happyness :-)

  • tasks comes from debian-edu svn (from the tasks dir, but somehow also the changelog version is used. Which h01ger doesnt understand, because the changelog says lenny-test but still, other images are build too)
  • wantedpkglist*.txt is generated from tasks
  • wantedpkglist is given to debian-cd to populate the cd
  • missingpkglist is generated by comparing the list of packages on the CD with the list of wanted packages.

someday someone might describe here whats needed to debug failing builds

<h01ger> white, pere, Werner, sep: can you give me hints how to debug the magic cd build?
<h01ger> i'm willing to update the docs :)
<h01ger> why do we have dillo on our cd? its nice but outdated and quite unmaintained..
<white> well i guess becoming the builder user and then checking the Makefile in the ~builder/.../CD-administrator dir and running the build commands should eventually give you the error
<h01ger> how do i run the build commands?
<white> no idea to be honest, it should be in the Makefile, one of the commands there
<pere> h01ger: setting some environment variable should give you a build with output to stdout.
<pere> you have to check the scripts to find it.  possibly build.sh.
<white> you can also debug it by just running the cronjob and checking what the script is doing via the process list
<pere> the build log is copied to ~ftp/cd-*/ now, so you could have a look.

debian-cd

We are using two different branches of debian-cd. The svn://svn.debian.org/debian-cd/etch_r0_build branch is used to build etch cd's. And svn://svn.debian.org/debian-cd/trunk is used to build post-etch cd's. The reason for this is, that when debian release a revision update, they update the etch branch and we'd have to backport those updates in our patch if we wanted to build etch using trunk.

the upstream code for the version of debian-cd we're using is at /home/builder/src/debian-edu/src/build/CD-administrator/debian-cd.[etch|trunk] The upstream code is updated from svn by the patch-debian-cd script and uses the revision number found in the debian-cd-revision file. This makes it easier to merge in ustream changes when we have time to update our patch. And also easier to move back incase of severe breakage.

Since we use 2 branches we also have 2 patch files. debian-cd.patch.[etch|trunk]

During every build the matching upstream branch is patched by debian-edu/src/build/CD-administrator/patch-debian-cd <$branch> with debian-edu/src/build/CD-administrator/debian-cd.patch.$branch the result is put into debian-cd, where the build is executed.

If you want to exclude certain packages from the cd, there is a exclude-etch list (also used for etch-test) in that patch.

cd-build logs

the logs from each cd build are commited to svn (debian-edu/html/logs/cd-build-$suite.log). the commits are silent, so no commit-mail is sent. You can read the logs in the individual cd-suite directories in http://ftp.skolelinux.no

force re-build

all developers can force a build by using the command /home/builder/bin/trigger_builds on user.skolelinux.org - trigger_build takes on argument: the image to rebuild or "all".

how to exclude packages from the cd

Get the curret status

  1. grab CD-administrator with svn co svn+ssh://$alioth-user-name@svn.debian.org/svn/debian-edu/trunk/src/build/CD-administrator

  2. cd CD-administrator
  3. patch using ./patch-debian-cd

edit the patch

  • /!\ make-patch is not working at the moment

  • edit the exlude file with vi debian-cd/tasks/exclude-etch

  • regenerate the patch with ./make-patch, this will generate a debian-cd.patch file.

  • Test that your patch works with ./patch-debian-cd $branch

  • commit your changes to the patchfile with svn commit debian-cd.patch

  • have a beer and wait for the cd to build

how to merge in changes from upstream

  1. increment debian-cd-revision, svn
  2. patch both the trunk and the etch branch using using ./patch-debian-cd [etch|trunk]

  3. if both patches succeeds, you can commit debian-cd-revision, and wait for the cd to build.
  4. if one (or both) of the patches fail, you'll have the half baked results in debian-cd.new. Check thru the rejects and adjust debian-cd.patch.[etch|trunk] as needed. Test that your modified patch works with ./patch-debian-cd [etch|trunk]

  5. commit the debian-cd-revision, and debian-cd.patch.$branch files
  6. have a $beverage while it's building.
  7. test the cd

How to build more images

There are a few thing one must in order to add a new cd to the build system.

  1. make the temp file area.
    • debian-cd uses a area on disk, for tempfile area, building the filesystem that becomes the iso, and making the iso.

      you need to make the directory owned by builder as /skolelinux/administrator/temp_storage/builder/$suite. For example /skolelinux/administrator/temp_storage/builder/lenny-test-dvd/  You should also check that the area have the needed free space 1.5G for a cd and FIXME: GB for the dvd

  2. Make a new config file called CONF-$suite, eg CONF-lenny-test-dvd.sh, and edit it to suite your needs
  3. add the suite to the Makefile, in the suites= list
  4. add the suite to the trigger tool, located in builder's ~/bin/trigger_builds
  5. after the first build. you should add the files wantedpkglist-$suite filelist-$suite pkgdeblist-$suite missingpkglist-$suite cdspacelist-$suite to svn. If you don't you will not recive commit mails after the build.
  6. if you add a new version, you must also make sure there is a symlink to our local -test repo in the temp_files/mirror-link

buildd admins

Members of the builder group on administrator are:

  • Petter
  • Steffen
  • Ronny

famous last words

... and any technology distinguishable from magic is not sufficiently advanced!