Title: Hurd Debian-Installer (draft)

Student: Jeremie Koenig <jk@jk.fr.eu.org>, jkoenig@{freenode,oftc}

Mentor: Samuel Thibault (youpi) <sthibault@debian.org>

Abstract: Debian GNU/Hurd is currently installed either using outdated CD images, or from an existing Debian GNU/Linux system using the 'crosshurd' package. The goal of this project is to modify debian-installer and the related packages to produce working Debian GNU/Hurd installation images.

Update 2010-05-24: The proposal has been accepted by Debian. I will keep track of my progress on the SummerOfCode2010/HurdDebianInstaller/JeremieKoenig/Roadmap page.

Background

I'm a 25 years old Computer Science student in Strasbourg (France), currently finishing my Licence (more or less equivalent to a bachelor's degree I think), after some years of interruption.

In particular I worked for some time as a developer on a Debian-based product (an "embedded" backup server with a web interface). This involved creating custom packages and modifying existing ones, automating the system's administration and installation, as well as some C, Perl, PHP and C# programming. Though the work in itself was fun and went rather smoothly, it was not manageable as a part-time job so I had to quit to resume my studies.

I have never been a regular Debian contributor, but I have used it for quite some time (see the list of bugs I reported) and am familiar with the developer tools. I have also done some debian-installer work in the past, namely helping with "oldworld" Macintosh support. I had to tweak the d-i image build scripts, package the miBoot bootloader and create the rsrce package (used to configure miBoot). Unfortunately, there were legal problems with miBoot and the Macintosh boot sector which the floppy used, so my work could not be completely integrated.

As for the Hurd I am less familiar with its guts, but I am enthusiastic about its overall design and I had tried it in the past. Preparing this application also involved some looking under the hood and tinkering with it, so I am confident that I would be able to acquire the necessary knowledge without too much of a hassle.

Project overview

Hurd is micro-kernel based system: most of the code usually found in the kernel of Unix-like systems is implemented in userspace, as a series of cooperating server processes. The filesystem is used as a namespace for the services that the system provides (the servers are said to translate parts of the filesystem), and the principle of capabilities is used to manage access to resources. This design offers great flexibility, as even a normal user can use to own servers, in addition to or as a replacement to the system ones.

I would work on a native debian-installer port for Hurd. Colin Watson and Samuel Thibault have already done some work in this direction, but much remains. Besides the installer itself, the project would involve at least ?BusyBox, GNU Mach and Hurd. Furthermore, I intend to coordinate with the kFreeBSD people and come up with generic solutions for non-Linux systems whenever possible.

Benefits to Debian

I believe that Debian, the universal operating system, benefits from non-Linux ports in general:

More specifically, a Hurd debian-installer would be an important step towards making hurd-i386 a viable Debian architecture (though, admittedly, lots of work would remain). Making debian-installer less tied to a particular kernel would benefit non-Linux Debian architectures in general.

Deliverables

My work would primarily consist of changes to existing software. I would submit them as I go, as patches against Debian packages and upstream where appropriate. I would also keep track of them, so I can submit them to Google after GSoC is over.

Project details

Current status

Samuel Thibault has some packages and patches which can be used to build "monolithic" debian-installer images, which at some point he could boot into a shell. I have fixed a bug in ext2fs which prevented them from booting but as of 2010-04-06, another problem with the bootstrap process remains (namely, it hangs after the exec server is started by ext2fs.)

Busybox

I have started working on "clean" patches for porting busybox. My first patches have been accepted upstream with some changes and the source from the upstream git repository builds on Hurd with a minimal configuration. However more applets will probably be needed as I progress with the installer.

In any case, some applets will need to be disabled in the Debian package on Hurd and kFreeBSD. This could be done either by using system-specific configuration files in the Debian package, or by modifying the build system of busybox: the current target system would be detected, and the irrelevant applets would be disabled through KConfig dependencies.

Genext2fs

The ext2fs translator only supports filesystems with 4KB blocks right now. On the other hand, genext2fs uses a fixed block size of 1024. The problem could be attacked on either side, but the genext2fs route seems much more simple. Samuel Thibault has a patch which changes the hardwired #define from 1024 to 4096, and sets the superblock's "creator OS" field to a system-specific value. In the longer term, however, these would need to be turned into command-line options.

GNU Mach

Mach will need to support using some of its multiboot modules as initial ramdisks. I have had an occasion to wander around the code for iopl and the "driver" part for this does not seem too hard. The user interface might be fitted into the boot script language, for instance a $(use-as-initrd) statement on a module's command line would trigger its usage as a ramdisk device.

The decompression could be done either by GRUB or by libstore. In either case the ramdisk could be used read-write.

Hurd

Crosshurd currently does some magic in the "native-install" script to set up translators on a newly installed system. This magic would need to be moved into the hurd package's postinst script so that it happens during d-i installations.

Tuning hurd-udeb and further changes might be necessary as well. For instance, there are currently workarounds for the installer in the Debian package's libstore bootstrap code and I strongly suspect that they will need some changes or cleaning up.

Network configuration

Apparently, netcfg has had Hurd support for some time (r548, by David Whedon starts it, the more recent r62649 by Samuel Thibault adds some fixes). However, I'm not sure that DHCP is supported, which it also isn't in a straightforward way on an installed Hurd system.

So the situation would need to be reviewed and support for configuring DHCP both during the install and on the target system would need to be implemented. For the record, Samuel Thibault has submitted a patch which adds Hurd support to dhcp3.

Partitioning

Not every filesystem can be used for a Hurd installation: support for passive translators is necessary. As a consequence, we have to use ext2 as the root filesystem, with special options at mkfs time as mentioned above for genext2fs. More generally, Hurd only has a subset of the filesystems which Linux understands. Partman would have to be modified to stay within these constraints on Hurd installations.

Bootloader installation

Both os-prober and grub-installer are Linux-specific for now, though there is some support for Hurd as a foreign OS. The Linux-specific parts would have to be moved into "hook" shell functions, and each system would provide a shell script snippet implementing them. debian/rules could then choose the relevant snippet at build time.

Multilingual console

Mach has a very basic console which could be used in the beginning.

Among other interesting features, the Hurd console has some UTF-8 support and its VGA driver can use any 512 glyphs at once in a dynamic fashion. So while I don't expect it to handle bidi or large glyphs, I believe it could be used for a fair number of languages nontheless.

The Hurd console uses BDF fonts, so a udeb similar to bterm-unifont would have to be created. Also, the keyboard driver does not support any kind of keymaps so this would need to be implemented, and integrated with kdb-chooser and /etc/default/keyboard-configuration.

Installation media

Right now, only the "netboot" and "monolithic" image types have some hurd-specific configuration, and only "monolithic" can be built (due to missing udeb deps on "netboot", IIRC). Ultimately, more image types should be supported, but this would probably require some shuffling of the pkg-lists to separate some system-specific parts.

Graphical installer

Though it's somewhat sluggish, Xorg runs on Hurd, and porting the graphical installer would be great for languages not supported by the Hurd console.

... and many other cool surprises!

I expect to be discovering much more work as I progress. But if we could know everything in advance, there would be no fun!

Project schedule

With this in mind, here is my attempt at a schedule. I'm confident that I would be able to achieve the first half of it even if an unexpected number of unexpected problems show up...

Timeframe

Stuff being done

Evaluation period

fix the bootstrap bug, maybe port init in busybox

Community bonding period

read some docs, try to get the installer starting

At this point I would have tried to get through a complete installation with manual tweaks and see what breaks

Week 1 (2010-05-24)

warm up: polish what we have (busybox, genext2fs), setup xen

Week 2 (2010-05-31)

fix the easy stuff (install-time netcfg, cdrom detection, whatever)

Week 3 (2010-06-07)

Mach initrd, Hurd tweaks

Week 4 (2010-06-14)

partitionning, bootloader

Week 5 (2010-06-21)

unexpected stuff, cleanup

Week 6 (2010-06-28)

call for testing, fix the bugs

At this point we would have a basic but working installer. Hopefully the summer would not be over yet...

Week 7 (2010-07-05)

Hurd console

Week 8 (2010-07-12)

more image types, better DHCP support

Week 9 (2010-07-19)

give a shot at the graphical installer

Week 10 (2010-07-26)

unexpected stuff, cleanup

Week 11 (2010-08-02)

shake the thing, call for testing, fix the bugs

Week 12 (2010-08-09)

fix the bugs, write documentation, fix the bugs

Travel

I expect to be able (and would be more than willing) to travel to Debconf 10. However I'm on a tight budget at the moment so I would need sponsoring for the travel and housing costs.

Other plans

I don't have any other other plans for this summer and I expect to be able to work on this project full-time for the complete duration of the GSoC program. My exams should be finished by May 21, so I don't expect them to be a problem either.

After GSoC

Obviously, even if the project is a success, lots of work would remain on Debian GNU/Hurd. I'm quite fond of Debian, both as an operating system and as an organization, and I have had other Debian-related projects (see my outdated website), so I hope that a participation to GSoC would be the foot in the door of my continued involvement :-)