Wiki

• FrontPage
• RecentChanges
• FindPage
• HelpContents
• JustusWinter

Student Application for Debian GNU/Hurd Debianish initialization

• Name: Justus Winter

• Contact/Email: 4winter@informatik.uni-hamburg.de, teythoon@freenode

• Background: I've used both Debian/Linux and Debian/Hurd systems for quite some time and I'm reasonably familiar with low level aspects of the system initialization on both systems. Back in the day when the d-i was not yet ported to Debian/Hurd, I created a live cd with a custom installer in the spirit of the OpenBSD installer ¹ . I consider myself fluent in the languages required for this task (english, c, bourne shell).

• Project title: Debian GNU/Hurd Debianish initialization

• Project details: The Debian GNU/Hurd port currently uses its own initialization scripts, which are neither really good, nor properly integrated with the rest of Debian. The goal of this project is to make this port use the default Debian init system (currently: sysvinit and its rc, sysv-rc), fixing bugs in it and in the associated init scripts, and then simply enable them instead of the GNU/Hurd upstream ones. This will probably also include integrating proper network setup, instead of the current static configuration of the pfinet translator.

• Synopsis: Use the default Debian init system to boot Debian/Hurd.

• Benefits to Debian: Supporting several kernels in Debian is a worthy goal. It helps to discover assumptions about one kernel that are hardcoded into the initialization scripts (sometimes called linuxisms). Being able to boot a Hurd systems using the default mechanisms provided by Debian also seems like a necessary precondition for a proper Debian/Hurd release. Another benefit for Debian is another pair of eyes going over Debians early initialization scripts, and a potential Debian Developer that is familiar with this aspect of Debian. There are quite a number of bugs filed for the initscripts package, looks like they could use a helping hand. [There is e.g. bug 693398 that deals with the *-bootclean scripts. biebl@ argues that those might be totally pointless for Linux and kFreeBSD since in those cases all the directories are on newly mounted ramdisks and that this is only of value for Hurd. But currently Debian/Hurd systems do not use these scripts at all. So me going over these scripts might resolve these kinds of bugs, resulting in less cruft and a simpler (and faster) startup for GNU/Linux & GNU/kFreeBSD.]

• Deliverables: Replacing the upstream GNU/Hurd initialization with the mere use of /etc/rcS.d scripts one per one, fixing bugs along. Then just use the standard Debian init/rc system.

• Project schedule:

  • Document what has to be done (and what's currently done) to boot a Debian/Hurd system. Compare this to the Debian/Linux initialization. Try to chart the differences and map concepts when they differ (esp. filesystem & network stuff, see below for a more detailed statement of the problem). Look for unexpected hurdles.

  • Add a Debian/kFreeBSD installation to my OS zoo. The kFreeBSD folks have to deal with supporting a non-Linux kernel using the Debian infrastructure, and looking how they handle this might be enlightening.
  • Try to find a smooth (as in incremental patches) route to preserve whatever needs preserving from the Hurd init system. Try to plug hurdish initialization stuff into the default Debian scripts.
  • Setup and document the test environment. I'm a big fan of automated package building and testing. I'm thinking about exploring a subhurd based test setup, probably also a qemu based one.

  • Hacking and bug fixing

  • Provide a deb repository for testing and evaluation.
  • Aftermath:
    • Provide (patched) packages on debian-ports.org
    • Push the new packages to Debian.

    • If there's a need to tweak any packages, file bugs with patches and lobby for the inclusion of said patches. I suspect this might be actually the hardest part

• Exams and other commitments: I've no exams scheduled in this period.

• Other summer plans: I'm going to go to the European Juggling Convention (2013-07-27 - 2013-08-03). The midterm evaluation is due on 2013-08-2, so I'd have to write up my report before that.

• Why Debian?: Debian is my operating system of choice.

• Why Hurd?: I'm interested in operating system design, especially in microkernel design. It empowers the (unprivileged) user to do cool stuff that she (usually) isn't allowed to do (like mounting file systems, doing chroot like stuff, setting up her own network stack). Even though Linux tries to catch up in this regard (think FUSE or the namespace work, enabling stuff like lxc), microkernels like Mach with the Hurd servers provide this by design. Since my university isn't doing anything in the area of OS research, I like playing around with Hurd in my free time.

• Are you applying for other projects in SoC? No.

Statement of the problem

Use of custom initialization scripts

Currently Debian/Hurd uses custom programs and scripts for early system bootstrapping. The programs and scripts are provided by the hurd package.

Life of a hurd system starts when the root filesystem translator starts /hurd/init. /hurd/init starts the proc and auth server and then executes /etc/hurd/runsystem.

The relevant scripts reside in /etc/hurd:

root@hurdbox ~ # ls -l /etc/hurd
total 12
-rwxr-xr-x 1 root root 2756 Apr  9  2012 rc*
lrwxr-xr-x 1 root root   27 Jan 16  2012 runsystem -> /etc/alternatives/runsystem*
-rwxr-xr-x 1 root root 4141 Dec  8  2011 runsystem.gnu*
root@hurdbox ~ # dpkg --search /etc/hurd
hurd: /etc/hurd

/etc/hurd/runsystem is started by /hurd/init and setups ttys and starts /etc/hurd/rc or falls back to starting an emergency shell.

/etc/hurd/rc is Debian/Hurds equivalent of /etc/init.d/rc, but it is doing a lot of other stuff too, like checking the root filesystem, remounting it rw, activating swap, cleaning up various files, working around various differences between Hurd and Linux. It is also a lot more limited, in fact it just runs all scripts matching /etc/rc{.boot,2.d}/S*.

So while the Hurd bootstrap has to be a bit different than a Linux bootstrap, there is no need for it to be this different. Also the rc script used by Debian/Hurd does not support the sysv style runlevel switching and does not execute K??-* scripts.

Survey of initialization scripts & differences between Hurd and Linux

Here is a list of scripts I found on my most recently installed Hurd box in /etc/rcS.d:

[1] Not part of the startup scripts, uses passive translators for persistence.

[2] Not normally done on hurdish systems, but could/should be done, see discussion about passive translators.

[3] Nowadays Hurd uses translators to access NICs (yay userspace drivers ) typically bound to /dev/ethX. Using a device name like /dev/ethX in /etc/network/interfaces kinda works, but does not play well with ifupdowns state machinery.

Stuff that the current Hurd scripts & programs do that have no equivalent on Linux:

• /hurd/init
  • starts and reparents proc and auth server
  • notifies (file system) translators before the system halts or reboots
• /sbin/runttys
  • Hurd uses a BSD style /etc/ttys file that specifies what gettys to spawn instead of a sysv style inittab. I'd consider this obsolete if sysvinit takes over this job. Might be interesting to see how kFreeBSD handles this.
• /etc/hurd/rc
  • removes all translators from /tmp/*

State of the initialization scripts

(Probably) thanks to the kFreeBSD folks some of the initialization scripts are kernel aware, e.g. checkroot.sh contains:

        KERNEL="$(uname -s)"
        MACHINE="$(uname -m)"
[...]
        case "$KERNEL" in
          Linux)
[...]
          *)
[...]
        esac

So there is a mechanism in place to deal with different kernels. In addition, urandom contains a kFreeBSD specific workaround.

This makes me believe that

• adding kernel specific behavior to these scripts is accepted, though it remains to be seen if it will be accepted for Hurd specific stuff.
• stuff that is too Linux specific is probably already skipped when run on non-Linux to accommodate the FreeBSD kernel. This should help us a lot.

Passive translators

There are some system configuration aspects that are handled quite differently on Hurd, prime examples are mounting of file systems and network configuration.

While on Linux this configuration is volatile (meaning that it is not preserved across system restarts) and has to be set up at boot time, Hurd provides a mechanism to encode system configuration as "passive translators" in filesystem nodes. Translators provide some kind of service (e.g. filesystem or network services) and are started on demand if a passive translator is set on a filesystem node.

For example the network configuration on my hurd box is currently done by means of passive translators. The passive translator can be queried using showtrans, the active translator can be inspected using fsysopts. Translators are just programs running in user space, my pfinet translator (Hurds network stack) is running as pid 100:

root@hurdbox /etc/hurd # showtrans /servers/socket/2
/hurd/pfinet --interface=/dev/eth0 --address=192.168.122.2 --netmask=255.255.255.0 --gateway=192.168.122.1
root@hurdbox /etc/hurd # fsysopts /servers/socket/2
/hurd/pfinet --interface=/dev/eth0 --address=192.168.122.2 --netmask=255.255.255.0 --gateway=192.168.122.1
root@hurdbox /etc/hurd # ps --pid 100
  PID TT STAT     TIME COMMAND
  100  - S<o  27:40.58 /hurd/pfinet --interface=/dev/eth0 --address=192.168.122.2 --netmask=255.255.255.0 --gateway=192.168.122.1

It is an open (design) question how to deal with this discrepancy between Hurd and Linux. As passive translators have their share of problems as well, it might be best not to use passive translators but to just start active translators at boot time as specified by the same configuration files as Debian/Linux uses.

More information on translators can be found at http://www.debian.org/ports/hurd/hurd-doc-translator.