GNU/kFreeBSD explained
Steven Chamberlain <steven@pyro.eu.org>
DebConf15, Heidelberg
Overview
- A sub-project within Debian, using only packages from the official archive,
- but notably not using the Linux kernel:
- not because we don't like it,
- and not because of the license (we still bundle GNU software, and use the GNU libc).
- but notably not using the Linux kernel:
Overview
A complete operating system (one does not simply apt-get install kfreebsd).
- From FreeBSD we take the kernel (including hardware drivers),
- non-free blobs removed to comply with DFSG,
- and some BSD utilities as needed, to boot and administer it.
Some history
- 1999 - debian-bsd@ mailing list created
- FreeBSD users envied Debian's packaging system,
- and Debian's neatly organised file structure (FHS, /etc/init.d),
- but still wanted to use the FreeBSD kernel, at least.
- Early interest in collaboration on packaging.
- 2001 - minimal Debian chroot bootstrapped on NetBSD, using BSD libc (Matthew Garrett)
- FTP space, and bandwidth (lol, dialup) were major issues at the time.
Some history
- 2002 - idea to use GNU libc instead, like GNU/Hurd, making packages easier to port
- First successful bootstrap (Nathan Hawkins) was lost in a fire.
- 2003-05-06 - self-hosting GNU/kFreeBSD chroot (i386) released on people.d.o (Robert Millan)
- 2006-03 - amd64 port
Some history
- 2009-04-05 - accepted into unstable
- 2011 - released with squeeze (a "technology preview")
- having 85% of Debian source packages.
- 2013 - released with wheezy:
- having 89% of Debian source packages,
- full support for ZFS, also in the installer and in GRUB2,
- BSD jails working,
- PF firewall,
- Apache, MySQL, PHP, Ruby, Python, Java etc.
- Possible to install a Debian chroot on FreeBSD,
- and vice-versa: possible to run FreeBSD in a chroot on Debian.
ZFS backup strategy
- Periodically rsync your files to a ZFS fileserver;
- the data transfers are incremental, saving bandwidth,
- you may --delete any that no longer exist at the source.
- Create a snapshot immediately after.
- You can delete older snapshots arbitrarily:
- later snapshots remain intact, because if the deleted snapshot had newly-added files, those are folded into the next snapshot by date;
- possible to use a logarithmic algorithm, you can prefer keep recent snapshots spaced more closely together, and a few older snapshots with longer time intervals between.
- zfsutils provides an implementation of this, easily enabled from /etc/default/zfs
ZFS backup strategy
NAME USED - REFER - tank/srv/mail@autosnap-2014-11-10 19.2G - 154G - tank/srv/mail@autosnap-2015-03-18 8.33G - 185G - tank/srv/mail@autosnap-2015-05-21 6.69G - 203G - tank/srv/mail@autosnap-2015-06-22 6.37G - 210G - tank/srv/mail@autosnap-2015-07-24 3.32G - 225G - tank/srv/mail@autosnap-2015-08-01 2.70G - 227G - tank/srv/mail@autosnap-2015-08-05 2.44G - 228G - tank/srv/mail@autosnap-2015-08-07 1.58G - 229G - tank/srv/mail@autosnap-2015-08-09 445M - 229G - tank/srv/mail@autosnap-2015-08-10 145M - 229G -
In production
- Stable: long continuous uptimes (e.g. 400 days)
Performance: http://www.phoronix.com/scan.php?page=article&item=debian_kfreebsd_jess&num=3
- Seen powering git.gnupg.org for a time;
has been spotted in Tokyo (Spring 2013 OSC): http://henrich-on-debian.blogspot.com/2013/02/open-source-conference-2013-tokyospring.html
For jessie
- 2015 - kfreebsd was rejected as an official Debian release architecture
Releasing jessie-kfreebsd
- jessie-kfreebsd release candidate is being built right now!
- Keep an eye out for the coming announcement.
New in jessie-kfreebsd
- Kernel of FreeBSD 10.0:
- Enhancements from OpenZFS project: LZ4 compression, improved performance,
- KMS graphics for Intel, AMD and nVidia cards: accelerated 3D graphics and video,
- a new framebuffer console, enabling more character sets,
- firmware loading: can use existing firmware-linux-(non)free packages.
- More than 90% of Debian source packages (although no GNOME3 desktop any more).
- Many usability issues fixed.
The GNU/kFreeBSD desktop
- disk encryption is possible (GELI)
- ZFS filesystem:
- compression, checksumming, redundancy (RAID, or copies=2), snapshots (every hour?).
- Choice of desktops (XFCE, LXDE, KDE, MATE):
- iceweasel browser (no Chromium), libreoffice suite, OpenJDK 7.
- GLX rendering, HD video playback,
- ACPI S3 suspend/resume (depending on hardware, YMMV).
How jessie-kfreebsd works
- New jessie-kfreebsd suite - a snapshot of testing packages from jessie release day.
- New kfreebsd-jessie-proposed-updates queue, to track jessie p-u,
- plus kfreebsd-specific fixes at our discretion: +kbsd8u1.
- jessie-kfreebsd/updates on security.d.o - security updates triggered by dak.
- Stable release with security updates - so that it can run on DSA-administered buildds.
How jessie-kfreebsd works
- Example /etc/apt/sources.list:
deb http://httpredir.debian.org/debian jessie-kfreebsd main deb-src http://httpredir.debian.org/debian jessie-kfreebsd main deb http://security.debian.org/ jessie-kfreebsd/updates main deb-src http://security.debian.org/ jessie-kfreebsd/updates main
- Many thanks to FTP Master and Security teams for setting this up, and use of their infrastructure.
How the userland is structured
- kfreebsd-10 source package, built from upstream SVN /sys/, with non-free stuff removed.
- kfreebsd-kernel-headers built from that. Preferably only used by:
- FreeBSD-specific utilities, e.g. mount, ifconfig, netstat
- FreeBSD userland libraries, e.g. libcam, libkvm
- cross-platform libraries, e.g. libusb
- GNU libc and compiler
- When the kernel changes, only the above packages should be affected by a transition.
- The libraries abstract away kernel-specific details.
Patterns to avoid
- Trying to special-case for every existing operating system:
#define PLATFORM gnukfreebsd
- OR:
#ifdef linux ... #elif __FreeBSD__ ... #elif __DragonFly__ ... #elif __GNU_kFreeBSD__ /* Please don't do this! */ ... #endif
Slightly better
- Distinguish between kernel or userland:
#ifdef __linux__ ... #else ... #endif
- OR:
#ifdef __GLIBC__ ... #else ... #endif
Ideally
- Test for specific features you need (e.g. autoconf macros) so that it works for any OS that (later) implements it:
#ifdef HAVE_DTRACE ... #endif
Worst-case scenario
An example of what isn't portable: https://sources.debian.net/src/util-linux/2.26.2-9/sys-utils/eject.c/
- Uses many Linux-specific kernel interfaces:
#include <linux/cdrom.h> #include <linux/fd.h> #include <sys/mount.h> #include <scsi/scsi.h> #include <scsi/sg.h> #include <scsi/scsi_ioctl.h>
- The tool also suffered some feature-creep:
Package porting example
- Some software builds and works right away!
- Make sure debian/control Architecture field does include kfreebsd:
- amd64 means linux-any; say any-amd64 to build on other kernels
- failure to build on kfreebsd or hurd is fine - it shouldn't block testing migrations
- Most software supports GNU/Linux and FreeBSD; pick and choose different parts
- Usually there are many ways to solve a problem
My own objectives
- Support old hardware for as long as possible:
- floppy drives, tape drives, ISA network cards - whatever our users might have;
- hardware can be useful, for as long as we provide a stable OS for it;
- be an alternative for any situation where Linux has issues, and to help in debugging those issues.
My own objectives
- Allow old software and installed systems to exist for as long as possible:
- rewriting application software costs time and effort
- FreeBSD users often follow a stable branch for many years;
- Debian LTS was a really good idea.
- Would like to ease migration of FreeBSD systems to Debian, or vice-versa.
- Retain familiarity, expertise:
- it's annoying to need to be required to learn new systems;
- developing for POSIX, how to administer UNIX systems, and System V init, are well understood by many people; and whole shelves of books explain them.
My own opinions
- Ideal applications should work on any kernel
- The GNU/kFreeBSD project seems like useful package QA
- Microkernels like GNU/Hurd look promising for the future; GNU/kFreeBSD porting work may help that.
- Avoiding kernel internals makes transitions easier, and big kernel changes easier too
Diversity in free software is good
- Opportunities for more collaboration between Debian and BSDs
- Mixing OSes can give extra redundancy for a high-availability cluster?
- Adds extra security to the Tor network for example
- Reproducible builds? Bootloader and kernel packages should have identical binaries regardless of the kernel of the system that built it
What's next?
- We need your feedback, to find and fix usability issues (including installer)
- Configure for a better out-of-box experience (sane defaults)
- Plan for the next release:
- should we seek architecture requalification?
- or make a separate suite for a rolling, or testing release? (squeeze-kfreebsd)