Embedding Debian has various problems, some of which are solved in different ways by stuff like Emdebian, flashybrid, and the ADS root builder. This is an attempt at one solution that should work for most use cases, while also being a lot easier to set up, and more flexible than any of these.
Joey posted some example use cases in his blog: <http://kitenet.net/~joey/blog/entry/embedding_debian.html>
Like flashybrid, it assumes a system that has some limited flash memory, and a large removable disk. Like the ADS root builder, it tries to only put the minimum files needed to run the embedded system on flash, and it determines this based on the system being exersised to catch all needed files.
The idea is to have a daemon that adds files to a jffs2 (or other filesystem in flash) on the fly as they are accessed from the disk.
To start, the system would boot as follows:
- Mount jffs2 to /media/flash.
- Mount a usb drive (or nfs, etc) as /media/disk.
- Use unionfs to make / be the union of /media/flash and /media/disk. Files in /media/flash take precidence; files not available on it will come from /media/disk.
- Run rootsync, in the background.
- Finally, exec /sbin/init.
rootsync would do the following:
Use inotify to watch for all file access in the combined root filesystem. A good method for doing this is described here: <http://mail.gnome.org/archives/dashboard-hackers/2004-October/msg00022.html>
- Each time a file is read, check to see if it's available in /media/flash. If not, copy it from disk to flash, and log this. The logs will probably turn out to be very important for many reasons.
- Each time a file in /media/disk is modified, copy it to the flash.
- Let the user throw a switch to turn off/on copying of files to the flash. That could be useful if they want to apt-get some peice of software, or compile something.
- Provide a way to exclude certian files or directories that do not belong on the flash from being copied.
So a system booted this way could start with an empty jffs2, and as it boots, the jffs2 would be populated with the files needed to boot. Then you run your apps and they are copied to the jffs2. You can use df and look in /media/jffs2 to see what files are on the jffs2. If you accidentially add a file you don't need, just delete it from /media/flash.
After the system is booted and exersised and the jffs2 tweaked for busybox etc, it only needs to be shut down and then the jffs2 should be bootable standalone.
It would also be possible to support apt-get upgrades, which upgrade both the disk *and* the flash simulantaneously. This would be a special mode in which /media/flash is configured to be read-only for unionfs, and rootsync watches for file/directory removals/modifies, and makes sure that the flash and disk are kept in sync, but does not add any newly accessed files to the flash. So when apt downloads a .deb, that file is written only to /media/disk; when the deb is unpacked and changes /bin/ls, which is present in /media/flash already, that file is changed both on /media/flash and on /media/disk.
- Is unionfs too buggy? aufs seems to be used more and more.
- Problems with inode numbers changing? This could affect postfix, but does not seem to affect start-stop-daemon, since inode numbers will only change when the unionfs is remounted.
- Will unix sockets and named pipes keep working when /media/disk is unmounted? Ie, can screen sessions be reattached? Maybe a hack with /var will be needed to make these be created on the flash partition.
- What exactly needs to be excluded by default to prevent /media/flash from gathering runtime cruft. Log files?
- How to deal with symlinks? inotify cannot be used to differentiate between accesses to symlinks and accesses to the files they point to. So when a program is run and /lib/ld-linux.so.2 is accessed, inotify generates an event for the access of /lib/ld-2.3.6.so. To get the symlinks added too, some special appoach will be needed. One might be to keep copies of all symlinks in flash and create any missing symlinks there during the startup scan. However, this is problimatic since it might add lots of extraneous symlinks to flash.
The plan is to investigate whether this will work by building a prototype that consists of an "flash" directory, and a directory containing a debootstrapped system. Union these together, and use inotify to detect file accesses, copying the accessed files over to the "flash" directory. Then the debootstrapped directory can be removed from the union to see how well the "flash" directory works. Playing around with this prototype will hopefully turn up any big problems.
An early version of this prototype is now available, for now it's in git at http://git.kitenet.net/?p=joey/src.git;a=blob;f=misc/unreleased/rootsync/rootsync
Joey Hess <firstname.lastname@example.org> is planning to work on this.