Bootstrappable Debian - Student Proposal


alkmim@ic.unicamp.br

alkmim on irc.debian.org #debian-bootstrap




The objective of this project is to enable the automated bootstrap of new Debian ports. We can summarize all the project in two items:

  1. Make a tool to manage bootstrap process (including instructions of how to use it) using the Botch toolchain
  2. Write patches and file bugs filed for all the multiarch, cross-build and profile patches needed for the bootstrap process.

As described in the project description from the Debian ideas list [5], this is a very big problem and possible all of this can't be accomplished in just one GSoC.


As we can see in [6] the Automatic Bootstrap is one of the going projects of Debian. As soon as this project is finished all the bootstrap process is gonna be easier even for non-experts in building Debian packages. Also, if we can do it in an automatic way, the effort to do this will be reduced. If all patches be accepted by the upstream maintainers, we will avoid solving cyclic dependencies or reduced dependencies problems every time a port is done.


At the end of the Google Summer of Code, the following results should be obtained:


Project details

Small introduction

Every time a new architecture is developed, the Debian Linux needs to be ported to be used on it. Therefore, porting any operational system to a new architecture is always a difficult task since the bootstrap needs to be done 'from scratch'. While doing the first bootstrap of an architecture some challenges need to be faced. In order to do the bootstrap to a new architecture we need to have packages built that can run in the new architecture. This can be done by using multiarch or cross-built packages. Therefor we don't need to have all packages of system crossbuilt or multiarch. We just need to have a minimal build system that is sufficient to satisfy the build-dependencies of the rest of the packages.

Meanwhile, this raises another challenge that is the cyclic build-dependencies. In order to build/crossbuild packages that have cyclic build-dependencies we need to break the cycle(s) avoiding one or more build-dependencies. This is also a very difficult task because patches need to be applied. Another problem that raises when doing the bootstrap of a new architecture is while building packages inside the minimal build system. Despite all the effort in trying to crossbuild and have multiarch versions of all packages needed to solve the build-dependencies of the rest of the packages, some of them are very hard to be crossbuilt. So, we need to decide whether we should build the packages inside the minimal system or crossbuild them.

Ojectives

The Bootstrappable Debian project has the objective to enable the bootstrap of new Debian ports. As explained previously it is a very difficult project since we still have a lot of problems to solve. A very good explanation about the Debian bootstrap can be found in [7] and a detailed TODO list can be found in [8].

  1. Build a tool that enables an automated bootstrap of new Debian ports. The main functionality of this tools is to build the packages for a Debian OS for a specified architecture from scratch. The exact parameters that the tool should manage are not defined yet, but this will be discussed during the project. One possibility is that the user should specify the set of packages that should be included in the new environment. To accomplish that, the tool should use the Botch toolchain to determine which packages are available for the minimal build system (that need to be crossbuilt or are multiarch), which packages needs to be built inside the new environment natively and the build order of those packages.

  2. Patch all packages needed to enable the crossbuild process to be done in practice. This is a hard step of the project because to enable the crossbuild part of the process we need to modify some packages sources in Debian. This needs to be done to solve cyclic dependencies and file bugs in some packages related with the crossbuild.

  3. Patch all packages that are going to be built inside the minimal build system. As soon as we got a minimal build system we can start building the packages of the new port natively. While building packages inside the minimal build system, we might lack some build-dependencies that have not being crossbuilt (e.g. they are very hard to crossbuild). So we need to build those packages avoiding the build-dependencies we don't have. This might be accomplished using the new Build Profile standard that enable the addition of information about staged builds.

The Build Profiles standard is the way found to insert sufficient information on debian sources packages to enable the build with the reduced build-dependencies. Through the environment variable DEB_BUILD_OPTIONS we can specify which profile to use while building a package. Build profiles extend the Build-Depends format with a syntax similar to architecture restrictions but using < and > instead. More about the Build Profiles can be found in [10].

In order to test the crossbuilt Debian system, we plain to use QEMU to emulate an armhf architecture to test if the bootstrap process for this architecture has been done successfully. After crossbuild and build all packages for the armhf architecture, we can create a repository using the built packages and use programs like debootstrap to create a new chroot to test the bootstrapped armhf architecture.

During this 2 weeks working on the project proposal to submit for the GSoC 2013, I finished some of the first patches we need to build packages inside the minimal build system with reduced build-dependencies. The patches may be found in ?[4]. Also, I started working on patches to build the Ubuntu crossbuild toolchain for the armhf architecture in Debian. These patches are not ready yet.






* Are you applying for other projects in SoC? Yes. I'm also applying for the ?RouteFlow project but i'm giving a lot more effort to Debian project them to the other since I prefer to work on the Debian project.


* References [1] http://www.tecnolivre.com.br/ [2] http://www.lrc.ic.unicamp.br/~alkmim/gsoc2011/ [3]|http://www.lrc.ic.unicamp.br/~alkmim/debian/multiarch/ [4] http://www.lrc.ic.unicamp.br/~alkmim/debian/GSoC2013/bootstrap/patchs-alkmim/ [5] http://wiki.debian.org/SummerOfCode2013/Projects#Bootstrappable_Debian [6] http://wiki.debian.org/DebianBootstrap#Automated_Bootstrapping [7] http://wiki.debian.org/DebianBootstrap [8] http://wiki.debian.org/DebianBootstrap/TODO [9] https://gitorious.org/debian-bootstrap/botch [10] http://wiki.debian.org/DebianBootstrap#Mechanisms