Emdebian Developer Guide

This is intended to provide tips and status information on the emdebianisation of Debian packages, problems encountered, solutions found and details of problematic packages. In due course, the advice here will be distilled into a dedicated Developer Guide and into the currently unformed Emdebian Policy documentation.

Anything that expands or clarifies issues in the emdebian-tools manpages can also be added here.

See also EmdebianQuickStart and Embedded_Debian.

Test repositories

When fixing bugs in the cross-built packages (maybe due to differences between Debian and busybox behaviour), it is useful to have a local test repository and add it to the apt sources for your test device. reprepro can generate a suitable repository using this configuration file conf/distributions:

Origin: Debian
Label: Emdebian-unstable
Suite: unstable
Codename: unstable
Version: 0.1
Architectures: i386 uclibc-i386 arm uclibc-arm armel uclibc-armel powerpc uclibc-powerpc source
Components: main
UDebComponents: main
Description: Emdebian unstable package repository

Adapt the list of architectures to suit. Then add a virtual host to apache:

<VirtualHost *:80>
        DocumentRoot    "/path/"
        ServerName      "my-mirror"
</VirtualHost>

The path is to the directory that contains the conf/ directory of the repository, i.e. adding conf/distributions to the declared path gives a valid path to the file. Use this apt source:

deb http://my-mirror/ unstable main

When adding packages to the mirror, only increment the emdebian version string emdebuild -n such that if the package is uploaded to the Emdebian repository that it is the next logical version. To replace packages in your local test repository, use:

$ reprepro -b /path/ remove unstable $package
$ reprepro -b /path/ include unstable /path/to/$package_foo.changes

i.e. remove the existing binary package (repeat for multiple binary packages from a single source package) and then include the rebuilt package by passing the path to the .changes file. Alternatively, use:

$ reprepro -b /path/ includedeb unstable /path/to/$package_foo.deb

to update only a single binary. (This will leave your repository in an incomplete state as the source package will be out of date compared to the binaries - wrong details in the .dsc and the .diff.gz - whether this matters or not depends on what you expect from a local test repository.)

Mass bug filing

Emdebian patches need to be incorporated into Debian but this process needs to be done carefully. The emdebian patch files are not suitable for inclusion into Debian directly and there is no explicit support for creating Debian patches in emdebian-tools because it is difficult to automate. The process is roughly:

  1. Copy the Debian source into ../branches/ and unpack it with dpkg-source -x

  2. Move the unpacked source directory to $PACKAGE.old. e.g. mv apt-0.7.9 apt.old

  3. Unpack the source again - you now have a safe directory to use in preparation of the patches, $PACKAGE.old
  4. View the emdebian-rules.patch from the emdebian build and migrate those changes into the debian/rules file for a Debian build. It can be helpful to use meld here:

meld ../trunk/$PACKAGE-$VERSION/debian/rules $PACKAGE-$VERSION/debian/rules

Meld supports migrating parts of the patch into the branch rules file (in this example) just by clicking arrows. This allows you to see which patch fragments should be sent upstream.

  1. Skip over patch lines where rules like dh_installman is dropped - leave those calls in the Debian build

  2. Ensure other changes are wrapped in conditionals so that the effects are only activated during a cross-build
  3. If there are changes necessary to the Debian process (e.g. if ./configure does not support --build and --host and it had to be regenerated), ensure that any new dependencies or other changes are carried through to debian/control and that the changes to the Debian build configuration are usable and actually work when not cross-compiling.
  1. Run dch -n to add a changelog entry so that you can compare any .diff.gz and .dsc.

  2. Start the build using normal Debian build scripts - usually debuild.

  3. Compare the files you had to change with diff -u and prepare patches

    1. pushd ../
      diff -u $PACKAGE.old/debian/rules $PACKAGE-$VERSION/debian/rules > rules.diff
      popd
  4. File a bug (or add to an existing bug) using the 'crossbuilt' usertag and 'codehelp@debian.org' user.

    1. bts user codehelp@debian.org , usertag NUMBER + crossbuilt

Cache Files

Cache value handling is being revamped

Expect changes in how cache files and cached values are handled - various changes are being planned within dpkg-cross and testing continues. The following advice applies for packages that have not been cross-built before. Contact debian-embedded@lists.debian.org for info on the updated methods.

Packages consisting of compiled C code and using the autotools can support a cache file of values to help the ./configure script to identify settings for the target that differ from the architecture performing the build. Cache files can also be used to provide the answer when letting ./configure calculate the result would end in failure. Typically, this is due to ./configure trying to compile a test program on the build architecture using the target compiler - resulting in trying to run an arm binary on, say, amd64. To prevent such disasters, the value that the test program is trying to calculate can be provided in a cache file and autotools will skip the test program.

Currently, values for these cache files need to be elucidated the hard way: Let the build fail, work out where it failed, find the cache variable that would prevent the failing code from being run and then look up the value for that variable in the build logs of the native build in Debian.

./configure --cache-file=$(DEB_HOST_GNU_TYPE).cache

Identifying the actual values for the cache file can seem a bit of a black art. Always work from the ./configure script, not configure.in or configure.ac. Load the script in your favourite editor (don't save it or modify it) and search for whatever term was specified in the failure message. When you find the string that matches the error report, look for a line like:

if test "${ac_cv_func_regcomp+set}" = set; then

The value you need to set, in this example, would be ac_cv_func_regcomp. To find the actual value to pass (generally yes or no), view the native build log in Debian. (Go to the PTS page for the source package, scroll down and click on Build Logs, click on your host architecture and search for the same string that you looked for in ./configure). In this case, the arm native build for krb5 answers yes to this particular check, so the arm-linux-gnu.cache file includes a line:

ac_cv_func_regcomp=yes

Sometimes, the value is preprocessed or read from a list and various other methods that make the actual value hard to identify. In such cases, copy the entire block into a separate editor window and run it as a shell file - predefining whatever values may be necessary (e.g. cross_compiling=yes). This is a sample test script using a sample from the krb5 configure script:

cross_compiling=yes
ac_cv_file__etc_environment=no
ac_cv_file__etc_TIMEZONE=no

for ac_file in /etc/environment /etc/TIMEZONE
do
ac_safe=`echo "$ac_file" | sed 'y%./+-%__p_%'`
{ echo "$as_me:$LINENO: checking for $ac_file" >&1
echo $ECHO_N "checking for $ac_file... $ECHO_C" >&1; }
if { as_var=ac_cv_file_$ac_safe; eval "test \"\${$as_var+set}\" = set"; }; then
  echo $ECHO_N "(cached) $ECHO_C" >&1
else
  if test "$cross_compiling" = yes; then
    { { echo "$as_me:$LINENO: error: Cannot check for $as_var file existence when cross compiling" >&1
echo "$as_me: error: Cannot check for file existence when cross compiling" >&2;}
   { (exit 1); exit 1; }; }
else
  if test -r $ac_file; then
    eval "ac_cv_file_$ac_safe=yes"
  else
    eval "ac_cv_file_$ac_safe=no"
  fi
fi
fi
if eval "test \"`echo '$ac_cv_file_'$ac_safe`\" = yes"; then
  { echo "$as_me:$LINENO: result: yes" >&1
echo "${ECHO_T}yes" >&1; }
    ac_tr_file=HAVE`echo $ac_file | sed 'y%abcdefghijklmnopqrstuvwxyz./-%ABCDEFGHIJKLMNOPQRSTUVWXYZ___%'`
  cat >>confdefs.h <<_ACEOF
#define $ac_tr_file 1
_ACEOF

else
  { echo "$as_me:$LINENO: result: no" >&1
echo "${ECHO_T}no" >&1; }
fi
done

This script was used to determine why ac_cv_file_etc_environment=no was not working in the cache file. This line:

    { { echo "$as_me:$LINENO: error: Cannot check for file existence when cross compiling" >&1

was modified to:

    { { echo "$as_me:$LINENO: error: Cannot check for $as_var file existence when cross compiling" >&1

so that when the script failed, the actual value was reported, in this case, requiring an additional underscore character.

(./configure uses various redirects of STDOUT and STDERR so you also need to modify >&5 and >&6 to >&1 etc.

emdebuild protects the cache file so that the entered values are retained for the patch.

emlocale replaced by eminstalltdeb

Packages are being updated to remove the old emlocale patches - ensure you remove individual POFILES handling, previously recommended here, from debian/rules and the list of locale packages from debian/control. eminstalltdeb now handles these automatically and is run by emdebuild.

You may still need to remove the old emlocale files:

rm -rf debian/diffutils-locale-*

Adding cross-build detection

If debian/rules uses autotools yet does not use ---build and --host, the build process will probably fail to detect the cross-compiler. This is usually a bug in the upstream package.

Caution

Be careful not to add extra spaces to the end of the variable assignment when copying from this page because 'make' will include that in the equality test. 'arm-linux-gnu ' is not identical to 'arm-linux-gnu' and 'DEB_BUILD_GNU_TYPE=$(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE)' is not identical to 'DEB_BUILD_GNU_TYPE=$(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) ' - use an editor that shows the whitespace.

To fix, add:

DEB_HOST_GNU_TYPE=$(shell dpkg-architecture -qDEB_HOST_GNU_TYPE)
DEB_BUILD_GNU_TYPE=$(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE)

then add an extra pair of options to either confflags or ./configure

ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE))
CROSS= --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE)
else
CROSS= --build $(DEB_BUILD_GNU_TYPE)
endif

./configure $(CROSS) ...

or for packages that only use make but not configure:

ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE))
CROSS=CC=$(DEB_HOST_GNU_TYPE)-gcc
else
CROSS=
endif

Note that, for arm, the value of $(CROSS) is CC=arm-linux-gnu-gcc. There is no value $(CC) set in the above line, despite how it may appear from a C or perl perspective.

$(MAKE) $(CROSS) ...

CDBS packages

If CDBS results in binaries of the wrong architecture, check the CFLAGS being passed by default (a bug has been filed but is not yet fixed in CDBS). Remove the settings by adding to debian/rules:

DEB_CONFIGURE_SCRIPT_ENV=

DEB_BUILD_OPTIONS

Syntax:

ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))
        $(MAKE) check
endif

ifeq (,$(findstring nodocs,$(DEB_BUILD_OPTIONS)))
        install foo.1 debian/foo/usr/share/man/man1/
        ...
endif

empty packages

Section removed because the current scripts use 'emgrip' to remove documentation files, rather than patching the source.

Running preprocessors in the cross build

Some builds (avahi, gconf) need to preprocess some code using build machine binaries outside the normal toolchain. In many cases, the package will attempt to locate these binaries using pkg-config and will therefore locate binaries for the host machine which will not run on the build machine.

To ensure the right binaries are used, locate the variable used by $(MAKE) to hold the path to the binary and override that variable in the command to make. In CDBS, use DEB_MAKE_INVOKE+=BIN_VAR=/usr/bin/foo. In debhelper/dpkg, add the command before the call to make. Make should then notice that the processed files are up to date and not attempt to build them again with the host binaries.

i386 builds on amd64

sudo debootstrap --arch i386 sid /opt/emdebian/i386 http://ftp.uk.debian.org/debian/

Remember, you must set a deb-src in /etc/apt/sources.list inside that chroot, then install the other packages:

$ sudo chroot /opt/emdebian/i386/
# apt-get update
# apt-get install emdebian-tools
# apt-get update
# apt-get dist-upgrade

The extra update/upgrade cycle is because installing emdebian-tools adds the Emdebian repository which often contains updates for emdebian-tools and related packages.

Problematic packages

  1. perl-base Tries to run cross-built miniperlmain and possibly other generated binaries during make.

  2. openssl - ar is not passed the correct options when cross-building, despite the r tag being specified in the (perl based) Makefile.org. This halts the build when arm-linux-gnu-ar fails to get the r option that is used in a native build.

  3. libcap - compiles an internal program to generate a header file then tries to execute that file.

  4. apititude - needs to build in a chroot where the apt based dependencies can be handled correctly.

  5. gcc-??? - too complex for its own good. Frequent breakages in gcc are all the more problematic when Emdebian only needs two of the dozens of packages built from the gcc source: libgcc1 and libstdc++6. Mostly fixed now.

  6. coreutils - recent versions have started using 'help2man' to generate the manpages but help2man tries to execute the compiled binary to use the error output in the manpage. Hence this fails completely in a cross-build. A patch is used in Emdebian to remove 'man' from the list of subdirectories in the Makefile.am (actually the patch removes doc and test too) but if anyone wants a cross-built coreutils with manpages, some form of double-build is going to be needed or persuading upstream that help2man really isn't a good idea.

dh_shlibdeps errors

dh_shlibdeps
dpkg: /usr/arm-linux-gnu/lib/libacl.so.1 not found.
dpkg-shlibdeps: failure: dpkg --search gave error exit status 1
dh_shlibdeps: command returned error code 256

Check for the existence of the file within the build:

$ find . -name libacl.so.1
./debian/libacl1/lib/libacl.so.1
./libacl/.libs/libacl.so.1

and re-run dh_shlibdeps with the -l option:

$ dh_shlibdeps -l ./debian/libacl1/lib/
dpkg-shlibdeps.orig: warning: could not find any packages for libattr.so.1

Note that this example comes from libacl1 but the reference is to libattr - this gives you a clue that you need to use apt-cross - this is an Build-Cross-Depends package.

$ sudo apt-cross -i libattr1

Then change the dh_shlibdeps rule in debian/rules to specify to look for the dependency information within the current build tree using -L and -l:

dh_shlibdeps -L libacl1 -l ./debian/libacl1/lib/


CategoryEmdebian