How to split a package into several smaller packages

You want to create multiple binaries packages from one source package.

Please examine the php-java-bridge /debian/* files php-java-bridge source tree, as this source package contains many tricks and hints leveraging debhelper scripts and avoiding pitfalls. That source tree is being constantly updated with improved Debian Policy compliance, new features and tricks. This text was based in version 5.3.2.1.1 of the source tree, containing the init.d script section, deprecated since previous 3.2.x versions, but kept for guidance for your packaging efforts.

Ideas collected from other package sources, discussion lists, Debian Mentors hints and guidance debian-mentors mailing list, Debian Policy, maintainers guide, man pages.

Special attention to /debian/rules and subpackage_name.* files, like php-java-bridge-j2ee.install , php-java-bridge-j2ee.dirs and so on.

The approach used is a fine grained control, specifying each control file and its contents for each individual target.

This allows you to compile each target individually. Usefull when you have big programs that take hours to compile. Or you want to compile each target with different options. Or you have specific individual installation procedures list thread: simplifying the targets for spliting.

Also, see the README.Debian file php-java-bridge source tree, containing many hints and instructions.

Please, note that this source package is a dynamic work in progress, constantly evolving. All suggestions are welcome at the php-java-bridge-users mailing list.

This text concept is explaining from bottom to up abstraction level building the packages.

Readings

Splitting a Debian source package into several smaller binary packages is not trivial.

Before you start, you must read the essential guides

and feeling already comfortable creating single binary packages.

Variables in rules makefile

##!/usr/bin/make -f
# -*- makefile -*-

You must have to remove the first of the two # at first line, left here because the wiki security does not allow the make line.

You MUST remember that /debian/rules is a makefile. It IS NOT a shell batch script.

Read the following code snippets:

        BUILDDIR := debian/build
        DESTDIR := ${CURDIR}/${BUILDDIR}

build-stamp: configure 
        dh_testdir

The BUILDDIR was set to allow use of dh_install for .war files. Also, it was set to outside the binary packages trees that will be generated. Notice that BUILDDIR is relative path and DESTDIR is absolute path and both are pointing to the same actual place. Some of the rules need absolute path arguments, others relative paths.

The Debian package makefile must be fully relocatable.

The build directory must be at a package tool discretion (usually at some source temporary subdirectory) and must not need root privileges.

For the installation, the destination directory MUST NOT be hard coded into the source.

If the upstream source has hardcoded directories, you must apply patches even before the configure target rule.

You must know the different types of makefile variables reading Make manual - variables flavours.

The effects of using a simply expanded variable ( := ) or a recursively expanded variable ( = ) are very different in a rules makefile.

At our example we want the simply expanded variable behaviour.

Shell command output into rules makefile

We want also collect some shell command output into some variables. Make manual - shell function

There are some misconceptions that you should avoid thread: variables from shell output in rules makefile.

        PHP_EXT_DIR := $(shell /usr/bin/php-config --extension-dir)
        PHP_INCLUDE_DIR := $(shell /usr/bin/php-config --include-dir)

Leveraging dpatch

dpatch is a clever development tool. Its use is not limited nor specific to multiple binary packages. You could use it for single binary packages too. This topic could be safely skipped if you do not intend to modify upstream code.

If you must have to apply some modifications to the upstream code before packaging for Debian, then one of the easiest modes is to use dpatch.

When you execute dpatch in a shell prompt, it opens a session and from now on until you exit the session, all you do will be recorded and converted to a patch for your upstream code into /debian/patches directory.

A good practice is to create small patches with clear, self explanatory and different names. For example, 10_relocate_build , 20_relocate_install, 30_correct_env_variable.

For example, supose that the local source tree is located as below and you want to create a patch for MakefileFrag:

cd ~/compilation/php-java-bridge

dpatch-edit-patch patch 10_MakefileFrag_patch

Copy around from a personal template or edit the Makefile.frag into the ./debian directory. Exit dpatch shell. All your modifications were being recorded and will result in the defined patch file name into the ~/compilation/php-java-bridge/debian/patches directory.

You must read the dpatch man page.

The following code snippet will apply the patches to source, walking through all patches into /debian/patches directory in the ascendant patchnamefile order.

And then stamp the configure step.

configure: patchsource configure-stamp

patchsource: patch-stamp

patch-stamp:
        dpatch apply-all
        dpatch cat-all >patch-stamp

configure-stamp:
        dh_testdir
        phpize
        ./configure --with-java=/usr/lib/jvm/java-1.5.0-sun --prefix=${DESTDIR}
        touch configure-stamp

Cleaning and deapplying patches

You must have a clean rule. After analyzing the upsgream source, we realized that a distclean should be used, followed by a rm -rf at the builddir. Also, if you applied patches, you must deapply them in reverse order.

The dpatch helper will take care of this if you created the patches in a planned naming convention.

clean: clean-patched unpatch

clean-patched:
        dh_testdir
        dh_testroot
        rm -f build-stamp configure-stamp
        -$(MAKE) distclean
        dh_clean
        rm -rf ${BUILDDIR}

unpatch:
        dpatch deapply-all
        rm -rf patch-stamp debian/patched

quilt

Quilt patch management tool has been recommended among Debian Developers as a better alternative to dpatch. It is similar in some concepts and commands.

Gquilt GUI for the quilt tool is a graphical user interface for the quilt tool.

Compiling the binaries

There are some reserved target names that you should use with much care.

clean, configure, build, install, binary, binary-arch, binary-indep.

You should remember that build and install are implicit binary targets and you DO NOT need to call them at each target or anywhere. At THIS source example, the upstream code needs 2 cycles of them. So, at THIS example, they are included at the binary target, for the needed second cycle.

The binary target is important and has its own topic. It is there that we coordinate all process, at top level abstraction.

The rule calls the "configure" phony target and then test if it is at the correct directory for the step and if the /debian/control file exists. Finally, touch the build-stamp file.

build: build-stamp

build-stamp: configure 
        dh_testdir
        $(MAKE)
        #docbook-to-man debian/php-java-bridge.sgml > php-java-bridge.1
        touch $@

The phony targets

You must read the Make Manual, about phony targets Make manual - the phony targets.

Here, we intend to use them as like "subroutines". We listed all targets that are not intended to be directly called, for tight control of execution.

.PHONY: build build-stamp clean clean-patched binary-indep binary-arch binarydocs binary install configure configure-stamp installdocs installbasic unpatch patchsource patch-stamp  php-java-bridge-j2ee php-java-bridge-devel php-java-bridge-docs php-java-bridge-j2ee-docs

The relocatable install

Now we will use the "dirs" file, some "package_name.dir" files, and some inline commands to install into the relocated tree.

The "dirs" file is applicable to the default package, the first target rule. The "package_name.dirs" files are applicable to respective package_name targets. The debhelper dh_installdirs will create the listed directories if they not exist.

usr/bin
usr/sbin
etc/php5/conf.d
var/lib/tomcat5/webapps

Please, note the --keep (or -k) option for dh_clean. It keeps the debian files between binary packages generations. Study the man page for this and other options.

installbasic:
        dh_testdir
        dh_testroot
        dh_clean --keep 

install: installbasic
        $(MAKE) install DESTDIR=${DESTDIR}

This "installini" rule will be used to install some files in desired directories.

Use the dh_installdirs and dh_install instead or cp or mv for cleaner and tracked installation.

Note the absence of leading "/" (for relocation) and one per line to avoid jumping to next rule (a debhelper behaviour of this version).

installini:
        dh_install java.ini etc/php5/conf.d
        dh_install mono.ini etc/php5/conf.d

This rule could be substituded by a suitable "binary_package_name.install" file, used by dh_install. This is the approach of the more recent versions of the example source tree, as it seems a more manageable and modularized way to accomplish the results.

Init.d script

This init.d section is deprecated at upstream source tree. We kept it here for didatic purpose, as you may need it for YOUR package.

The recommended readings LSBInitScripts Init Script Actions Debian Policy Console messages from init.d scripts, LSB Init scripts conventions .

The example source tree php-java-bridge source tree has 2 files for this function, the first calling the second one and both outside final destinations. Also, they are not fully compliant with Debian Policy for init scripts. We used them as starting points and created new files and installed at the final destinations.

The second file is the actual daemon, so we included it at the php-java-bridge.install file.

php-java-bridge /usr/sbin

This line will point the final location for the daemon file.

And very important: it is one of the predefined locations for automatically attribute modification to executable. The debhelper script dh_fixperms change the attribute for you at its predefined directories. Read the dh_fixperms perl source file to discover the other predefined directories.

After, we create a php-java-bridge.init file, (trying to be) compliant with Debian Policy and LSB. Please, see the example source tree php-java-bridge source tree. You must place and name the file "debian/binary_package_name.init" for debhelper processing.

Please, notice the strict formatting LSB header and read the recommended links (see ["LSBInitScripts"] and Init Script Actions for explanations:

### BEGIN INIT INFO
# Provides:          php-java-bridge
# Required-Start:    apache2
# Required-Stop:     apache2
# Should-Start:      $all
# Should-Stop:       $all
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Allow php scripts access java apps
# Description:       The PHP/Java Bridge allows PHP scripts to access java based
#                    applications running in a java application server
### END INIT INFO

The init script call the actual daemon we instructed to install at the executable directory. This init script is using the best practices for Debian systems to date. Despite Debian Policy 9.3 compliant it is still not fully LSB compliant, as it lacks a few commands. Debian Policy 9.4 will require this, and we will improve the script.

Then, we will include the instructions at the rules file to move the two files and change the permissions in compliance to Debian Policy. The daemon and init script are set to executable, root owned. (We show below only the relevant rules lines to this topic)

binary-arch:
        dh_install -a
        dh_installinit -a
        dh_fixperms -a

The splitting begins

The compiled code is installed (almost) as the original upstream code intended.

It was sanely relocated.

Now you will distribute the installed files inside the multiple binary packages.

The debhelper -a, -i, -p, [blank], options

Failure to understand these guidances (thread: arguments of some dh_* into rules) lead to weird packaging behaviour and weird lintian errors, along with misplaced files inside packages.

The -a option is for rule acting on ALL architecture DEPENDENT packages at this WHOLE file, not only at where it is written.

The -i option is for rule acting on ALL architecture INDEPENDENT packages at this WHOLE file, not only at where it is written.

The -p$@ to use target name package files in /debian as parameters. This option defines that a given rule will act ONLY at the specified package/target where it is written. The -p@ can not pass between phony and target. You have to repeat the rules for each target.

The [blank] option (no option) is for the debhelper acting on ALL packages/targets of this WHOLE file, not only at where it is written.

The first binary independent target

/debian/php-java-bridge-j2ee.dirs file

usr/bin
usr/sbin
etc/php5/conf.d
var/lib/tomcat5/webapps

/debian/php-java-bridge-j2ee.docs file

ChangeLog
CREDITS
FAQ.html
NEWS
PROTOCOL.TXT
README
README.GNU_JAVA
README.MONO+NET
VERSION

/debian/php-java-bridge-j2ee.install file for some file almost moved "by hand" after the make install. You use this file with dh_install instead a shell mv or cp.

java-servlet.ini etc/php5/conf.d

/debian/control snippet.

Please, note the "all" architecture argument. It will be used with the -i option ahead.

Package: php-java-bridge-j2ee
Architecture: all

Use the dh_installdirs and dh_install instead or cp or mv for cleaner and tracked installation.

Note the inline dh_install using dynamic variables referring to some directory configured at the begining of the rule file. (thread: how pass variables from files for dh_install into rules?, http://lists.debian.org/debian-mentors/2007/04/msg00031.html).

The inline code does not have a leading "/" for relocating.

See how debhelper apps is leveraged, executing the hard detailed work for you.

Also, the /debian/control file must have some special variables for use here substitution variables ${misc:Depends}, ${python:Depends}, ${perl:Depends} and package_name.substvars and control files. If they are not used, only a warning will be issued. They do not make harm. For sake of future builds, place them on the control file.

php-java-bridge-j2ee: 
        dh_testdir
        dh_testroot
        dh_installdirs  -p$@
        dh_installchangelogs  -p$@  ChangeLog
        dh_installdocs  -p$@
#       dh_installexamples -p$@
# using dh_installdirs and dh_install for this task
# note the absence of leading "/"
        dh_install  --fail-missing --package=$@ ${BUILDDIR}$(PHP_EXT_DIR)/JavaBridge.war var/lib/tomcat5/webapps
        dh_installman  -p$@
        dh_link  -p$@
        dh_strip  -p$@
        dh_compress  -p$@
        dh_fixperms  -p$@
        dh_installdeb -p$@
        dh_shlibdeps  -p$@
        dh_gencontrol  -p$@
        dh_md5sums  -p$@
        dh_builddeb  -p$@

The second binary independent target

/debian/php-java-bridge-devel.docs file. It does not have redundant copyright file, nor install instructions, as this is a binary package, managed by dpkg.

ChangeLog
CREDITS
CVS_RELEASE.sh
FAQ.html
NEWS
PROTOCOL.TXT
README
README.GNU_JAVA
README.MONO+NET
VERSION

/debian/php-java-bridge-devel.examples file for the debhelper placing at the correct directory. The examples/* line avoid nested examples directory, placing the subdirectories inside the first examples dir.

examples/*
install.sh.in
php-java-bridge.spec
test.php
test.bat
test.sh
run-tests.php
security
tests.m4
tests.mono+net
tests.php5
tests.quercus
unsupported
server/php/
server/test/
php_java_lib

/debian/control snippet.

Please, note the "all" architecture argument. It will be used with the -i option ahead.

Package: php-java-bridge-devel
Architecture: all

pitfall-> packages may have "-" in their name. Do not create intermediate (phony) rules with this character or things may go really weird and unexpected (as of april 2007 versions).

See how variables are used at dh_install to point to directories that may be different as different related packages are released (php5). (thread: how pass variables from files for dh_install into rules?, http://lists.debian.org/debian-mentors/2007/04/msg00031.html cont., Make manual - variables flavours.

php-java-bridge-devel:
        dh_testdir
        dh_testroot
        dh_installdirs  -p$@
        dh_installchangelogs  ChangeLog
        dh_installdocs  -p$@
        dh_installexamples  -p$@
        dh_install  --fail-missing -p$@ ${BUILDDIR}$(PHP_EXT_DIR)/*.jar $(PHP_INCLUDE_DIR)/ext/php-java-bridge
        dh_installman  -p$@
        dh_link  -p$@
        dh_strip -p$@
        dh_compress  -p$@
        dh_fixperms  -p$@
        dh_installdeb  -p$@
        dh_shlibdeps  -p$@
        dh_gencontrol  -p$@
        dh_md5sums  -p$@
        dh_builddeb  -p$@

The third binary independent package

/debian/php-java-bridge-docs.docs file

documentation

/debian/control snippet.

Please, note the "all" architecture argument. It will be used with the -i option ahead.

Package: php-java-bridge-docs
Architecture: all

The flux control trick here is to use a phony rule to call the targets and specify desired files at the specific configuration file.

The -p@ can not pass between phony and target. You have to repeat the rules for each target.

binarydocs: php-java-bridge-docs php-java-bridge-j2ee-docs

php-java-bridge-docs: 
        dh_testdir
        dh_testroot
        dh_installdirs  -p$@
        dh_installchangelogs  -p$@  ChangeLog
        dh_installdocs  -p$@
        dh_installman  -p$@
        dh_link  -p$@
        dh_strip  -p$@
        dh_compress  -p$@
        dh_fixperms  -p$@
        dh_installdeb  -p$@
        dh_shlibdeps  -p$@
        dh_gencontrol  -p$@
        dh_md5sums  -p$@
        dh_builddeb  -p$@

The fourth binary independent package

/debian/php-java-bridge-j2ee-docs.docs file

server/documentation/

/debian/control snippet.

Please, note the "all" architecture argument. It will be used with the -i option ahead.

Package: php-java-bridge-j2ee-docs
Architecture: all

The -p@ can not pass between phony and target. You have to repeat the rules for each target.

php-java-bridge-j2ee-docs:
        dh_testdir
        dh_testroot
        dh_installdirs  -p$@
        dh_installchangelogs  -p$@  ChangeLog
        dh_installdocs  -p$@
        dh_installman  -p$@
        dh_link  -p$@
        dh_strip  -p$@
        dh_compress  -p$@
        dh_fixperms  -p$@
        dh_installdeb  -p$@
        dh_shlibdeps  -p$@
        dh_gencontrol  -p$@
        dh_md5sums  -p$@
        dh_builddeb  -p$@

The first binary dependent package

/debian/control snippet.

Please, note the "any" architecture argument. It will be used with the -a option ahead.

Package: php-java-bridge
Architecture: any

The package name is implicit. We reuse other phony rules to reduce rule size.

Close attention to the execution path.

Also, notice the use of --list-missing argument to help discover forgotten files during packaging.

We desire to NOT install a file that was built. To accomplish this, we use the --exclude argument. The --exclude argument MUST be used with an explicit definition of what and where to install, otherwise it is ignored. You could define files at the invocation rule or at a debian/binary_package_name.install file, with or without wildcards.

If you use the file approach, you could NOT use variables in it.

As we are using path defined at compilation time, we MUST place the definition at invocation line.

# Build architecture-dependent files here.
# Caution: the -a option is for rule acting on ALL architecture DEPENDENT 
# packages at this WHOLE file, not only at where it is written.
binary-arch:
        dh_testdir
        dh_testroot
        dh_installdirs -a
        dh_installchangelogs -a ChangeLog
        dh_installdocs -a
        dh_install --exclude=JavaBridge.war -a --fail-missing ${BUILDDIR}$(PHP_EXT_DIR)/* $(PHP_EXT_DIR)
        dh_installman -a
        dh_link -a
        dh_strip -a
        dh_compress -a
        dh_fixperms -a
        dh_installdeb -a
        dh_shlibdeps -a
        dh_gencontrol -a
        dh_md5sums -a
        dh_builddeb -a

The closing: defining the binary rules

Here you define the whole thing. It is the top level abstraction rule. You control the packaging flux here.

# build and install are implicit, but THIS package source needs 2 cycles.
# so the first cycle is implicity and this entry is for the second cycle.
binary: build install binary-arch binary-indep

That is it. You have a complete set of files and rules ready for building your first multiple binary packages software.


CategoryDeveloper