Differences between revisions 153 and 154
Revision 153 as of 2014-02-19 12:20:19
Size: 16493
Editor: SimonKainz
Comment: add codespell to package list
Revision 154 as of 2014-03-11 12:56:34
Size: 16504
Editor: SimonKainz
Comment: added duck to installed packages, add duck to automated tests
Deletions are marked like this. Additions are marked like this.
Line 140: Line 140:
sudo apt-get install lintian lintian4python libcrypt-ssleay-perl libwww-perl cppcheck devscripts pyflakes libperl-critic-perl desktop-file-utils mp3check checkmp3 mp3val fontforge-nox freetype2-demos gettext-lint jlint pngcheck jpeginfo vorbis-tools oggz-tools lacheck fdupes python-magic libxml2-utils similarity-tester moreutils pep8 blhc libconfig-model-perl libconfig-model-dpkg-perl bfbtester elfutils hlint adequate i18nspector epubcheck iwyu python-jpylyzer libb-lint-perl codespell sudo apt-get install lintian lintian4python libcrypt-ssleay-perl libwww-perl cppcheck devscripts pyflakes libperl-critic-perl desktop-file-utils mp3check checkmp3 mp3val fontforge-nox freetype2-demos gettext-lint jlint pngcheck jpeginfo vorbis-tools oggz-tools lacheck fdupes python-magic libxml2-utils similarity-tester moreutils pep8 blhc libconfig-model-perl libconfig-model-dpkg-perl bfbtester elfutils hlint adequate i18nspector epubcheck iwyu python-jpylyzer libb-lint-perl codespell duck
Line 215: Line 215:
duck

There is some new and interesting documentation on packaging:

If you need to create rapidly a package, use equivs:

  • read first ?how to create an empty package

  • then read how to create a ?package with some files

If you prefer to gain real knowledge about Debian packaging:

Translation(s): English - Italiano - 简体中文


This is a pragmatic approach to learning how to create Debian packages.

Debian packaging is not that hard. Read first the introduction to Debian Packaging.

If the information below doesn't answer your questions, please look in the New Maintainer's Guide and in Debian Policy

The Debian package management chapter of the Developers Reference contains lots of useful information for handling all kinds of problems one runs into with apt and dpkg.

If you want to create an official Debian package, make sure it's not already packaged.

If somebody is working on the package, you should contact them if you intend to make an official package together.

For a slightly more modern way of managing all of this, check out git-buildpackage. TODO: write this page around git-buildpackage, instead of quilt, for a less baroque and more familiar experience.

Initial compilation

Before starting there are some risks you should be aware of:

WARNINGS:

  • All these operations are preferably done in a chroot environment for safety/security reasons. pbuilder and sbuild are such environments. Please consult the pbuilder or sbuild documentation.

  • running make on a system can be a security risk! It is recommended to check beforehand that the Makefile does not contain any funny stuff. Obscure/specially crafted applications might fall into this case.

  • do NOT run make install, unless you are absolutely sure is safe to do that (check the install target of the makefile). This is a security risk and you may risk to break and/or compromise your system.

Steps:

  • Try to compile the package, but do not install it.
    • most compile failures will be the result of missing dependencies (usually they are mostly missing *-dev packages)
    • note every new package which you needed to install for compilation
      • some of the packages that were already installed on your system might be necessary, too (note: objdump -p | grep NEEDED will show exactly the needed libraries)
      • these packages will end up in the build-dependency list of the package
    • note quirks and tricks needed for the package to compile (missing files, file modifications, etc.), you will need to reproduce later in the package build rules
  • If you think you need specific software not already packaged, make sure that:
    • the needed software is not present in another package
    • the package is not being worked on (ITP (Intent To Package) is pending)

    • are you sure it's not packaged? :) You could check other Debian based distributions, too.

It is best to create a chroot jail in which to build the application. This mitigates security and system corruption problems. In addition it ensures that any local changes to your machine will not interfere with the build.

If the package compiles, then it means you may have figured out all the dependencies. Don't worry, this will be checked later in a chroot environment. Try the application in a chroot environment (some things might be needed in the chroot jail - for example, proc may need to be mounted, sys to be mounted, etc.). Broken installation scripts are, unfortunately, very common and you probably don't want to run a test installation yet. You might prefer doing the tests in a qemu virtual machine in snapshot mode - changes are not permanent in snapshot mode and you can keep you development machine safe from malicious/broken makefiles.

''Debianization''

After the first compilation, it's time to create the Debian specific part of the package.

Debianize the package by using dh_make.

File debian/control: Add to the Build-Depends (sometimes even Build-Depends-Indep) the list of packages needed to be installed for the application to compile (remember the list done previously). You should leave out any packages that are listed in /usr/share/build-essential/essential-packages-list or /usr/share/build-essential/list and also leave out any packages that listed as dependencies by any of those packages.

File debian/rules: If any quirking was done then:

  • automate the quirking and the changes needed to be done in the debian/rules file under a patch target
  • make sure there will be an unpatch target that reverses all the changes done in the patch target
  • make the clean target depend on the unpatch target
  • make the build target depend on the patch target

Review each of the template files (debian/*.ex). If your package needs that feature, then customize the file as needed and rename it without the ".ex".

Delete all the unneeded templates. Don't worry, they can be restored at any time with "dh_make -a".

Make sure that all the directories you will place files in are listed in the debian/dirs file without a leading slash (/).

Make sure that the files are installed in the proper place (under a directory, not on the root system). Take care at the install target from the application makefile. If the application uses autoconf and automake, it may be enough to set the environment variable DESTDIR, e.g. "make DESTDIR=$(CURDIR)/debian/packagename". (dh_make will set this up automatically.)

Note: very often the upstream package will install files under /usr/local. DO NOT INSTALL ANY FILES IN THERE.

Initial compilation of the package

TODO: add notes

Building Debian packages

To make sure that a Debian package meets all build dependencies and is not influenced by anything specific to the user's environment, packages should be built in a chroot environment. Tools like pbuilder can be used for this.

When working on a package, a faster rebuild can be done with 'debuild'. But then, all build-dependencies must be satisfied in the installation where the package is built. All necessary packages can be installed automatically with apt-get build-dep. A complete example for building the foo package looks like this:

 $ apt-get source foo
 $ cd foo-0.0.1
 $ sudo apt-get build-dep foo
 $ debuild -i -us -uc -b

This is usually enough for you to backport packages.

Testing and Check the package

All tests in chroot

Why is it a good idea to use a chroot?

  • Security
  • Testing
  • No risk to break the developer's system
  • Wrongly installed packages can be identified easily (ls -lR in the root of the chroot before and after the build; diff -u before.list after.list)
  • Missing build dependencies are identified easier, thus avoiding serious FTBFS (Fails To Build From Source) bugs

  • The chroot system is a pure Debian system belonging to one suite only (e.g., Sarge, Etch or Sid if the chroot was built properly with debootstrap or cdebootstrap)

Testing the package

licensecheck -r *

TODO: add notes

Lintian

Test all the packages with lintian.

lintian package-version.changes

Piuparts

Test all the .deb packages with piuparts.

piuparts binpackage-version.deb

Check points for any package

  • Is there a man page for every executable file in the package? In some rare cases it makes sense to have one man page for multiple binaries in a package.
  • Does it have a Debian menu entry? (daemons usually don't need this) Does it have a .desktop file?
  • If there are architecture independent things in the package and if they are big enough, are those packaged in a separate package with arch all?

  • Does the package compile cleanly? This does not mean that you should redirect the errors to /dev/null

  • Does lintian scream when checking the .changes file?
  • What does piuparts have to say about the .debs?
  • Checklist for NEW

  • Checklist for sponsors

  • Some automated checks:

sudo apt-get install lintian lintian4python libcrypt-ssleay-perl libwww-perl cppcheck devscripts pyflakes libperl-critic-perl desktop-file-utils mp3check checkmp3 mp3val fontforge-nox freetype2-demos gettext-lint jlint pngcheck jpeginfo vorbis-tools oggz-tools lacheck fdupes python-magic libxml2-utils similarity-tester moreutils pep8 blhc libconfig-model-perl libconfig-model-dpkg-perl bfbtester elfutils hlint adequate i18nspector epubcheck iwyu python-jpylyzer libb-lint-perl codespell duck
export BUILDPATH=/home/foo/path/to/foo-1.2.3
cd $BUILDPATH
lintian ../*.changes
lintian4py ../*.changes
cme check dpkg
cme check dpkg-control
cme check dpkg-copyright
debuild && debuild && debuild
uscan --verbose
licensecheck --check=. --recursive --copyright .
grep -i warn ../*.build
blhc --all ../*.build
cppcheck -j8 --quiet -f .
jlint.sh 2>&1 | fgrep -v 'Verification completed: '
pyflakes .
pep8 --ignore W191 .
perlcritic -1 .
suspicious-source
bfbtester -x 10 -a debian/*/bin debian/*/usr/bin debian/*/sbin debian/*/usr/sbin debian/*/usr/games
hlint .
codespell --quiet-level=3
find -iname \*.sh -print0 | xargs --no-run-if-empty --null -n1 sh -n
find -iname \*.sh -print0 | xargs --no-run-if-empty --null -n1 checkbashisms
find \( -iname \*.pm -o -iname \*.pl \) -print0 | xargs --no-run-if-empty --null -n1 perl -wc
find \( -iname \*.pm -o -iname \*.pl \) -print0 | xargs --no-run-if-empty --null -n1 perl -MO=Lint 2>&1 | grep -v ' syntax OK$'
find -iname \*.php\* -print0 | xargs --no-run-if-empty --null -n1 php -l -f
find \( -iname \*.c -o -iname \*.cxx -o -iname \*.cpp -o -iname \*.cc -o -iname \*.h -o -iname \*.hpp -o -iname \*.hh -o -iname \*.hxx \) -print0 | xargs --no-run-if-empty --null -n1 include-what-you-use
find -iname \*.desktop -print0 | xargs --no-run-if-empty --null -n1 desktop-file-validate
find -iname \*.mp3 -print0 | xargs --no-run-if-empty --null mp3check --error-check --anomaly-check
find -iname \*.mp3 -print0 | xargs --no-run-if-empty --null checkmp3
find -iname \*.mp3 -print0 | xargs --no-run-if-empty --null mp3val
find \( -iname \*.ttf -o -iname \*.otf -o -iname \*.sfd -o -iname \*.pfa -o -iname \*.pfb -o -iname \*.bdf -o -iname \*.pk -o -iname \*.ttc -o -iname \*.pcf \) -print0 | xargs --null --no-run-if-empty --max-args=1 fontlint
find \( -iname \*.ttf -o -iname \*.otf \) -print0 | xargs --no-run-if-empty --null -n1 ftvalid
find \( -name \*.po -o -name \*.pot \) -print0 | xargs --max-args=1 --no-run-if-empty --null msgfmt --check --check-compatibility --check-accelerators
find \( -name \*.po -o -name \*.pot \) -print0 | xargs --no-run-if-empty --null POFileChecker
find \( -name \*.po -o -name \*.pot \) -print0 | xargs --no-run-if-empty --null POFileSpell
find \( -name \*.mo -o -name \*.po -o -name \*.pot \) -print0 | xargs --no-run-if-empty --null i18nspector
find -iname *.png -print0 | xargs --no-run-if-empty --null pngcheck -q
find \( -iname \*.jpg -o -iname \*.jpeg \) -print0 | xargs --no-run-if-empty --null jpeginfo --check --quiet | fgrep -v "[OK]"
find \( -iname \*.jp2 -o -iname \*.j2k -o -iname \*.jpf -o -iname \*.jpx -o -iname \*.jpm -o -iname \*.mj2 \) | xargs jpylyzer --wrapper | xmllint --format - | egrep 'fileName|isValid' | tr -d \\n | sed 's_</isValidJP2>_&\n_g;s_ *<fileName>__g;s_</fileName> *__g;s_</\?isValidJP2>_ _g;s_False_is an invalid JPEG2000 file_g' | sed '/True *$/d'
find \( -iname \*.ogg -o -iname \*.oga -o -iname \*.ogv \) -print0 | xargs --no-run-if-empty --null ogginfo -q | grep -v 'Processing file ' | cat -s
find \( -iname \*.ogg -o -iname \*.oga -o -iname \*.ogv \) -print0 | xargs --no-run-if-empty --null oggz-validate
find -iname \*.xml -print0 | xargs --no-run-if-empty --null xmllint --noout
find -iname \*.tex -print0 | xargs --no-run-if-empty --null lacheck
find -iname \*.epub -print0 | xargs --no-run-if-empty --null -n1 epubcheck -quiet | egrep -v '^(Check finished with warnings or errors| *)$'
find -iname *.py -print0 | xargs --no-run-if-empty --null grep 'environ *\[.HOME.\]'
find -iname \*paypal\*.png -o -iname \*paypal\*.gif -o -iname \*flattr\*.png -iname \*flattr\*.gif
find \( -iname \*.png -o -iname \*.gif -o -iname \*.jpg -o -iname \*.jpeg \) -print0 | xargs --no-run-if-empty --null grep -iE 'inkscape|gimp'
find ! \( -iname \*.icns -o -iname \*.bmp -o -iname \*.ico -o -iname \*.png -o -iname \*.gif -o -iname \*.jpg -o -iname \*.jpeg -o -iname \*.mo -o -iname \*.gz -o -iname \*.zip -o -iname \*.pdf -o -iname \*.bz2 -o -iname \*.tar -o -iname \*.odt -o -iname \*.docx -o -iname \*.torrent -o -iname \*.pyc -o -iname \*.pyo -o -iname \*.o -o -iname \*.wav -o -iname \*.ttf -o -iname \*.ogg -o -iname \*.oga -o -iname \*.ogv -o -iname \*.xcf -o -iname \*.so -o -iname \*.so.\* -o -iname \*.gmo -o -iname \*.debug -o -iname \*.tga \) -print0 | xargs --no-run-if-empty --null isutf8
find ! -type d | xargs file  | grep ': *ELF ' | sed 's/: +*.*//' | xargs --no-run-if-empty eu-elflint --quiet --gnu-ld --strict
find -type f -iname \*.gz -print0 | xargs --no-run-if-empty --null gzip --test
find -type f -iname \*.bz2 -print0 | xargs --no-run-if-empty --null bzip2 --test
find -type f -iname \*.xz -print0 | xargs --no-run-if-empty --null xz --test
find \( -type f -iname \*.zip -o -iname \*.jar -o -iname \*.pk3 -o -iname \*.wz \) -print0 | xargs --no-run-if-empty --null unzip -t
find -type f -iname \*.lzma -print0 | xargs --no-run-if-empty --null lzma --test
find -type f -iname \*.lzo -print0 | xargs --no-run-if-empty --null lzop --test
find -type f -iname \*.lz -print0 | xargs --no-run-if-empty --null lzip --test
find -type f -iname \*.uu -print0 | xargs --no-run-if-empty --null uudecode -o /dev/null
grep -r '/tmp/' .
grep -r '/home' .
grep -r '/srv' .
grep -r '/opt' .
grep -r 'PATH' .
grep -r 'PERLLIB' .
grep -r 'x86_64-linux-gnu' .
grep -r 'site-packages' .
grep -rC4 'system *(' .
grep -rC4 'popen *(' .
grep -ri 'fixme' .
grep -ri 'todo' .
grep -ri 'hack' .
fdupes -r .
find -type f | sim_text -ipTt 75 | tac | column -t
apt-get install $(sed -n 's/^Package: //p' debian/control | tr '\n' ' ') ; debi ; apt-get -f install ; adequate $(sed -n 's/^Package: //p' debian/control | tr '\n' ' ')
duck
echo TODO: jslint puppet-lint acheck linkchecker ssl-cert-check

This page is still incomplete; please help by adding information to it. Do not forget that the information here should be in a concise form.

Examples

  • The Oolite package was created following this technique (actually, this page was written during Oolite's initial packaging stage)
  • There are interesting examples in the PDF file included in the package : packaging-tutorial.