Contributors: RaphaelHertzog
Contents
- Summary
- Advantages of new formats
- Implementation
-
FAQ
- How to convert a source package?
- Does a 3.0 (quilt) source package need to build-depend on quilt?
- Is the README.source file needed with 3.0 (quilt) source packages?
- How to use multiple upstream tarball in 3.0 (quilt) format?
- Why should I convert my package to 3.0 (quilt) format?
- Will my packages be backportable to etch?
- I get lintian errors and warnings with new source formats
- I converted my package but it fails to build or fails to unpack on all buildds
- Why aren't -p0 patch supported in 3.0 (quilt) format?
- How many packages have already been converted?
- But, how do I actually work with the 3.0 (quilt) format?
- Validation of all tools with new source formats
Summary
This page summarizes information concerning the switch to the new source package formats "3.0 (quilt)" and "3.0 (native)". This switch is planned to happen in the squeeze development cycle.
Advantages of new formats
- Supports bzip2 and lzma along with the usual gzip.
- Note: lzma is not allowed on ftp-master.debian.org but xz might be in squeeze+1.
- Supports multiple upstream tarballs.
- Supports inclusion of binary files (no need to uuencode a Debian specific PNG icon for example)
You don't have to repack the upstream tarball to strip the debian directory. (The debian directory is automatically replaced by the content of the .debian.tar.{gz,bz2,lzma} file at unpack time)
Debian-specific changes are no longer stored in a single .diff.gz but in multiple patches in debian/patches/. It is compatible with quilt (hence its name) but does not require its usage as dpkg-source is able to do everything needed by itself. It applies patches at extraction time and update the patch series at build time.
- NMU workflow is now really: unpack, hack, rebuild (for all 3.0 (quilt) packages).
Implementation
- Lenny's dpkg has all the support for the new source formats. This is to avoid problems when working in Lenny with Squeeze source packages.
DAK (Debian Archive Kit) has been updated to cope with the new source package formats. See 457345 for history.
- The plan to change the default source format has been dropped. Instead it is planned to make debian/source/format mandatory at some point and to not have any default format.
FAQ
How to convert a source package?
You need to put "3.0 (quilt)" or "3.0 (native)" in debian/source/format to indicate the desired format to dpkg-source (see dpkg-source(1) for more information):
mkdir debian/source ; echo '3.0 (quilt)' > debian/source/format ; dch 'Switch to dpkg-source 3.0 (quilt) format'
Native packages should not need any other change.
When you switch to "3.0 (quilt)", there are other changes that you might want to do:
If your .diff.gz modifies upstream files, you should really put those changes in separate quilt patches first, otherwise all those changes will be merged in a single quilt patch named debian/patches/debian-changes-<version>. Consider documenting those patches by following DEP-3.
- If you use another patch system (like dpatch or dbs), consider switching to quilt (you can then simplify your debian/rules, see next point).
You should remove everything related to quilt in debian/rules (patch/unpatch logic, cleanup of quilt stamp file and its .pc directory). Note that if you keep those rules, you must unpack the source package with a recent version of dpkg (>= 1.15.5.4) to ensure that dpkg-source creates the .pc files that quilt itself expects. Be aware that some buildds (running an old sbuild) do unpack the source package outside of the build chroot (on a lenny system running an old version of dpkg << 1.14.28), so keeping those rules will make the package fail to build on those buildds.
Apply all the patches from debian/series before building your package: the working directory standard form with "3.0 (quilt)" is with patches applied.
Does a 3.0 (quilt) source package need to build-depend on quilt?
If you drop the quilt usage in debian/rules (patch/unpatch logic), then no.
Is the README.source file needed with 3.0 (quilt) source packages?
No. Since dpkg-source -x applies the patch, you don't have to explain how the patch system works. If lintian gives you the patch-system-but-no-source-readme warning, upgrade to the latest version in sid.
How to use multiple upstream tarball in 3.0 (quilt) format?
Simply put a file <source>_<version>.orig-<component>.tar.{gz,bz2,lzma} near the traditional .orig tarball. The content of that tarball should also be already unpacked in the component subdir of the unpacked source package. The next dpkg-source -b will pick it up automatically.
Note: using a dash in the component requires dpkg >= 1.14.27, so you can't upload such packages until lenny's 4th point release (Debian 5.0.4).
Why should I convert my package to 3.0 (quilt) format?
Pick the reasons that matter to you:
by keeping patches separate and properly documented (with DEP-3):
the Debian changes are more likely to be reviewed (for example thanks to http://patch-tracker.debian.org)
- any upstream contributor is more likely to find and merge the patches (if you haven't forwarded it properly)
- other distributions can reuse our patches
- even if you don't have any upstream patch right now, next time that someone must NMU your package, they can cleanly add a patch (with a proper DEP-3 header) without having to modify the build system
- same applies for derivative distributions that have to modify your packages... you're more likely to be able to find something valuable to merge if their changes are kept as separate patches instead of merged within the .diff.gz
- debian/rules can be simplified by getting rid of the patching/unpatching code (or the uuencoding/uudecoding code if you ship a supplementary binary file like an icon)
- the upstream tarball is bzip2-compressed and you don't want to have to convert it
- the upstream tarball contains a debian directory and you don't want to repackage it just to remove that directory
- the upstream author releases the software in multiple tarballs
- you don't have to add -i -I every time that you manually build the package from the VCS repository (the VCS-specific files are ignored by default)
- in the long run it's best to standardize on a single patch system (new contributors need to learn a single system, more people can help you, etc.) and quilt appears to be that patch system.
Will my packages be backportable to etch?
With 3.0 (quilt), backporting to etch is possible if you don't use multiple upstream tarballs.
If this is the case, you can trivially backport by doing the following:
- unpack the source package with a recent dpkg-source (lenny or newer)
- convert any .orig.tar.bz2 into .orig.tar.gz
- remove debian/source/format (or put "1.0" in it)
- do any other usual backporting (tweaking build-dependencies, etc.)
rebuild (on an etch system) with dpkg-buildpackage -i.pc to avoid including the internal directory
The changes coming from the quilt patches will be in the .diff.gz but it will work.
I get lintian errors and warnings with new source formats
The following warnings are incorrect for the new Debian source formats, and are fixed in the latest Lintian:
unsupported-source-format 3.0 (quilt/native) is wrong (see 552707)
You should always use the latest Lintian version from Sid to test your packages.
I converted my package but it fails to build or fails to unpack on all buildds
If you get a failure like:
dpkg-source: info: applying XXXX 1 out of 1 hunk FAILED -- saving rejects to file xxx.rej
You probably have quilt patches that only applies with fuzz (i.e. they only apply if patch ignores context lines). quilt applies those without failing but dpkg-source doesn't. The solution is to refresh the patches, for example with those commands :
quilt pop -a while quilt push; do quilt refresh -p ab; done
dpkg-dev 1.15.5.3 and later prevent you from building source packages with such patches.
Another problem might be that you converted it to use the "3.0 (quilt)" format but are still using quilt in your debian/rules. The buildd might unpack the source package with an old version of dpkg that does not create quilt's .pc directory. If you then try to apply the patches again they will fail to apply. Until a fixed dpkg is installed everywhere, you should not call quilt during the build process.
Why aren't -p0 patch supported in 3.0 (quilt) format?
See sub-thread starting here.
How many packages have already been converted?
http://upsilon.cc/~zack/stuff/dpkg-v3/
But, how do I actually work with the 3.0 (quilt) format?
To actually take advantage of this new format, see the Quilt for Debian Maintainers guide. It explains how to add and edit patches and integrate things in your workflow.
Validation of all tools with new source formats
For all applications, please be sure to test the 3 important cases:
- 3.0 (native)
- 3.0 (quilt)
- 3.0 (quilt) with multiple upstream tarballs
It's also interesting to test with various compressions methods (gz, bz2, lzma).
Some sample source packages are available here. You can add it to /etc/apt/sources.list with deb-src http://people.debian.org/~hertzog/packages/ debsrc3.0/. They have the following characteristics:
- sample1: 3.0 (quilt) with orig.tar.gz / debian.tar.gz and additional tarballs with all compressions schemes
- sample2: 3.0 (quilt) with orig.tar.bz2 / debian.tar.bz2
- sample3: 3.0 (quilt) with orig.tar.lzma / debian.tar.lzma
- sample4: 3.0 (native) with tar.gz
- sample5: 3.0 (native) with tar.bz2
- sample6: 3.0 (native) with tar.lzma
Current test results:
- lintian
Ready except for source packages with multiple tarballs.
- apt-get source
- Works.
- dput
- Works.
- sbuild (in Debian sid)
- Works.
Sample build log with 3.0 (quilt): sample1_1.0-1-i386-20090730-1247
Sample build log with 3.0 (native): sample6_1.0-i386-20090730-1246
- svn-buildpackage
Not working yet: http://lists.debian.org/debian-devel/2009/03/msg00353.html
- svn-inject doesn't support importing 3.0 (quilt) package (even with a single upstream tarball)
Patch is waiting in 388567
- bzr-builddeb
- According to changelog for bzr-builddeb version 2.4:
Some support for v3 source formats (Closes: 562991)
- Those that look quite a lot like v1 are supported well.
- .tar.bz2 tarballs are supported for import, building, merge-upstream,
- etc., but only enabled for the latter with a --v3 switch for now.
- Multiple orig.tar.gz is not supported.
- .tar.lzma is not supported awaiting pristine-tar support.
Testing is still needed for more complex use cases.
- git-buildpackage
- git-buildpackage; works for building 3.0 (native) and 3.0 (quilt) packages
- git-import-dsc: needs to learn about new source format.
- git-import-orig: needs to learn about multiple tarballs and lzma.
- hg-buildpackage
- Assumes that upstream source is a single .tar.gz.
- Depends on tar recognising which compression method is used when importing an upstream tarball.
- tla-buildpackage
Not working yet: 557333
- Not tested yet for 3.0 (native), different compression methods and muiltiple tarballs.
- dupload
- Works.
- pbuilder/cowbuilder
- Works.
- approx
- Works.
- reprepro
works (at least with the examples in http://people.debian.org/~brlink/v3test)
- reprepro's changestool
- works, but extracing Section and priority from .debian.lzma only since 3.9.0
- debmirror
Not tested yet (but should work as it works based on file names).
- devscripts uscan and uupdate
Wishlist: handle multiple upstream tarballs. 531321
- devscripts uupdate
Wishlist: message to point to debian.tar.gz not just diffs. 575820
- devscripts debdiff
Wishlist: handle debian.tar.gz diff. 575821
- other devscripts tools ?
A new debreview tool would be welcome to replace zless *.diff.gz: 575394
- DEHS
- Works.
- Launchpad/Soyuz (Ubuntu and PPAs):
- apt-cacher-ng
- support added in version 0.4.3-1
- What else should we check ?
Please try using new source packages with all your usual tools (in particular if you maintain them) and file bugs if you encounter problems and usertag them appropriately (user hertzog@debian.org / tag 3.0-quilt-by-default). Also update the list above with any relevant information (including pointers to bugs).