Differences between revisions 58 and 97 (spanning 39 versions)
Revision 58 as of 2012-11-15 18:51:56
Size: 26008
Editor: PaulWise
Comment: Producing OSS
Revision 97 as of 2015-11-20 16:35:58
Size: 36724
Editor: ?Thomas Goirand
Comment:
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
This page is aimed at upstream developers who would like their software to be packaged for Debian ([[AdvantagesForUpstream|why would you want this?]]), or whose software is already packaged for Debian. It attempts to document what we do with your software, the resources we make available to you and what you can do to make the packaging process smoother. If you require any clarifications or have any questions about this document, please direct them to the [[http://lists.debian.org/debian-upstream/|debian-upstream]] mailing list. For futher, more extensive advice on making your project more friendly to Debian and other Free Software distributions, please refer to the '''[[#external-advice|External advice]]''' section below. This page is aimed at upstream developers who would like their software to be packaged for Debian ([[AdvantagesForUpstream|why would you want this?]]), or whose software is already packaged for Debian. It attempts to document what we do with your software, the resources we make available to you and what you can do to make the packaging process smoother. If you require any clarifications or have any questions about this document, please direct them to the [[http://lists.debian.org/debian-upstream/|debian-upstream]] mailing list. For further, more extensive advice on making your project more friendly to Debian and other Free Software distributions, please refer to the '''[[#external-advice|External advice]]''' section below.
Line 13: Line 13:
The source code archive should only contain source code, never any files that are created for the specific system during compilation. We recognize that it is fairly common to ship lexer and parser code or compiled documentation as part of the source package so administrators building from source do not need to install the build tools, however we need to rebuild all generated files to make sure that they can really be built from source, so please include methods to regenerate these files. For some guidelines about source for artwork, audio etc, please take a look at the games advice in the '''[[#external-advice|External advice]]''' section above.

It appears that the "rebuild everything" rule is applied inconsistently, for example, most packages do not rebuild the files generated by autoconf and automake. This is mainly for historical reasons, since these tools broke often during automatic builds.

Please do not include other packages that are also shipped separately inside your source archive, or if you do, please make sure that these can be reliably ignored. If a security issue is found in other included packages, it is far easier to rebuild one package than to scan the entire archive for all copies of this code and patch them individually (this happened for zlib, for example).

(why we'd like to have a single .tar.gz archive from you, at least for the time being)

Many upstream code sources tend to include a rough {{{/debian}}} directory among source files to ease bleeding-edge package compilation and installation on debian (and derived) systems; while this is a good effort, it would be better to leave it out of the final tarball as it can interfere with debian's own packaging effort. Keeping it only in your VCS repository is usually a much saner default.
The source code archive should contain source files only, never any files that are automatically generated from other files. We recognize that it is fairly common to ship lexer and parser code or compiled documentation as part of the source package so administrators building from source do not need to install the build tools, but we need to rebuild all generated files to make sure that they can really be built from source, so please include methods to regenerate these files. For guidelines about source for artwork, audio etc, please take a look at the games advice in the '''[[#external-advice|External advice]]''' section above.

The "rebuild everything" rule is currently applied inconsistently. For example, most packages do not rebuild the files generated by autoconf and automake. This is mainly for historical reasons, since in the past these tools would often break during automatic builds.

Some projects include a rough {{{/debian}}} directory among source files to ease bleeding-edge package compilation and installation on debian (and derived) systems. While this is a good effort, it is better to leave it out of the final tarball as it can interfere with debian's own packaging effort. Keeping it only in your VCS repository is usually a much saner default if it lives in a specific packaging branch, which mimics what Debian package maintainers do using git-buildpackage. Though leaving the debian folder in the normal branch can also interfere if the package maintainer is using an upstream git packaging workflow (for example: git tag based git-buildpackage workflow).


=== No inclusion of third party code ===

Please do not include other code (like libraries) or data that are also shipped separately inside your source archive, or if you do, please make sure they can be reliably ignored. Instead of shipping third party libraries you should rather make sure your program will be link nicely against recent versions of these libraries. If a security issue is found in one of the bundled packages, it is far easier to rebuild one package than to scan the entire archive for all copies of this code and patch them individually (this happened for zlib, for example).

It is even worse if you maintain your private forks of third party libraries. This is not only troublesome for distributors but in the long run also to your own project. You should always make sure that the patches you might need for your specific application will be backported to the library upstream - that's simply how Free Software works.

To make sure your software will run with different versions of third party code it is way better to provide test cases you can run at any time to get reproducible results (which is also an additional profit for your own project).
Line 26: Line 31:
Please provide an archive of your source code as tar.{gz,bz2,lzma,xz} file since that's what our toolchain can currently work with.

Debian needs to make sure that it distributes the source code for all binaries in its archives. Debian achieves this by building all binaries itself, from your source code.

If your software depends on other libraries, then Debian also needs to make sure that your software compiles and works with the version of these libraries available in Debian. Debian may compile your software against another version of some library than you do. Therefore it's not of any help for Debian if you include convenience copies of these dependencies in your source tarball.

If you distribute precompiled binaries with your source code, then the Debian Maintainer has extra work to strip out these binaries and repackage your tarball. However this is not an ideal situation, since Debian would prefer to build the software packages it provides from the original tarballs you released.
Please provide an archive of your source code as tar.{gz,bz2,lzma,xz} file. That's what our toolchain can currently work with.

Debian needs to make sure that it distributes the source code for all binaries in its archives. Debian achieves this by building all binaries itself from your source code.

If your software depends on other libraries, then Debian also needs to make sure that your software compiles and works with the version of these libraries available in Debian. Debian may compile your software against a different version of some library than you do. Therefore it's not of any help for Debian if you include convenience copies of these dependencies in your source tarball.

If you distribute precompiled binaries with your source code, then the Debian Maintainer has extra work to strip out these binaries and repackage your tarball. This is not an ideal situation. Debian would prefer to build the software packages it provides from the original tarballs you released.

Line 36: Line 43:
The [[DFSGLicenses]] page has a lot of information about the various licenses currently found in Debian. The [[DFSGLicenses]] page describes the licenses currently found in Debian.
Line 39: Line 46:
 * There is usually no need for different "types" of files to have different licenses. It really complicates matters as all the license terms need to be followed at the same time, and if two are incompatible, we cannot distribute the package at all.  * There is usually no need for different types of files to have different licenses. It really complicates matters as all the license terms need to be followed at the same time, and if two are incompatible, we cannot distribute the package at all.
Line 48: Line 55:
Each time you reach a point in development of your software where you feel everything generally works and you've fixed a variety of bugs or implemented some interesting new features, please release a downloadable tarball with a version number. This doesn't necessarily mean that you have to keep supporting that version afterwards if you're short of resources, so long as you make that clear. But please don't develop only in a version control repository and with snapshots, or do releases only as tags. It makes it much more difficult to grab a stable point that you think generally works. A distro needs to pull in this stable point rather than an automatic snapshot which inadvertently happened in the middle of some known transition or partial development. Each time you reach a point in development of your software where you feel everything generally works and you've fixed a variety of bugs or implemented some interesting new features, please release a downloadable tarball with a version number. This doesn't necessarily mean that you have to keep supporting that version afterwards if you're short of resources, so long as you make that clear. But please don't develop only in a version control repository and with snapshots, or do releases only as tags. It makes it more difficult to grab a stable point that you think generally works. A distro needs to pull in this stable point rather than an automatic snapshot which inadvertently happened in the middle of some known transition or partial development.
Line 58: Line 65:
=== Tarballs ===

If your build system is autotools please generate your tarballs by running `make distcheck`, which builds and verifies the tarballs. Other build systems will have standard targets to create source tarballs, please use them.

If you make tarballs of your releases, please sign them with your OpenPGP keys. This wiki provides further information on [[Keysigning|creating and signing GPG keys]] and [[Keysigning/Coordination|finding somebody to sign your key]]. External advice is available for [[https://help.riseup.net/en/security/message-security/openpgp/best-practices|best practice OpenPGP/GPG key management]].
Line 59: Line 72:
Debian's Policy requires that Debian packages follow the Filesystem Hierarchy Standard for file locations. It should also be noted that Debian packages are not installed into /usr/local. Thus Debian config files are found in /etc, and manual pages in /usr/share/man. If your program normally installs these to a different location please be sure to provide a mechanism to change the locations without patching the source. Use of environment variables for this purpose is recommended. Debian's Policy requires that Debian packages follow the [[FilesystemHierarchyStandard|Filesystem Hierarchy Standard]] for file locations. Debian packages are not installed into /usr/local. Thus Debian config files are found in /etc and manual pages in /usr/share/man. If your program normally installs to a different location please be sure to provide a mechanism to change the locations without patching the source. Use of environment variables for this purpose is recommended.
Line 61: Line 74:

== User home directories ==

When reading/writing cache, configuration, data or sockets for your software, please follow the [[http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html|XDG Base Directory Specification]] to determine the right paths. There are [[XDGBaseDirectorySpecification#libraries|libraries available in Debian]].

When accessing files relative to the user's home directory, first check the HOME environment variable (using [[DebianMan:3/getenv|getenv(3)]] or similar) and fall back on information from the password database (using [[DebianMan:3/getpwuid|getpwuid(3)]] or similar) when HOME is not present.
Line 73: Line 92:
 * We will try to run it regularly and report the results at [[ci.debian.net]].
Line 75: Line 95:
 * Be careful about security. Respond promptly to security problems.
 * Be careful about security.
 * Have a well-publicised preferred method for reporting security issues and respond promptly to reports about security problems.
 * Request a CVE identifier for every security issue. A guide for requesting CVE identifiers can be found from Kurt's [[http://people.redhat.com/kseifrie/CVE-OpenSource-Request-HOWTO.html|HOWTO]] and include the CVE identifier(s) in commit logs, NEWS and ChangeLog files and announcements.
 * If you fix a security issue don't try to hide the fixed problem. This only makes it harder for distros to patch issues. Evil attackers are still going to find out, which problem existed and on what versions.
 * When announcing security fixes, provide patches or links to commits for each version the issue was fixed in.
 * Either provide security fixes for old versions that are distributed in the stable releases of distros or provide support/help to distro security teams and package maintainers.
 * [[https://www.debian.org/security/faq#contact|Contact the Debian security team]] in case of any questions.

== Libraries ==

Assign a SONAME, track ABI compatibility and change the SONAME when ABI compatibility is broken. Avoid breaking ABI if it is possible. You can use the [[http://ispras.linuxbase.org/index.php/ABI_compliance_checker|ABI Compliance Checker]] as a method of double-checking your own ABI compatibility tracking.

Use private libraries instead of public shared libraries if your library is only meant for use in your application.

Use 'hidden' as the default symbol visibility and only whitelist public parts of the API.
Line 87: Line 122:
Using Git as your VCS might be a [[http://whygitisbetterthanx.com|very good choice]]. Git is also the most widely used [[http://upsilon.cc/~zack/stuff/vcs-usage/|VCS for packaging in Debian]] and mandatory for Fedora. If you're using Git  then you make it much easier for (many) Debian maintainers to submit patches and review changes between versions.

Debian needs to make sure that it really packages your original source code; source code repositories can be [[http://www.koch.ro/blog/index.php?/archives/153-On-distributing-binaries.html|compromised]]. One easy way to do so is to rely on the tag signing capability of Git together with the web-of-trust. So please tag your releases with
Using Git as your VCS might be a [[http://whygitisbetterthanx.com|very good choice]]. Git is also the most widely used [[http://upsilon.cc/~zack/stuff/vcs-usage/|VCS for packaging in Debian]] and mandatory for Fedora. If you're using Git then you make it easier for some Debian maintainers to submit patches and review changes between versions.

Debian needs to make sure that it really packages your original source code; source code repositories can be [[http://www.koch.ro/blog/index.php?/archives/153-On-distributing-binaries.html|compromised]]. One easy way to do so is to rely on the tag signing capability of Git together with the web-of-trust. So please tag your releases with the following command. You can also [[http://mikegerwitz.com/papers/git-horror-story|sign every commit]] that enters your repository.
Line 94: Line 130:
This wiki provides further information on [[Keysigning|creating and signing GPG keys]] and [[Keysigning/Coordination|finding somebody to sign your key]]. This wiki provides further information on [[Keysigning|creating and signing GPG keys]] and [[Keysigning/Coordination|finding somebody to sign your key]]. External advice is available for [[https://help.riseup.net/en/security/message-security/openpgp/best-practices|best practice OpenPGP/GPG key management]].
Line 99: Line 135:
You should never assume that the top-level directory name is any particular value or matches any particular pattern.
 * You're free to use any build system you like. Debian uses a Makefile ([[debian/rules]]) to call your build system but that does not mean that you should use make too.
 * However there are a few experiences we've made with different build systems. Please see below for your particular system.
 * You should never assume that the top-level directory name is any particular value or matches any particular pattern.

==== Generated files ====

We need to be able to build everything from source. To ensure that build-dependencies are still working, we also like to rebuild everything from scratch periodically so it is best for generated files to not be available in the tarball. Those that are (such as generated build systems - autotools) need to be regenerated when their source changes or they are removed. We encourage maintainers of Debian packages to rebuild all generated files, including build systems.

==== Verbose builds ====

We need to record as much detail from the build log as possible so that we can debug build failures and other strange issues. Your build system may have a silent building option to hide the full commands (including arguments) used during building. If it does, we recommend that it be off by default or that one of the following commands enables verbose building.

{{{
./configure --disable-silent-rules
make V=1
}}}
Line 129: Line 179:
If you use {{{config.guess}}} and {{{config.sub}}}, please make a habit of always copying the latest versions from ftp.gnu.org or some other source (such as {{{/usr/share/misc}}} on a Debian system with DebPkg:autotools-dev installed) into your package when you release. If you use {{{autoreconf}}}, it can do this for you automatically provided that you keep your local system version up to date. Most of your users may not care about the changes, but Debian often does, since Debian supports a wide variety of architectures.

(we like it, but keep it fresh :) )
If you use {{{config.guess}}} and {{{config.sub}}}, please make a habit of always copying the latest versions from ftp.gnu.org or some other source (such as {{{/usr/share/misc}}} on a Debian system with DebianPkg:autotools-dev installed) into your package when you release. If you use {{{autoreconf}}}, it can do this for you automatically provided that you keep your local system version up to date. Most of your users may not care about the changes, but Debian often does, since Debian supports a wide variety of architectures.
Line 134: Line 182:
(we sort of like it)

Some {{{make}}} variables are reserved to the user, and the [[http://www.gnu.org/software/libtool/manual/automake/User-Variables.html#User-Variables|Automake manual]] and the [[http://www.gnu.org/prep/standards/standards.html#Command-Variables|GNU coding standards]] advise never to use them for switches that are required for proper compilation of the package. When a Debian binary package is built, default environment variables are prepared by {{{dpkg-buildflags}}} (In Debian Wheezy: {{{CFLAGS}}}, {{{CPPFLAGS}}}, {{{CXXFLAGS}}}, {{{FFLAGS}}} and {{{LDFLAGS}}}), to allow the build system to override the corresponding variables in the {{{Makefile}}}. We therefore strongly recommend to follow the above advice, and to make your makefiles use these variables (where relevant) in a way that our build system can override them.

Some {{{make}}} variables are reserved to the user, and the [[http://www.gnu.org/software/libtool/manual/automake/User-Variables.html#User-Variables|Automake manual]] and the [[http://www.gnu.org/prep/standards/standards.html#Command-Variables|GNU coding standards]] advise never to use them for switches that are required for proper compilation of the package. When a Debian binary package is built, default environment variables are prepared by {{{dpkg-buildflags}}} (In `dpkg` 1.17.0: {{{CFLAGS}}}, {{{CPPFLAGS}}}, {{{CXXFLAGS}}}, {{{FFLAGS}}}, {{{GCJFLAGS}}} and {{{LDFLAGS}}}), to allow the build system to override the corresponding variables in the {{{Makefile}}}. We therefore strongly recommend to follow the above advice, and to make your makefiles use these variables (where relevant) in a way that our build system can override them.
Line 140: Line 187:
Please don't use SCons: we will have to re-implement many standard features of autoconf/automake, including DESTDIR, out of tree builds, cleaning and more. Please don't use SCons. It is hard to use it correctly. For instance SCons is designed to ignore environment variables such as CFLAGS (unless your add code for this). It also does not support DESTDIR out of the box. As an upstream you have to explicitly add code for that (or Debian has to patch). Support for SONAMEs (library versioning) is also absent. The general observation is that many projects, that use SCons, do not have a working install target. Since projects work around these limitations individually there is no way to just use a SCons project in Debian, but more work is required to invoke it correctly.

The gentoo wiki has a detailed list of [[http://wiki.gentoo.org/wiki/SCons#Why_you_should_NOT_use_SCons_in_your_project|shortcomings]].

If you choose to use SCons anyway, please ensure that the usual environment compiler variables (CC, CFLAGS, ...) and path variables (DESTDIR, BINDIR, LIBDIR, ...) are honoured. There is a [[http://scons.org/wiki/Installer|recipe]], that addresses some of these.
Line 144: Line 195:
Using waf as build system is discouraged. One of the reasons is the recommendation to ship a waf executable in every single package using it, instead of using a system wide one. Also note that just shipping the waf executable (which contains a binary blob) is considered to be not complient with the Debian Free Software guidelines by the FTP Team. Please see [[http://bugs.debian.org/645190|#645190]] and UnpackWaf for more details on the issue and how to avoid it, if you have to use waf. Using waf as build system is discouraged. One of the reasons is the recommendation to ship a waf executable in every single package using it, instead of using a system wide one. Also note that just shipping the waf executable (which contains a binary blob) is considered to be not compliant with the Debian Free Software guidelines by the FTP Team. Please see [[DebianBug:645190|#645190]] and UnpackWaf for more details on the issue and how to avoid it, if you have to use waf.

==== waf discussion ====

'''TODO: please address this objection'''

It is true that waf pushes the "carry the build system with the
project" line. In fact Gradle has done something along these lines as
well. Indeed SCons supports this way of working. In a global context, it
is a very good idea, even if it is conflict with the Debian way of
working. But like SCons, Waf works very well with an installed Waf, the
project supplied Waf can be ignored. The Waf executable is not a binary
blob really, it is just an encoded source distribution which has to be
decoded. If the people had investigated properly this comment would just
not have been made. Actually the comments on the indicated issue explain
this very clearly. Sadly other comments wilfully misrepresent the
status quo.
Line 177: Line 244:
 * http{{{}}}://packages.qa.debian.org/''package'' where package is the name of the package in Debian. This page is a portal to all kinds of information. You can subscribe to Package Tracking System (PTS) to be mailed when there is a new release or when a user reports a new bug.  * http{{{}}}://packages.qa.debian.org/''package'' where package is the name of the package in Debian. This page is a portal to all kinds of information. You can [[https://www.debian.org/doc/manuals/developers-reference/resources.html#pts-commands|subscribe to Package Tracking System (PTS)]] to be mailed when there is a new release or when a user reports a new bug.
Line 205: Line 272:
CrossBuildPackagingGuidelines - thinks to keep in mind so that your software is able to be cross-compiled.
Line 210: Line 279:

= Language extensions in scripts =

Debian policy says in section [[https://www.debian.org/doc/debian-policy/ch-files.html#s-scripts|10.4 Scripts]]

  When scripts are installed into a directory in the system PATH, the script name should not include an extension such as .sh or .pl that denotes the scripting language currently used to implement it.

Here you can find good reasons for this:

 1. [[http://en.wikipedia.org/wiki/Filename_extension#Command_name_issues|Wikipedia about command names]]
  The use of a filename extension in a command name appears occasionally, usually as a side effect of the command having been implemented as a script (in Bourne shell, Python, etc.) and the interpreter name being suffixed to the command name, a practice common on systems like Windows and Mac OS X, which rely on globally set associations between filename extension and interpreter, but sharply deprecated in UNIX-derived systems like Linux and Apple's Mac OS X, where the interpreter is normally specified as a header in the script. ...
 1. [[http://www.talisman.org/~erlkonig/documents/commandname-extensions-considered-harmful|Commandname extensions considered harmful]]
  Commands should never have filename extensions. Rely on interpreter directives instead or some other paradigm that prevent the implementation from being exposed, or worse yet, lied about, within the very name of the command.
 1. [[http://lists.debian.org/debian-policy/2003/04/msg00031.html|one out of several good mails on Debian policy list]]

= Media types =

If you created a new file format, consider registering its [[WikiPedia:Internet_media_type|media type]] (formerly called ''MIME type'') to the IANA (http://www.iana.org/form/media-types). This will ensure that applications distributed in Debian and other operating systems will recognise this file type.
Line 220: Line 307:
''[[http://spot.livejournal.com/308370.html|How you know your Free or Open Source Software Project is doomed to FAIL]]'' ([[http://www.theopensourceway.org/wiki/How_to_tell_if_a_FLOSS_project_is_doomed_to_FAIL|wiki version]]) by Tom Callaway. See also the associated ''[[http://www.socallinuxexpo.org/scale9x/presentations/why-you-fail|This is why you FAIL]]'' talk by Tom at SCALE 2011. /!\ includes some advice that large successful projects like Linux, Qt, and GTK do not heed. ''[[http://spot.livejournal.com/308370.html|How you know your Free or Open Source Software Project is doomed to FAIL]]'' ([[http://www.theopensourceway.org/wiki/How_to_tell_if_a_FLOSS_project_is_doomed_to_FAIL|wiki version]]) by Tom Callaway. See also the associated talks ''[[http://www.socallinuxexpo.org/scale9x/presentations/why-you-fail|This is why you FAIL]]'' at SCALE 2011 and and ''[[http://www.oscon.com/open-source-2015/public/schedule/detail/41110|This is why you fail: The avoidable mistakes open source projects STILL make]]'' at OSCON 2015. /!\ includes some advice that large successful projects like Linux, Qt, and GTK do not heed.
Line 276: Line 363:

''[[http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002802|Ten Simple Rules for the Open Development of Scientific Software]]'' by Andreas Prlić and James B. Procter ([[http://lwn.net/Articles/530956/#Comments|LWN comments]]

''[[CrossBuildPackagingGuidelines|Guidelines for cross-build friendly packages]]''

''[[https://hacks.mozilla.org/2013/05/how-to-spread-the-word-about-your-code/|How to spread the word about your code]]'' by Peter Cooper and Robert Nyman

''[[http://blog.smartbear.com/open-source/how-to-turn-your-pile-of-code-into-an-open-source-project/|How to Turn Your Pile of Code into an Open Source Project]]'' by Andy Lester

''[[http://ebb.org/bkuhn/blog/2013/12/05/non-profit-home.html|Considerations on a non-profit home for your project]]'' by Bradley M. Kuhn

''[[http://blogs.gnome.org/bolsh/2014/04/17/choosing-a-license/|Choosing a license]]'' by Dave Neary

''[[http://lackof.org/taggart/hacking/FOSS-build/|FOSS Build and Install Common Practices]]'' by Matt Taggart

''[[https://fedoraproject.org/wiki/User:Duffy/FundamentalTheoremOfDevelopingFLOSS|The Fundamental Theorem of Developing FLOSS]]'' by Máirín Duffy ([[http://blog.funnelfiasco.com/?p=1746|followup]])

''[[http://yakking.branchable.com/posts/expectations/|Expectations of free software developers]]'' by Lars Wirzenius

''[[http://www.inventati.org/frx/essays/softfrdm/whatissource.html|What is source code?]] by Francesco Poli /!\ Possibly controversial

Introduction

This page is aimed at upstream developers who would like their software to be packaged for Debian (why would you want this?), or whose software is already packaged for Debian. It attempts to document what we do with your software, the resources we make available to you and what you can do to make the packaging process smoother. If you require any clarifications or have any questions about this document, please direct them to the debian-upstream mailing list. For further, more extensive advice on making your project more friendly to Debian and other Free Software distributions, please refer to the External advice section below.

Initial Packaging

Pristine Upstream Source

The source code archive should contain source files only, never any files that are automatically generated from other files. We recognize that it is fairly common to ship lexer and parser code or compiled documentation as part of the source package so administrators building from source do not need to install the build tools, but we need to rebuild all generated files to make sure that they can really be built from source, so please include methods to regenerate these files. For guidelines about source for artwork, audio etc, please take a look at the games advice in the External advice section above.

The "rebuild everything" rule is currently applied inconsistently. For example, most packages do not rebuild the files generated by autoconf and automake. This is mainly for historical reasons, since in the past these tools would often break during automatic builds.

Some projects include a rough /debian directory among source files to ease bleeding-edge package compilation and installation on debian (and derived) systems. While this is a good effort, it is better to leave it out of the final tarball as it can interfere with debian's own packaging effort. Keeping it only in your VCS repository is usually a much saner default if it lives in a specific packaging branch, which mimics what Debian package maintainers do using git-buildpackage. Though leaving the debian folder in the normal branch can also interfere if the package maintainer is using an upstream git packaging workflow (for example: git tag based git-buildpackage workflow).

No inclusion of third party code

Please do not include other code (like libraries) or data that are also shipped separately inside your source archive, or if you do, please make sure they can be reliably ignored. Instead of shipping third party libraries you should rather make sure your program will be link nicely against recent versions of these libraries. If a security issue is found in one of the bundled packages, it is far easier to rebuild one package than to scan the entire archive for all copies of this code and patch them individually (this happened for zlib, for example).

It is even worse if you maintain your private forks of third party libraries. This is not only troublesome for distributors but in the long run also to your own project. You should always make sure that the patches you might need for your specific application will be backported to the library upstream - that's simply how Free Software works.

To make sure your software will run with different versions of third party code it is way better to provide test cases you can run at any time to get reproducible results (which is also an additional profit for your own project).

Source only tarball

Please provide an archive of your source code as tar.{gz,bz2,lzma,xz} file. That's what our toolchain can currently work with.

Debian needs to make sure that it distributes the source code for all binaries in its archives. Debian achieves this by building all binaries itself from your source code.

If your software depends on other libraries, then Debian also needs to make sure that your software compiles and works with the version of these libraries available in Debian. Debian may compile your software against a different version of some library than you do. Therefore it's not of any help for Debian if you include convenience copies of these dependencies in your source tarball.

If you distribute precompiled binaries with your source code, then the Debian Maintainer has extra work to strip out these binaries and repackage your tarball. This is not an ideal situation. Debian would prefer to build the software packages it provides from the original tarballs you released.

Licenses

The DFSGLicenses page describes the licenses currently found in Debian.

  • Please do not write your own license text if you can at all avoid it. Depending on your wishes, the GPL, LGPL or a BSD-style license will most likely be appropriate, and it is far easier to tell whether something is allowed if we can look at past discussions of the same text.

  • There is usually no need for different types of files to have different licenses. It really complicates matters as all the license terms need to be followed at the same time, and if two are incompatible, we cannot distribute the package at all.
  • Please document copyright attribution in each file, and add statements placing the code under the chosen license to the copyright holders. Please place these statements at the top of the file, not in the middle or at the end. This is important for various reasons. Email addresses or company contacts for copyright holders should be listed either in the individual files, or in a file called "AUTHORS"

  • Include the full text of all licenses used in the top-level directory.
    • If more than one license is in use in the package, a file "COPYING" that lists files and their licenses, and individual files "GPL" "LGPL" with the unmodified license text would be ideal.

    • Packages with a single license can simplify this by using the full license text as the "COPYING" file.
  • Please keep copyright information up to date.
  • If your chosen license is GPL, please don't depend on libraries with incompatible licenses (such as OpenSSL)

Releases and Versions

Each time you reach a point in development of your software where you feel everything generally works and you've fixed a variety of bugs or implemented some interesting new features, please release a downloadable tarball with a version number. This doesn't necessarily mean that you have to keep supporting that version afterwards if you're short of resources, so long as you make that clear. But please don't develop only in a version control repository and with snapshots, or do releases only as tags. It makes it more difficult to grab a stable point that you think generally works. A distro needs to pull in this stable point rather than an automatic snapshot which inadvertently happened in the middle of some known transition or partial development.

Each time you make a new release, give it a new version number which is greater than the previous version number. (This sounds obvious, but it's amazing how often this doesn't happen.) If anything changes, please give the new release a new version number. Even if what changed was fixing an obvious mistake or accident and the previous version wasn't usable, always give the new release a new version number.

Version numbers are only useful if they increase every time something changes. There are an infinite number. You won't run out. It's far more useful to know that version numbers always increase and that everyone with version N has identical code than it is to hit psychologically-significant numbers or to set version numbers to meet past plans about what would go into which version.

Be bold about increasing version numbers. Don't get trapped in sub-sub-sub-versions. Don't be scared about crossing magic numbers. Call it v1.0 not v0.9.8.93. You will be moving on to v1.1 soon enough when problems are found.

Consider committing to long-term support of versions of your software in stable releases of major distros. If this is a problem, discuss it with the distros before they make a release.

Tarballs

If your build system is autotools please generate your tarballs by running make distcheck, which builds and verifies the tarballs. Other build systems will have standard targets to create source tarballs, please use them.

If you make tarballs of your releases, please sign them with your OpenPGP keys. This wiki provides further information on creating and signing GPG keys and finding somebody to sign your key. External advice is available for best practice OpenPGP/GPG key management.

Hardcoded Paths

Debian's Policy requires that Debian packages follow the Filesystem Hierarchy Standard for file locations. Debian packages are not installed into /usr/local. Thus Debian config files are found in /etc and manual pages in /usr/share/man. If your program normally installs to a different location please be sure to provide a mechanism to change the locations without patching the source. Use of environment variables for this purpose is recommended.

  • Support foo.d for configuration.

User home directories

When reading/writing cache, configuration, data or sockets for your software, please follow the XDG Base Directory Specification to determine the right paths. There are libraries available in Debian.

When accessing files relative to the user's home directory, first check the HOME environment variable (using getenv(3) or similar) and fall back on information from the password database (using getpwuid(3) or similar) when HOME is not present.

Dependencies

  • Document dependencies: license, version (range), where can it be retrieved from, what is it homepage
  • Document whether dependencies are optional
  • Avoid dependency versions not available in the latest release of major distros, unless that takes a lot of effort.
  • Avoid dependencies on unreleased versions (such as VCS snapshots)
  • Avoid dependencies on the patches applied in specific distributions or your own patches
  • If you need to patch a dependency please try to get your patch officially accepted by the dependency authors and document this process: When has this patch been sent to which email list or which issue tracker (with url)

Test suites

  • Have an automatic test suite to run during build time.
  • If possible, have a test suite to run against the installed software.
  • We will try to run it regularly and report the results at ?ci.debian.net.

Security

  • Be careful about security.
  • Have a well-publicised preferred method for reporting security issues and respond promptly to reports about security problems.
  • Request a CVE identifier for every security issue. A guide for requesting CVE identifiers can be found from Kurt's HOWTO and include the CVE identifier(s) in commit logs, NEWS and ?ChangeLog files and announcements.

  • If you fix a security issue don't try to hide the fixed problem. This only makes it harder for distros to patch issues. Evil attackers are still going to find out, which problem existed and on what versions.
  • When announcing security fixes, provide patches or links to commits for each version the issue was fixed in.
  • Either provide security fixes for old versions that are distributed in the stable releases of distros or provide support/help to distro security teams and package maintainers.
  • Contact the Debian security team in case of any questions.

Libraries

Assign a SONAME, track ABI compatibility and change the SONAME when ABI compatibility is broken. Avoid breaking ABI if it is possible. You can use the ABI Compliance Checker as a method of double-checking your own ABI compatibility tracking.

Use private libraries instead of public shared libraries if your library is only meant for use in your application.

Use 'hidden' as the default symbol visibility and only whitelist public parts of the API.

Bug tracking

  • Have and use an open bug tracker.
  • Have and use an open version control system.

Version Control System

Please provide read access to your version control system.

Git

Using Git as your VCS might be a very good choice. Git is also the most widely used VCS for packaging in Debian and mandatory for Fedora. If you're using Git then you make it easier for some Debian maintainers to submit patches and review changes between versions.

Debian needs to make sure that it really packages your original source code; source code repositories can be compromised. One easy way to do so is to rely on the tag signing capability of Git together with the web-of-trust. So please tag your releases with the following command. You can also sign every commit that enters your repository.

git tag -s $YOUR_VERSION

This wiki provides further information on creating and signing GPG keys and finding somebody to sign your key. External advice is available for best practice OpenPGP/GPG key management.

Your Build System

General Considerations

  • You're free to use any build system you like. Debian uses a Makefile (?debian/rules) to call your build system but that does not mean that you should use make too.

  • However there are a few experiences we've made with different build systems. Please see below for your particular system.
  • You should never assume that the top-level directory name is any particular value or matches any particular pattern.

Generated files

We need to be able to build everything from source. To ensure that build-dependencies are still working, we also like to rebuild everything from scratch periodically so it is best for generated files to not be available in the tarball. Those that are (such as generated build systems - autotools) need to be regenerated when their source changes or they are removed. We encourage maintainers of Debian packages to rebuild all generated files, including build systems.

Verbose builds

We need to record as much detail from the build log as possible so that we can debug build failures and other strange issues. Your build system may have a silent building option to hide the full commands (including arguments) used during building. If it does, we recommend that it be off by default or that one of the following commands enables verbose building.

./configure --disable-silent-rules
make V=1

No Downloads

You should never assume that Internet access is available during building. Debian is often installed from CDs or DVDs in places without connectivity, where those packages could not be rebuilt.

Out-of-Tree Builds

Please ensure that your software can be built from outside the source tree, both from in entirely different locations and from specific subdirectories of the source tree.

If you are using autotools then you can check this by running make distcheck. This will export a release tarball and run some dummy builds using it.

The reason for this is that if your software can be built in different ways, we need to build all of them in different directories. Sometimes we also need to be able to build your software for multiple architectures at the same time.

Cleaning the Tree

(if out-of-tree works, we can rm -rf the build tree, otherwise we need clean targets that remove anything that can be automatically rebuilt, including generated source)

Including a file in your distribution and then modifying it as part of the build is confusing and leads to spurious differences that appear to be Debian-local modifications unless the packager takes special care. Files of this type should instead be generated from a template with a different file name, the way that Autoconf handles config.h, generated from config.h.in, and the generated file removed in your clean or distclean target and not included in the distribution.

Installation

Please honor the DESTDIR environment variable or make variable and use its contents as a prefix in front of all installation paths. (Supporting changing prefix at make install time without changing any of the paths built into the software is also good practice, but not strictly necessary if DESTDIR is used.) Automake's make distcheck will verify that your package supports this, as well as out-of-tree builds and other best practices, so if you're using Automake, that's a nice check to do before a release.

After running make, running make install should not rebuild any files and in particular shouldn't encode any new paths. Otherwise, it's very hard to install the software in a temporary staging area without including the paths to the staging area in the final software (which in the case of RPATH can even be a security concern).

Assume that the directories into which you're installing files may not already exist and create them if necessary using mkdir -p (possibly via the $(MKDIR_P) macro provided by Autoconf and Automake), install -d, installdirs, or some similar technique.

Optimisation

You should never optimise for the CPU that is being used for the build by default. Debian buildd machines often support more CPU features than the baseline supported by individual Debian architectures, so if you optimise for the build machines then users will probably get crashes due to illegal instructions. If you want to build additional optimised versions of libraries, they should be installed in hardware capabilities directories documented in the ld.so(8) manual page.

Autoconf and Automake

If you use config.guess and config.sub, please make a habit of always copying the latest versions from ftp.gnu.org or some other source (such as /usr/share/misc on a Debian system with autotools-dev installed) into your package when you release. If you use autoreconf, it can do this for you automatically provided that you keep your local system version up to date. Most of your users may not care about the changes, but Debian often does, since Debian supports a wide variety of architectures.

Make

Some make variables are reserved to the user, and the Automake manual and the GNU coding standards advise never to use them for switches that are required for proper compilation of the package. When a Debian binary package is built, default environment variables are prepared by dpkg-buildflags (In dpkg 1.17.0: CFLAGS, CPPFLAGS, CXXFLAGS, FFLAGS, GCJFLAGS and LDFLAGS), to allow the build system to override the corresponding variables in the Makefile. We therefore strongly recommend to follow the above advice, and to make your makefiles use these variables (where relevant) in a way that our build system can override them.

SCons

Please don't use SCons. It is hard to use it correctly. For instance SCons is designed to ignore environment variables such as CFLAGS (unless your add code for this). It also does not support DESTDIR out of the box. As an upstream you have to explicitly add code for that (or Debian has to patch). Support for SONAMEs (library versioning) is also absent. The general observation is that many projects, that use SCons, do not have a working install target. Since projects work around these limitations individually there is no way to just use a SCons project in Debian, but more work is required to invoke it correctly.

The gentoo wiki has a detailed list of shortcomings.

If you choose to use SCons anyway, please ensure that the usual environment compiler variables (CC, CFLAGS, ...) and path variables (DESTDIR, BINDIR, LIBDIR, ...) are honoured. There is a recipe, that addresses some of these.

waf

Using waf as build system is discouraged. One of the reasons is the recommendation to ship a waf executable in every single package using it, instead of using a system wide one. Also note that just shipping the waf executable (which contains a binary blob) is considered to be not compliant with the Debian Free Software guidelines by the FTP Team. Please see #645190 and UnpackWaf for more details on the issue and how to avoid it, if you have to use waf.

waf discussion

TODO: please address this objection

It is true that waf pushes the "carry the build system with the project" line. In fact Gradle has done something along these lines as well. Indeed SCons supports this way of working. In a global context, it is a very good idea, even if it is conflict with the Debian way of working. But like SCons, Waf works very well with an installed Waf, the project supplied Waf can be ignored. The Waf executable is not a binary blob really, it is just an encoded source distribution which has to be decoded. If the people had investigated properly this comment would just not have been made. Actually the comments on the indicated issue explain this very clearly. Sadly other comments wilfully misrepresent the status quo.

Java

As it happens way too often: please do not ship any prebuilt class or jar files or any other generated files without accompanying source code and do not download such files during the build process. Please document each dependency and in particular: where can its source code be obtained.

Maven is a very popular build tool for Java code and it is easy to use for Debian packaging, too. It does provide a documentation of dependencies in the pom.xml files and we can point Maven to dependencies available in Debian and detain it from downloading anything.

Perl

Please use one of the standard Perl build systems, either Module::Build or ExtUtils::MakeMaker, and please try not to modify it in too many odd ways. That helps greatly in preparing packages.

Please remember that builds need to be done unattended and provide some mechanism to set defaults or bypass any prompts with known answers.

(we like stuff from CPAN, why?) (plugins => see above)

Here is a link to the debian-perl group's internal policy regarding perl packages in debian pkg-perl policy.

Python

(they have a standard way of packaging too) (plugins => see above)

  • please do not break API/ABI unless you really have to (it's very common among Python developers to break it in every second release)
  • please do not bundle local copy of 3rd party modules, we will remove it anyway
  • please do not depend on unreleased (sic!) or unstable versions of 3rd party libraries

OCaml

Please help us in supporting architectures for which the native code compiler (ocamlopt) is not available. Practically this mean to support in your build tool (makefile or corresponding) one of the following 2 behaviors:

  1. automatically detect whether ocamlopt is available or not, build native code objects/executables only if this is the case, install native code executables only if they have been built
  2. provide different make targets (or corresponding tool invocations) to build and install bytecode stuff vs native code stuff. Usual names for that are: all vs opt (to build bytecode/native code stuff) and install vs install-opt

(other languages with their own deployment tools)

After initial packaging

If your package has been packaged for Debian, thank you and congratulations. We hope that you will get new users. There are some resources that you may want to take advantage of:

  • http://packages.qa.debian.org/package where package is the name of the package in Debian. This page is a portal to all kinds of information. You can subscribe to Package Tracking System (PTS) to be mailed when there is a new release or when a user reports a new bug.

  • http://bugs.debian.org/package where package is the name of the package in Debian. This page lists all bugs that have been reported by Debian users. We would be delighted if you would help to solve them, propose patches or fix them upstream.

Note that subscription to PTS is recommended but optional. The Debian maintainer will forward you bugs that need your help or your interest.

  • Mailing lists: debian-devel and debian-project.

Branching

(why we maintain ancient versions in stable) (what to do if you think we shouldn't do this)

If you fix bugs, please try to have only the bug fix in the commit. This will let us more easily backport the fix to the version included in a Debian release.

Upgrades

(converting old system wide data) (converting old user data)

Downstream Patches

(we need your help in understanding your package) (please add unit tests even for functionality you believe is trivial so we can see if a patch breaks something)

  • If distros tell you something in your code makes life hard for them, seriously consider fixing it.
  • Accept improvements to build systems, so distros can avoid using workarounds for a long time.

Porting

You can view the build logs for packages at buildd.d.o and buildd.d-p.o. These can be useful for discovering build failures and compiler warnings that occur on less common architectures.

Be mindful about portability. Debian releases for a number of architectures and kernels.

GettingPorted - things to keep in mind when developing a program or packaging it so that it remains portable.

CrossBuildPackagingGuidelines - thinks to keep in mind so that your software is able to be cross-compiled.

Branding

Do not brand your software as part of any particular distribution, since that will need to be removed every time your software enters a new distribution.

Optionally provide build-time or run-time ways for distributors to add branding and other per-distribution settings to your software.

Language extensions in scripts

Debian policy says in section 10.4 Scripts

  • When scripts are installed into a directory in the system PATH, the script name should not include an extension such as .sh or .pl that denotes the scripting language currently used to implement it.

Here you can find good reasons for this:

  1. Wikipedia about command names

    • The use of a filename extension in a command name appears occasionally, usually as a side effect of the command having been implemented as a script (in Bourne shell, Python, etc.) and the interpreter name being suffixed to the command name, a practice common on systems like Windows and Mac OS X, which rely on globally set associations between filename extension and interpreter, but sharply deprecated in UNIX-derived systems like Linux and Apple's Mac OS X, where the interpreter is normally specified as a header in the script. ...
  2. Commandname extensions considered harmful

    • Commands should never have filename extensions. Rely on interpreter directives instead or some other paradigm that prevent the implementation from being exposed, or worse yet, lied about, within the very name of the command.
  3. one out of several good mails on Debian policy list

Media types

If you created a new file format, consider registering its media type (formerly called MIME type) to the IANA (http://www.iana.org/form/media-types). This will ensure that applications distributed in Debian and other operating systems will recognise this file type.

External advice

Here are some similar guides for upstream developers:

Ubuntu's Upstream Guide

"Distribution-friendly projects" by Diego Pettenò - part 1, part 2, part 3.

How you know your Free or Open Source Software Project is doomed to FAIL (wiki version) by Tom Callaway. See also the associated talks This is why you FAIL at SCALE 2011 and and This is why you fail: The avoidable mistakes open source projects STILL make at OSCON 2015. /!\ includes some advice that large successful projects like Linux, Qt, and GTK do not heed.

Packaging Unix software by Adam Sampson

Releasing FLOSS for Source Installation by David A. Wheeler.

The java packaging nightmare... by Vincent Fourmond

Upstream hints for Java developers by folks at FOSDEM 2010

The real problem with Java in Linux distros by Thierry Carrez (LWN comments)

How to be forkable and not get forked by Mel Chua

Free Software Project Management HOWTO by Benjamin Mako Hill

A blog series by François Marier:

Rules for distro-friendly packages by Enrico Weigelt

Releasing Free/Libre/Open Source Software (FLOSS) for Source Installation by David A. Wheeler

How to be a good upstream for games by Debian/Fedora/etc games teams

Writing a C library by David Zeuthen /!\ (includes some advice that can be considered controversial)

How to be a good upstream, a FOSDEM 2010 talk by Petteri Räty of Gentoo (there's also a video available at the FOSDEM 2010 distributions devroom video archive).

Physics Software Rant by Kevin B. McCarty

The cost of going it alone by Dave Neary (LWN comments)

Managing Copyright Information within a Free Software Project by the Software Freedom Law Center (LWN comments)

Community Distribution Patent Policy FAQ by the Software Freedom Law Center with input from Stefano Zacchiroli (LWN comments)

It isn't open source if it doesn't pass "The patch test" by Andy Oliver

Don't call it "open source" unless you mean it by Christian Heilmann (LWN comments)

The open source way by Red Hat

Open Source Community, Simplified by Max Kanat-Alexander

Producing Open Source Software by Karl Fogel

Ten Simple Rules for the Open Development of Scientific Software by Andreas Prlić and James B. Procter (LWN comments

Guidelines for cross-build friendly packages

How to spread the word about your code by Peter Cooper and Robert Nyman

How to Turn Your Pile of Code into an Open Source Project by Andy Lester

Considerations on a non-profit home for your project by Bradley M. Kuhn

Choosing a license by Dave Neary

FOSS Build and Install Common Practices by Matt Taggart

The Fundamental Theorem of Developing FLOSS by Máirín Duffy (followup)

Expectations of free software developers by Lars Wirzenius

What is source code? by Francesco Poli /!\ Possibly controversial