debian/watch

The file named 'watch' in the debian directory is used to check for newer versions of upstream software is available and to download it if necessary. The download itself will be performed with the uscan program from the devscripts package. It takes the path to the debian directory that uses the watch file as an argument or searches the directories underneath the current working directory.

Basically a watch file will have this format:

  version=3
  http://somesite.com/dir/filenamewithversion.tar.gz

To allow the version to remain unspecified, it is expressed as a wildcard using regular expression in Perl format:

  version=3
  http://somesite.com/dir/filename_(.+).tar.gz

The uscan program will then execute a "dir" command and check all the files in that directory for the highest version number.

uscan also supports ftp protocol. Unfortunately, when working with http, HTML pages do not always contain directory listing with files. The complete (or relative) path of the .tar.gz file to be downloaded will appear as a hyperlink within the web page. We hence need two types of information, the path to the page announcing the file and a regular expression to grep for the right link:

  version=3
  http://somesite.com/path link_from_href-1.0.tar.gz

"http://somesite.com/path" is the site from where you are downloading the source and "link_from_href.1.0.tar.gz" is obtained from the HTML source code (from the "<a href=" tags).

If http://somesite.com/path HTML code has "<a href=foo/program-1.0.tar.gz>" inside it, for example, you will use

  http://somesite.com/dir foo/program-(.+)\.tar\.gz

Watch files with errors are generally wrong on this second part.

Generally you want a slightly more flexible regex for the tarball name so that if upstream switches tarball naming schemes or compression formats you are covered:

  http://somesite.com/dir foo/program-(.+)\.(?:zip|tgz|tbz|txz|(?:tar\.(?:gz|bz2|xz)))

For FTP sites it changes a little, but it's basically the same thing (man uscan will help you).

But the nice part is to test it:

  uscan --no-download --verbose

If it's not working as expected, you can use debug to see what it's fetching and what it's (not) matching:

  uscan --no-download --verbose --debug

Common mistakes

Cryptographic signature verification

If your upstream provides cryptographic signatures for their packages in the same place that the packages are available for download, and you know what OpenPGP key or keys will be used to sign these packages, uscan can verify these signatures for you if you give it the pgpsigurlmangle option.

For example, for OpenSSH (which uses a .asc suffix for the package signatures), you'd place Damien Miller's public key ascii-armored in debian/upstream/signing-key.asc:

gpg --armor --export '59C2 118E D206 D927 E667  EBE3 D3E5 F56B 6D92 0D30' >> debian/upstream/signing-key.asc

and then update debian/watch to say:

version=3
opts=pgpsigurlmangle=s/$/.asc/ ftp://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-(.*)\.tar\.gz 

This lets you ensure that the package was not tampered with in transit, and that it came from the developer you expected it to come from. A valid signature does not mean that the contents of the package are somehow magically perfect and DFSG-free and policy compliant, of course!

Uncommon sites

If you have a site that has version numbers in some form but doesn't have hrefs containing them and the URL mangling capabilities of uscan are not enough, you can create a redirector. The Debian QA folks run one called fakeupstream.cgi (SVN) for lots of different upstream sites. If you want to add a new one, please submit a wishlist bug report assigned to qa.debian.org with or without a patch.

Common sites

Files hosted on various big project hosting sites can be specified with the URLs below. SourceForge has a special "redirector" URL (see more details in 'man uscan'). This allows the upstream URLs to change without the need to adapt watch files of affected packages.

SourceForge

  # qa.debian.org runs a redirector which allows a simpler form of URL
  # for SourceForge based projects. The format below will automatically
  # be rewritten to use the redirector.
  http://sf.net/audacity/audacity-src-(\d\S*)\.(?:zip|tgz|tbz|txz|(?:tar\.(?:gz|bz2|xz)))

See also: http://anonscm.debian.org/viewvc/qa/trunk/wml/watch/

GitHub

opts=filenamemangle=s/.+\/v?(\d\S*)\.tar\.gz/<project>-$1\.tar\.gz/ \
  https://github.com/<user>/<project>/tags .*/v?(\d\S*)\.tar\.gz

The 2nd regexp matches most often used tags like '0.1.2' or 'v0.1.2'. It may need to be adapted if upstream uses other version tags. You may also want to use releases instead of tags.

Google Code

http://code.google.com/p/<project>/downloads/list?can=1 .*/<project>-(\d\S*)\.(?:zip|tgz|tbz|txz|(?:tar\.(?:gz|bz2|xz)))

Gitorious

Some reports say this works:

version=3 
opts=filenamemangle=s/\S*download=//g \ 
http://qa.debian.org/cgi-bin/fakeupstream.cgi?upstream=gitorious/<user>/<project> \ 
.*=<project>(?:[_\-]v?|)(\d[^\s/]*)\.(?:tar\.xz|txz|tar\.bz2|tbz2|tar\.gz|tgz) 

Bitbucket

https://bitbucket.org/<user>/<project>/downloads .*/(\d\S*)\.tar\.gz

The 2nd regexp matches most often used tags like '0.1.2' or 'v0.1.2'. It may need to be adapted if upstream uses other version tags.


CategoryDebianDirectory