Contents

Using git for Debian Python Team packages

Using git for Debian Python Team packages

This article describes common workflow on maintaining Debian packages using Git within the Debian Python Team (DPT).

Scope of this article

This article is about how to incorporate Git tool into packaging under DPT policy. Readers should already be familiar with regular Debian packaging without Git. If not, please start with Guide for Debian Maintainers and Debian New Maintainers' Guide.

There are many advantages to team-based package development. The Debian Python Team (DPT) has very excellent, knowledgeable, and experienced Python and Debian developers and our goal is to bring the very best Python experience to Debian users and developers. We're here to help each other and provide consensus (if not 100% agreement) on best practices. Collaboration has many benefits, and one of the things that aids effective collaboration is common tools and workflows surrounding packaging, while still providing individual maintainers the freedom to adapt those practices to their preferences, and the peculiarities of the packages.

One of the perks of being in a team is that all of our packages are managed in a similar way, using similar tools and workflows. This is great because any team member can immediately get the source of a package to do an update or a bug fix, and they don't have to first try to figure out how the package is version controlled. Plus, we can have nice wiki documentation which help new team members how to do things, and almost anyone on IRC can immediately give you advice. We use a common set of VCS procedures to ensure consistency among all of our packages.

It is a team requirement that all packages be managed using the same version control system.

Policies

Repository: https://salsa.debian.org, under the python-team
Workflow: provided by gbp (Tools contained in git-buildpackage).
Patch management regime: The Debian Python team recommends gbp pq as interface to quilt patches.

Git Branch Names

As a team we are strongly recommending you use DEP-14 style branch names, specifically:

debian/main (formerly debian/master) - The Debianized upstream source directory. In other words, this contains upstream source and a debian/ packaging directory. It is a source-full, patches-unapplied checkout.
pristine-tar - Contains the standard pristine-tar deltas.
upstream - The un-Debianized upstream source. This is what you get when you unpack the upstream tarball. This could also be upstream/<version> if required.

If you use other branch names, please have a good reason (not just personal preference, remember - take one for the team!), and you MUST document the differences in debian/README.source. The default branch names will help the repo interoperate better with other tools.

gbp pq creates a patch branch when you want to directly edit upstream source files for export to quilt patches, however this is for local use only and should not be pushed to remote servers.

Git Tag Names

There seems to be several tag styles in common use. However, we recommend using '/' as a separator, yielding tags like debian/0.2.0-1.

Setup and configure the system

Install git-buildpackage (e.g. apt install git-buildpackage)
Setup your personal defaults for git in $HOME/.gitconfig and for gbp in $HOME/.gbp.conf.
Settings which affect how the team interoperates should live in <repo>/debian/gbp.conf. Anyone who checks out the branch will see the same settings.

Procedures

In the following are some common scenarios and use cases. Please update these pages or discuss with the team via the mailng list debian-python if you find a more efficient way of doing something. Also feel free to add more use cases. Your experience is valuable.

Creating new repositories

A new team-maintained git packaging repository can be created at the Debian GitLab web front-end (salsa.debian.org) on team packages page.

Alternatively, authorized users may do it with the following command:

$ salsa --group python-team/packages create_repo <srcpkgname>

This command will create a git repository with the URL of https://salsa.debian.org/python-team/packages/<srcpkgname>. The argument --group will add the repository to the DPT's own Salsa group python-team/packages. If you are new to packaging it is recommended not to use that group and remove the --group argument from the call above. You can start any work within your own namespace and move the package to any other group later if needed and appropriate.

Later when creating the debian/control file, these Vcs-* fields need to be added to its header:

Vcs-Git: https://salsa.debian.org/python-team/packages/<srcpkgname>.git
Vcs-Browser: https://salsa.debian.org/python-team/packages/<srcpkgname>

Creating a new package

First, create the repository on salsa.debian.org.

Next, you can either import an existing .dsc file or start from scratch. But be aware that Debian-python team does not support the git-dpm workflow. Here are the steps from the manpage that seem to work well:

$ uscan   # Download your package's upstream original tarball
$ mkdir srcpkgname_1.0
$ cd srcpkgname_1.0
$ git init
$ gbp import-orig ../srcpkgname_1.0.orig.tar.gz --debian-branch=debian/main --pristine-tar
$ git checkout -b debian/main

You'll now be left on the debian/main branch, i.e. the packaging branch. You'll also have upstream (raw tarball contents) and pristine-tar branch. See the discussion here.

Fill in the rest of your debian/ directory, then do this:

$ git add debian/*
$ debcommit  # assuming you have a d/changelog entry, otherwise git commit

Checking out an existing package

Packages can be checked out individually, or managed as a group with mr tool.

Handling individual package

To check out packages individually, you might be able just doing this:

$ debcheckout -a <srcpkgname>

Using debcheckout requires the package to be already present in Debian archive with correct Vcs-* fields.

If debcheckout does not work, you may invoke gbp clone directly as an alternative:

$ gbp clone git@salsa.debian.org:python-team/packages/<srcpkgname>.git

The command gbp clone is preferable to git clone because gbp clone ensures that you'll have all the remote tracking branches set up properly. You can still use git clone but you'll have to check out the pristine-tar and upstream branches yourself.

You might also want to add this handy little convenience to your ~/.gitconfig file:

[url "git@salsa.debian.org:python-team/packages/"]
    insteadof = dpt:

Which allows you to do this:

$ gbp clone dpt:srcpkgname.git

After debcheckout or gbp clone, make sure you cd into the source git directory before working on the actual packaging.

Handling all team packages together

When handling a lot of team packages, checking them out individually may be troublesome. When using the mr tool to manage packages as a group, it is configured to not checkout all the team packages by default. The user can easily manage the ones that are checkout out, collectively.

$ sudo apt install mr
$ git clone git@salsa.debian.org:python-team/tools/packages
$ cd packages
$ echo $PWD/.mrconfig >> ~/.mrtrust
$ ./checkout PACKAGENAME  # For each package you care about

If your username differs between your local machine and Salsa, then you can add this snippet to ~/.ssh/config:

Host salsa.debian.org
    User <YOUR_SALSA_USERNAME>

To update the .mr-config file, run this:

$ git pull
$ ./make-mrconfig

And if there are any, commit and push changes.

To update all your checked out packages, run this:

$ ./quick-update

Ensure up-to-date

Before doing anything on an existing repository, always always always ensure you have pulled in any changes:

$ gbp pull --redo-pq

If you always remember to do this, you reduce the risk of causing accidental git conflicts.

Note that the --redo-pq option will discard any local changes to your local patch queue, and replace with the patches from debian/patches.

Building

Now you can build your package using whatever tools you like, including debuild, dpkg-buildpackage, and git-buildpackage.

Uploading

Push your repo to salsa.debian.org:

$ git push --set-upstream git@salsa.debian.org:python-team/packages/<srcpkgname>.git : --tags

At this point, I like to cd to a different directory and do a gbp clone git@salsa.debian.org:python-team/packages/<srcpkgname>.git and then continue working from the <srcpkgname> directory. To prove that all the branches got pushed correctly, in this fresh clone, checkout the debian/master, pristine-tar, and upstream branches.

Tagging

Once you've built and uploaded your package, you should tag the release.

$ gbp buildpackage --git-tag-only
$ git push --tags

Alternately, if you want to sign your tags:

$ gbp buildpackage --git-tag-only --git-sign-tags
$ git push --tags

New upstream release

$/!\$ For this to work correctly, debian/gbp.conf should already have the correct value debian-branch, otherwise gbp import-orig will merge to the wrong branch.

You should import the patch queue first. After switch again the correct debian branch, or import-orig can do the wrong thing.

$ gbp pq import
$ gbp pq switch

Using a debian/watch file (recommended):

$ gbp import-orig --pristine-tar --uscan

Using a tarball file:

$ gbp import-orig --pristine-tar /path/to/new-upstream.tar.gz -u 0.2

Here 0.2 is the new upstream version number.

If the upstream tarball is already in the form packagename_version.orig.tar.gz (E.g. package_0.2.orig.tar.gz), then the -u option is not required.

$/!\$ Important notes:

Don't forget to update debian/changelog.

Rebase the patches:

$ gbp pq rebase
$ gbp pq export

You will need to push the debian/master branch and also the upstream and pristine-tar branches.

$ git push origin : --tags

Patching

Patching (i.e. adding quilt patches) is easy. You import the patches to a git patch queue, edit the patch queue, then export back again.

In general, to patch your package do this:

$ gbp pq import
Edit patches
$ gbp pq export
$ vim debian/changelog
$ git add debian/changelog
$ git add debian/patches/*
$ git commit

Cherry picking upstream commits

Sometimes you need to cherry pick upstream commits, e.g. to fix a bug in the Debian version when upstream hasn't yet released a new version. A convenient way to do that, if upstream is developed in git, is to add a remote and cherry pick from that. For example:

$ git remote add upstream_repo git://example.com/foo.git
$ git fetch upstream_repo
$ gbp pq import
$ git cherry-pick any_upstream_commit
$ gbp pq export
$ vim debian/changelog
$ git add debian/changelog
$ git add debian/patches/*
$ git commit

Apply patches from file

If need to apply one or more patches related to packaging received as file, for example on bugtracker, you can use git am to quickly apply them including author and commit message.

$ git am 0001-Debian-fix.patch

After, amend the commit done for needed changes, for example adding or fixing the closes tag (Closes: #-1) with the correct number of the bug.

Converting git-dpm to gbp pq

Previously DPT used git-dpm at the workflow. This section documents how to convert from git-dpm to gbp pq.

Ensure all branches up to date.
Unapply all patches
Delete debian/.git-dpm config file.
Set debian-branch value to debian/master in debian/gbp.conf
Commit to debian/master branch (use git branch -m master debian/master to rename the branch).
Update .git/config branch debian/master to refer to merge = refs/heads/debian/master.
Refresh the patches.
On Salsa, set the default branch to the new debian/master branch.

Sample debian/gbp.conf file:

[DEFAULT]
debian-branch=debian/master

Sample steps to do the above (upstream branch will not be automatically checkout out locally):

gbp pull
git read-tree --reset -u upstream
git reset -- debian
git checkout debian
git rm debian/.git-dpm
cat <<EOF > debian/gbp.conf
[DEFAULT]
debian-branch=debian/master
EOF
git add debian/gbp.conf
git commit -m "Convert from git-dpm to patches unapplied format"
git branch -m master debian/master
git push origin -u debian/master

To refresh the patches:

gbp pq import
gbp pq export
dch -m "Refresh patches after git-dpm to gbp pq conversion"
git add debian/patches/
git add debian/changelog
debcommit

To make the new branch the default and delete the old branch:

Go to https://salsa.debian.org/python-team/packages/$1/edit.
Expand "General settings".
Select the new default branch.
Go to https://salsa.debian.org/python-team/packages/$1/settings/repository.
Expand "Protected branches".
Fix those settings.
Go to https://salsa.debian.org/python-team/packages/$1/branches.
Delete the old master branch.

Packages no longer maintained by DPT

Packages repositories can be archived on Salsa by owners of the Python team namespace. If you want an obsolete repositories to be archived you will need to contact one of the owners and ask them to do it.

FAQ

Q: What if I don't want to use gbp pq?
A: Please be nice to your fellow teammates, and reconsider! We want team members to easily know what workflow to use, by common convention. In rare cases, if some deviation from the team default is required, you MUST have a good rationale, and both the reason and workflow MUST be documented in the README.source file.

Q: Source-full or source-less branches?
A: Source-full branches please! There are lots of good reasons for this, including the ability to easily diff between upstream versions, say to double check that nothing untoward or undocumented has been added in a new upstream version. E.g. git checkout upstream and git diff upstream/0.11.3 would for example give you an overview of what's new in upstream's 0.11.4.

We require team maintained packages to contain the full upstream source, and we don't believe it will be an overwhelming performance or disk-space burden to do so. It is much easier to hack on packages, including generating the quilt patches, when a checkout of the repo gives you the full source. Source-less branches defeat much of the benefit of managing packages in git.

Q: Should we honor pull requests and allow for mirrors on ?GitHub or GitLab?
A: We will still require all team packages to be available on salsa.debian.org. Please do not host your team maintained packaging branches anywhere else; e.g. ?GitHub is not appropriate for the packaging branches because it is not free software. GitLab does have a free software Community Edition and someday it might be interesting to run one of those for DPT or Debian-at-large, but for now please do not submit merge requests on gitlab.com.

More information and resources

The official git-buildpackage documentation. Most team packaging workflow is essentially built on top of it.

Debian Policy Manual

When there is to update Debian policy on package is good to check the Upgrading checklist

Debhelper tool suite

dh - debhelper command sequencer

When there is to update debhelper compat of package is good to check the upgrade checklist.

CategoryGit