I use the Bazaar VCS for tracking all my package work. This workflow should function well in any modern DVCS, though terminology differences are inevitable.
The proposed patch management workflow in this document has many similarities with git-pq. It differs in these points:
- Patch branches all originate in upstream or in the branches they depend on. git-pq puts all patches on one line.
- The commit objects of patch branches are conserved in a normal packaging workflow even though users only share the usual upstream, master, build branches and release tags.
- This workflow allows to make use of GIT's merge capabilities.
For the purpose of this document, the master branch is the branch where /debian/ lives. Others like to call this branch e.g. "debian".
Short overview
- Maintain the following branches:
upstream: pristine upstream release
trunk: packaging work for Debian
one additional feature-foo branch per patch
Develop each change against upstream on a feature-foo branch
Generate each patch as cd feature-foo/ && bzr diff ../upstream > ../feature-foo.patch
This is how Martin F. Krafft summarized this proposal:
- you develop your features on branches, but you do not push the branch heads;
- the feature branches get merged into an integration/build branch, which is pushed. This way, all contributors get the commits;
- as part of the build process, the feature branches are exported to a debian/patches series, and each patch file includes additional information, such as dependency data, and also the SHA-1 of the feature branch head at the time when the patch was made;
- at a later stage, when someone wants to edit a patch, they can create a branch off the SHA-1, merge the branch into the build branch and provide the updated patch (with updated SHA-1), or just provide an updated patch file and let the maintainer update the branch with an interdiff.
One of the points Martin disliked was the merge of the patches in the integration branch. I pointed out an alternative:
As a variation of the described workflow you can establish a special branch that holds references to all feature branch commits in its history. The content of this branch does not matter. A status command should warn you if the head of any feature branch is not in the history this special branch. Another command could create a new commit in this special branch with the parent pointing to all new heads.
Most basic workflow, without patches, sane upstream
upstream | | master | | | /* merge, new debian version (n commits) |/ | * | new upstream version | | | * debianize, debian release (n commits) | / |/ * import upstream
Upstream needs clean up to be dfsg compliant
upstream | | dfsg_clean | | | | master | | | | | * new debian version | | /| | |/ | | * | merge, clean up (n commits) | /| | |/ | | * | | import new unmodified upstream | | | | | * debianize, debian release | | / | |/ | * merge, clean up (n commits) | / |/ * import unmodified upstream
Patches, dfsg free upstream
principles, requirements, concepts
- Patches should not be merged to the master branch, because it's hard to "unmerge".
- We want to use GIT's merge capabilities and other features, therefor we'd like to work on patches in seperate branches per patch.
- It should be much easier to use then topgit
- We don't want to be forced to pull/push douzens of patch branches.
- We don't want to (permanently) pollute the branch namespace
Design
- When a new release has been merged to master, a new build branch is forked from master: build-$UPSTREAM_VERSION
- Patch branches are only merged in such build branches, never in the master branch
- Later we can delete the build branches but we still keep the commits since we tag each Debian release
- The primary storage of patches are patch files in quilt format in debian/patches in the master branch
- We provide tooling to convert patch branches into annotated quilt patches and recreate *identical* patch branches from quilt files.
We add extra headers in [http://dep.debian.net/deps/dep3/ dep-3] format to the quilt files in debian/patches:
- git-commit: SHA-1 value of the HEAD of the patch branch
- git-base-commit: SHA-1 value of the base commit of the patch branch (only necessary if git-base-dependencies is not empty)
- git-dependencies: list of branch names of dependencies of this patch branch
Workflow
Creating patches:
- Checkout patch branch from upstream (or from the branch it depends on).
- Hack, commit
- Create an annotated quilt patch from the patch branch
- repeat the above for other patch branches
- merge the patch branches into the build branch
- Once the patches are merged in the build branch, the patch branches can be deleted.
Editing patches:
- Recreate the patch branches with informations from the quilt annotations
- Hack, commit
- update quilt files
- merge updated patch branches into build branch
Update upstream:
- Import upstream, merge into master
- recreate the patch branches
- either merge upstream changes into each patch branch or rebase patch branches
- hack on patch branches
- update quilt files
- checkout a new build branch from master
- merge updated patch branches into build branch
Tooling
The outlined workflow could be done manually without new tools. It's however only practical, if additional tools are provided. This section describes the necessary tools to be implemented.
environment variables (names can change):
- UPSTREAM_BRANCH - the name of the upstream branch, default "upstream", may need to be changed to dfsg_clean
- DEBIAN_BRANCH - the branch containing the debianization, most importantly the debian/patches folder
create quilt file
options:
- dependency branches [optionally, defaults to empty=UPSTREAM_BRANCH]
- base [optionally, defaults to SHA-1 of UPSTREAM_BRANCH]
- head [optionally, defaults to SHA-1 of current HEAD]
- name [optionally, defaults to current HEAD name]
- target [optionally, defaults to debian/patches/$BRANCH_NAME]
Creates a quilt patch file from a patch branch or updates an existing one, inheriting options from an exisiting quilt file.
Additionally there should be a wrapper command to invoke this command over all patches listed in debian/patches to update the quilt files.
checkout branch from quilt file
options:
- branch name (translates to the quilt file's name)
- checkout [boolean, defaults to true]
Creates a branch with the given name pointing to the commit specified in the git-commit line of the quilt file. Checks out the branch if checkout option is true.
Additionally there should be a wrapper command to invoke this command over all patches listed in debian/patches.
merge patch branch(es)
options:
- target branch
- name [optionally, n-times, names of branches to merge]
Merges all patch branches into the target branches. Creates the target branch if it doesn't exist yet.
status
options:
- build branch [optionally]
Gives the following informations:
- list of patches (according to files in debian/patches)
- which patches needs update (because dependency or upstream was updated)
- which patches don't have their commit objects in the object database
- which patches have not yet been merged in the build branch
Extensions
Detect cherry-picked patches available in upstream
Requirement: Upstream uses GIT and you track upstream in your repository Use case: You cherry picked a commit from upstream which was not yet available in the packaged version. When you later package an upstream version that contains the cherry-picked commit, the tools should note that and drop the patch.
Implementation: An additional dep-3 header field ( git-upstream-cherry-picked ) could contain a commit sha-1 (or a range commit range). It's then possible to detect, whether the commit is available in the current worked on upstream version.
Detect patches applied upstream via empty diff
When the diff between upstream and a patch branch becomes empty after a diff, then this means, that a patch has been applied upstream and the patch can be dropped.
TODO
- How to ask GIT for the first common commit of two branches?
- function needed to check, whether a commit/branch is in the history of another commit.
links
- man git-quiltimport
quilt headers http://dep.debian.net/deps/dep3/
Scratchpad
- some of the options dpkg-source runs patch with:
- --unified (Interpret the patch file as a unified context diff)
- --remove-empty-files (Remove output files that are empty after the patches have been applied. Normally this option is unnecessary, since patch can examine the time stamps on the header to determine whether a file should exist after patching. However, if the input is not a context diff or if patch is conforming to POSIX, patch does not remove empty patched files unless this option is given. When patch removes a file, it also attempts to remove any empty ancestor directories.
Change log
* 2011-08-16 Cribbed the bulk of this page from ThomasKoch/GitPackagingWorkflow and edited for my workflow.