Differences between revisions 24 and 25
Revision 24 as of 2010-04-01 00:15:34
Size: 7691
Comment:
Revision 25 as of 2010-04-12 21:58:33
Size: 10685
Comment: adding some initial clarity for the upstream branch
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
This page is work in progress. It should describe the layout of pkg-boinc's Git repositories.
Line 4: Line 3:
 * master = Packaging BOINC with git =

This page is work in progress. Please contact us if you are keen to receive an update. Such external stimuli often help :)

== Overview ==

The general workflow for using the repository is to first have all upstream code in the collaboratively maintained remote branch 'upstream'. One then checks out the branch one wants to work with for performing the Debian packaging (maybe 'master') and merges all of upstream into it. Git-buildpackage will then perform the building.

== Branches ==

When packaging with git, both the Debian packaging sources ''and'' the upstream sources are stored in the archive. And when, like we are, one is using git-buildpackage, some default names for the different components should be respected to keep the argument list short.

 * upstream - the branch holding the upstream sources
 * master - the branch holding the Debian directory that shall be packaged, one must have checked this one out when invoking git-buildpackage

There are many more branches, which are somewhat historic, representing various attempts to be very current or very feature-rich. The server package never made it into Debian, sadly, mostly because none of us Debian package maintainers needs it and we are too busy to just play with it more.
Line 9: Line 24:
 * upstream
And there is this special branch here
Line 11: Line 27:
Once you have cloned the repository, as describe below, a list of branches available on the server is made visible with which is not really required if one is only interested in the very latest source tree (as we are for boinc) but which plays into our hands once we support many different versions. Let's ignore this for now.

After a first checkout of the repository (cloning in git terminology, described below) a list of branches available on the server is made visible with
Line 14: Line 32:
git branch -r}}}
----
 The programs that are referred to in the following paragraphs are contained in the Debian packages: [[http://packages.debian.org/git-core|git-core]] [[http://packages.debian.org/git-buildpackage|git-buildpackage]] [[http://packages.debian.org/git-svn|git-svn]] [[http://packages.debian.org/pristine-tar|pristine-tar]]
git branch -r
}}}

== Preparation ==

The programs that are referred to in the following paragraphs are contained in the Debian packages: [[http://packages.debian.org/git-core|git-core]] [[http://packages.debian.org/git-buildpackage|git-buildpackage]] [[http://packages.debian.org/pristine-tar|pristine-tar]] [[http://packages.debian.org/git-load-dirs|git-load-dirs]]. Please run
{{{
apt-get install git-buildpackage pristine-tar git-load-dirs
}}}
to perform the installation.

And to build the package, also install the following build-dependencies
{{{
apt-get install dh-buildinfo docbook2x automake1.9 autoconf libssl-dev libcurl4-openssl-dev freeglut3-dev libxmu-dev libxi-dev libwxgtk2.8-dev libsqlite3-dev
}}}. This list is not unlikely to be outdated, since this page is not expected to be kept in completel sync. But don't worry, you will receive respective instructions should anything be missing.
Line 22: Line 52:
Line 30: Line 59:
Line 47: Line 75:
== Upstream ==
Line 48: Line 78:

The repository may be considered special in a way since the remote upstream branch already offers a complete upstream source tree. For an initial rerun of the packaging, nothing needs to be done. If this is your first contact with this page and/or git, then just proceed directly with the section 'Build Package'.
Line 51: Line 83:
== Regenerating upstream tarballs == === Regenerating upstream tarballs ===
Line 91: Line 123:
== Building packages == == Build Packages ==

Make sure you are in the master branch, if unsure, run 'git checkout master'. You will notice the directory named 'debian'.
Line 93: Line 127:

{{{
git buildpackage
}}}

Hm. One could improve that just a bit by not requesting signatures

{{{
git buildpackage -uc -us
}}}

git-buildpackage will learn about the version to pack from the debian/changelog file. And since no .orig.tar.gz file is present, it will be recreated from the upstream branch. Once this was done, the remainder works as with dpkg-buildpackage, with no further involvement of the package maintainer.

=== Variants for the invocation of git buildpackage ===

Packaging BOINC with git

This page is work in progress. Please contact us if you are keen to receive an update. Such external stimuli often help :)

1. Overview

The general workflow for using the repository is to first have all upstream code in the collaboratively maintained remote branch 'upstream'. One then checks out the branch one wants to work with for performing the Debian packaging (maybe 'master') and merges all of upstream into it. Git-buildpackage will then perform the building.

2. Branches

When packaging with git, both the Debian packaging sources and the upstream sources are stored in the archive. And when, like we are, one is using git-buildpackage, some default names for the different components should be respected to keep the argument list short.

  • upstream - the branch holding the upstream sources
  • master - the branch holding the Debian directory that shall be packaged, one must have checked this one out when invoking git-buildpackage

There are many more branches, which are somewhat historic, representing various attempts to be very current or very feature-rich. The server package never made it into Debian, sadly, mostly because none of us Debian package maintainers needs it and we are too busy to just play with it more.

  • debian/experimental
  • debian/<codename>

  • debian/backports/<codename>

  • debian/fb/<feature>

And there is this special branch here

  • pristine-tar

which is not really required if one is only interested in the very latest source tree (as we are for boinc) but which plays into our hands once we support many different versions. Let's ignore this for now.

After a first checkout of the repository (cloning in git terminology, described below) a list of branches available on the server is made visible with

git branch -r

3. Preparation

The programs that are referred to in the following paragraphs are contained in the Debian packages: git-core git-buildpackage pristine-tar git-load-dirs. Please run

apt-get install git-buildpackage pristine-tar git-load-dirs

to perform the installation.

And to build the package, also install the following build-dependencies

apt-get install dh-buildinfo docbook2x automake1.9 autoconf libssl-dev libcurl4-openssl-dev freeglut3-dev libxmu-dev libxi-dev libwxgtk2.8-dev libsqlite3-dev

. This list is not unlikely to be outdated, since this page is not expected to be kept in completel sync. But don't worry, you will receive respective instructions should anything be missing.

4. Accessing repositories

Check out the source of package <package> with:

git clone ssh://<alioth-login>@git.debian.org/git/pkg-boinc/<package>.git
cd *boinc*

where <package> is one of 'boinc', 'boinc-app-seti' and 'kboincspy' as inspectable via http://git.debian.org.

If you do not have an Alioth account, you can check out the repositories read-only:

git clone git://git.debian.org/git/pkg-boinc/<package>.git
cd *boinc*

That directory includes all the source code already, and it even shows a debian directory, all information seems available:

$ ls                                 
api                     checkin_notes_samples  db           
apps                    client                 debian     
_autosetup              clientctrl             depcomp    
bolt_checkin_notes.txt  clientgui              doc
[...]

5. Upstream

However, we are interested in the differences between that "everything we need locally" master and the original upstream version. git is very good at determining such differences since difference between source trees is what source code management systems are all about. We just need another branch, upstream to keep the original. And when the original updates, then we just have another branch to merge our Debian-packaging-motivated changes against.

The repository may be considered special in a way since the remote upstream branch already offers a complete upstream source tree. For an initial rerun of the packaging, nothing needs to be done. If this is your first contact with this page and/or git, then just proceed directly with the section 'Build Package'.

5.1. Regenerating upstream tarballs

Upstream .orig.tar.gz tarballs can be regenerated with the pristine-tar tool directly from the repositories. To do so create a local pristine-tar branch from the remote origin/pristine-tar branch first:

git checkout -b pristine-tar origin/pristine-tar

As a clarification to those new to git, when cloning, the source of our clone is referred to as "origin". The "checkout" when given the "-b" option creates a new branch, which happens to have the same name as the branch on the remote git server. Normally, one would branch from the local master, i.e. one adds or changes a series of files relative to what is already available. The "origin/" prefix above informs git to branch from the remote side, i.e. a completely new environment is established that is independent from the previously checked out code:

$ ls
boinc_6.2.14.orig.tar.gz.delta  boinc_6.2.18.orig.tar.gz.id    boinc_6.6.10.orig.tar.gz.delta
boinc_6.2.14.orig.tar.gz.id     boinc_6.4.5.orig.tar.gz.delta  boinc_6.6.10.orig.tar.gz.id
boinc_6.2.18.orig.tar.gz.delta  boinc_6.4.5.orig.tar.gz.id

All other files disappear.

The new branch is now listed next to the previously cloned master branch:

git branch

Now to regenerate for example the boinc_6.2.18.orig.tar.gz tarball from the boinc repository run:

pristine-tar checkout boinc_6.2.18.orig.tar.gz

This pristine-tar is some magic on its own. The idea is to share upstream's resources and have them nicely integrated with the git and git-buildpackage functionalities. See this link http://www.eyrie.org/~eagle/notes/debian/git.html for a nice summary on what is going on.

To find out which tarballs can be regenerated with the tool pristine-tar, check out what files there are in the branch with the same name:

git ls-tree pristine-tar

To switch back to the master branch, we now check out our own local master branch:

git checkout master

There will now many file be appearing that will no longer be seen when you checkout the pristine-tar branch again. The concept is that git shows only those files that are assigned to the current branch, that have been already present when the branch was created, or that are assigned to no branch at all. Thus, once you compile the code, all the object files belong to no branch and will stay even when you go to the pristine-tar branch again. This is somewhat annoying at times.

6. Build Packages

Make sure you are in the master branch, if unsure, run 'git checkout master'. You will notice the directory named 'debian'. Building packages with git-buildpackage is now as easy as running:

git buildpackage

Hm. One could improve that just a bit by not requesting signatures

git buildpackage -uc -us

git-buildpackage will learn about the version to pack from the debian/changelog file. And since no .orig.tar.gz file is present, it will be recreated from the upstream branch. Once this was done, the remainder works as with dpkg-buildpackage, with no further involvement of the package maintainer.

6.1. Variants for the invocation of git buildpackage

git buildpackage --git-pristine-tar

or if you already have the .orig.tar.gz tarball in <path/to/tarballs/>:

git buildpackage --git-tarball-dir=<path/to/tarballs/>

7. Uploading local changes to the servers

Every new idea worked at under git is prepared as a separate branch. This may for instance be a bug that is fixed. The branches currently existing on the servers seem sufficient. If you are a not a regular contributor, and even if you are, please consider to discuss the preparation of new branches with Rene and/or possibly Thomas. The idea previously prepared is then merged with the local representation of the server's branches.

git checkout master
# checking out the server's branch. Make sure the names are identical
git checkout -b debian/topic/boinc-server origin/debian/topic/boinc-server
# now start following your idea
git branch myIdeaToTest
# perform some wild editing
git add <i>files edited</i>
git commit -m "some summary"
# you possibly want to have several rounds of editing/adding/commits
git checkout debian/topic/boinc-server
git merge -m "bringing idea to life" myIdeaToTest
# uploading to upstream branch with the same name as the local one
git push origin

If you don't need your branch any more then you can delete it with

git branch -D myIdeaToTest

This way, the changes are done to the respective branch, not to the master. Changes to the latter would impose their applicability across all possible versions of boinc. If the idea of yours that is represented by the branch would be applicable, then merge it from multiple branches, i.e. from debian/etch and debian/sarge, prior to deleting it.

8. Don't do this but interesting to know

This tutorial is still evolving, and I am afraid it stops here for now. The following tidbits stil need to find their place.

8.1. Tracking upstream development

With git-svn upstream development can be directly tracked in your Git repository. For example if you are in the boinc repository, run the following command to clone BOINC's trunk from upstream's Subversion repository into your Git repository:

git svn clone --trunk=trunk/boinc --prefix=git-svn/upstream/ http://boinc.berkeley.edu/svn/ .

This creates a new remote branch git-svn/upstream/trunk which contains BOINC's complete trunk.

9. Misc

It is at times confusing with all those many branches. To show that info directly on the shell, consider to add the following to the .profile or .bashrc (found at http://techblog.floorplanner.com/2008/12/14/working-with-git-branches/):

parse_git_branch() {
  git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/ \[\1\]/'
}
export PS1="\[\033[01;37m\]${debian_chroot:+($debian_chroot)}\u@\h:\w\[\033[00;35m\]$(parse_git_branch)\[\033[00m\] \$"