Differences between revisions 44 and 47 (spanning 3 versions)
Revision 44 as of 2008-05-19 18:08:42
Size: 23816
Editor: ?AndresMejia
Comment:
Revision 47 as of 2008-05-21 07:44:14
Size: 24361
Editor: ?AndresMejia
Comment: include how to tag a release only without rebuilding
Deletions are marked like this. Additions are marked like this.
Line 160: Line 160:
To view a summary of the repository on your favourite web browser, go to the following address. To view a summary of the repository on your favorite web browser, go to the following address.
Line 193: Line 193:
}}}

You can also tag a release without rebuilding by supplying a different build command using the {{{--git-builder}}} option. In this case, we'll just perform a cleanup.
{{{
$ git-buildpackage --git-tag --pristine-tar --git-builder='fakeroot debian/rules clean'
Line 256: Line 261:
Now merge the branches and tags from svn into the master branch.
{{{
$ git-merge \
      $(find .git/refs/remotes -type f ! -regex '.*trunk$' ! -regex '.*trunk@[0-9]*$' \
      ! -regex '.*\/alioth\/.*' -print0 | \
      xargs -0 -r echo | sed 's|\.git/refs/||g')
}}}

?TableOfContents

The Debian Games Team currently uses either git or svn to help in maintaining packages. This page will help you get started in using either git or svn repositories.

Scheme

Most of the packages are independent from each other. In svn, a [trunk,tags,branches}/<package> scheme is used with only the debian directory stored. The .orig.tar.gz files in such a case are stored in /var/lib/gforge/chroot/home/groups/pkg-games/htdocs/tarballs in the alioth servers. They can be accessed online at [http://pkg-games.alioth.debian.org/tarballs]. In git, each package has it's own repository. The contents of the orig tarball are stored directly within each package's repository, inside the 'upstream' branch. The pristine-tar program can be used to regenerate an orig tarball from the upstream branch.

Setting up

The first thing to do is get an account on alioth.debian.org. Visit the following link.

The next step is to join the team. Visit the Alioth page for the team and click on "request to join".

Next, you will need an ssh key for each computer you plan on working on packages with. If you don't have a key on a particular system, create one.

$ ssh-keygen

This should have created an RSA key for SSH protocol 2 by default. Upload the contents of your public keys (~/.ssh/id_rsa.pub) to your account in Alioth. To do this, copy the contents of each public key for your machines and paste them at the following link.

Next, create an SSH configuration file (~/.ssh/config) for your user profile to work with the alioth servers. Here's how to do the following in a terminal session.

$ cat >>~/.ssh/config <<EOF
Host svn.debian.org
        User <username>

Host git.debian.org
        User <username>

Host alioth.debian.org
        User <username>
EOF

Now you should be able to logon from each of your machines by doing the following.

$ ssh svn.debian.org
$ ssh git.debian.org
$ ssh alioth.debian.org

Note: There's usually a one hour delay when uploading your ssh public keys. During this delay, you can still login with your password.

Git

This section describes how to create a git repository for a package as well as convert from the svn layout.

Environment variables

These are some environment variables that git uses when you commit changes. It's best to set these environment variables so that git doesn't rely on your system's username or hostname. Place this snippet in your ~/.bashrc configuration file.

# Git variables
export GIT_AUTHOR_NAME='<your name>'
export GIT_AUTHOR_EMAIL='<your email>'
export GIT_COMMITTER_NAME='<your name>'
export GIT_COMMITTER_EMAIL='<your email>'

Creating a git repository

These steps involve creating a brand new git repository for your package. If you are converting from svn, please see the "Converting from svn repository" section below.

Here's a simple way to create a git repository using the supplied setup-repository script in the /git/pkg-games directory on the alioth servers.

$ ssh <username>@git.debian.org
$ cd /git/pkg-games
$ ./setup-repository <package> '<package description>'

If you want to manually create the repository, the rest of this section describes how to do so.

$ ssh <username>@git.debian.org
$ cd /git/pkg-games
$ mkdir <package>.git
$ cd <package>.git
$ git --bare init --shared

Now edit the 'description' file in the directory you just created to one that is appropriate for the package.

$ echo "Packaging for <package>" >description

Let's also enable the use of 'hooks/post-update'.

$ chmod a+x hooks/post-update

The following will enable the games team to receive commit mails and irc notifications about commits. Please do the following.

$ git-config --add hooks.mailinglist "pkg-games-commits@lists.alioth.debian.org"
$ git-config --add hooks.cia-project debian-pkg-games
$ cat >hooks/post-receive <<END
#!/bin/sh
exec /usr/local/bin/git-commit-notice
END
$ chmod a+x hooks/post-receive

Please see the documentation about mr below, and add the new repository to list of games team repositories in the .mrconfig file.

Uploading a package to the repository

Now let's import the package into a git repository. This is easy using 'git-import-dsc' from the 'git-buildpackage' package. This is done from your local machine of course.

$ git-import-dsc --pristine-tar <package>.dsc

(The --pristine-tar option is optional; if you have pristine-tar installed this will store enough information in the git repository to reproduce the pristine upstream tarball later.)

A directory with the name of the source package should have been made. Change into that directory.

$ cd <package>

Ensure the tags that are created reflect if a package has been released. git-buildpackage creates tags for the debian branch and the upstream branch. If the tag created for the debian branch reflects an unreleased version, you'll have to delete that "debian" tag.

$ git-tag -l
$ git-tag -d 'debian/<unreleased version>'

Finally, push the repository up to the repository on alioth.

$ git-remote add alioth git+ssh://<username>@git.debian.org/git/pkg-games/<package>.git
$ git-push alioth master
$ git-push alioth upstream
$ git-push alioth pristine-tar
$ git-push alioth --tags

(Again the pristine-tar step is optional.)

Accessing a repository

To get a repository, do the following.

$ git-clone git+ssh://<username>@git.debian.org/git/pkg-games/<package>.git

You can also enjoy read-only access by using the following.

$ git-clone git://git.debian.org/git/pkg-games/<package>.git
$ git-clone http://git.debian.org/git/pkg-games/<package>.git

It is generally a good idea to track certain branches that have been pushed (namely 'upstream' and 'pristine-tar'). To track them, do the following.

$ git-checkout -b upstream origin/upstream
$ git-checkout -b pristine-tar origin/pristine-tar #Do this if a pristine-tar branch was pushed

If you want to track more, see what branches are available.

$ git-branch -r

You might see branches labelled 'tags/<some_version>'. These are really the "remotes" objects of the tags that were imported into the git repository when it was converted from svn. Although you can track these as well, it's better if you do a git-checkout of the specific tags instead as follows.

$ git-checkout remotes/tags/<some_tag>

See the section "Converting from svn repository" below for more details.

To view a summary of the repository on your favorite web browser, go to the following address.

http://git.debian.org/?p=pkg-games/<package>.git

Committing changes

To commit a change, first commit into your local repository.

$ git-commit -m "<some comment>"

You could also do a commit of all files that were changed using the '-a' option.

$ git-commit -a -m "<some comment>"

Push your changes.

$ git-push #If you cloned the repository
$ git-push alioth #If you created the repository using the above example

Updating

To keep updated simply do the following

$ git-pull #If you cloned the repository
$ git-pull alioth #If you created the repository using the above exmple

Tagging

To tag a release, you could build it using 'git-buildpackage' with the '--git-tag' option.

$ git-buildpackage --git-tag --pristine-tar

You can also tag a release without rebuilding by supplying a different build command using the --git-builder option. In this case, we'll just perform a cleanup.

$ git-buildpackage --git-tag --pristine-tar --git-builder='fakeroot debian/rules clean'

(As usual, the --pristine-tar bit is optional, but it will allow git-buildpackage to extract the pristine .orig.tar.gz.)

You could also manually tag the package. Ensure that you are in the correct branch (should be 'master').

$ git-checkout master
$ git-tag debian/<revision> -m "Debian release <revision>" #Please use this format

To tag an earlier revision, you'll need to find out it's commit object, then tag that object.

$ git-rev-list --pretty --since="1 month" --all #'1 month' is just an example of course
$ git-tag debian/<revision> <commit object> -m "Debian release <revision>"

Finally, push the tags

$ git-push --tags

Using pristine-tar

You can manually use the pristine-tar command to create a pristine-tar branch for generating the orig tarball later. This is useful in case you ommitted using the --pristine-tar option when running the git-buildpackage tools.

To create a pristine-tar branch, simply do the following.

pristine-tar commit <path_to_orig_tarball> upstream/<package_version>

This assumes you already imported the orig tarball to the git repository. upstream/<package_version> is the tag that was created when the orig tarball was imported using git-import-orig. Make sure to specify the tag object.

Then push the new 'pristine-tar' branch.

git-push origin pristine-tar #If you cloned the repository
git-push alioth pristine-tar #If you created the repository

Converting from svn repository

To convert from the svn repository, first setup the git repository on the alioth servers as described above. Afterwards, do the following on your local machine.

$ mkdir <package>
$ cd <package>
$ git-svn init --no-metadata \
    --trunk svn://svn.debian.org/svn/pkg-games/packages/trunk/<package> \
    --branches svn://svn.debian.org/svn/pkg-games/packages/branches/<package> \
    --tags svn://svn.debian.org/svn/pkg-games/packages/tags/<package>
$ git-svn fetch

Notes:

  • Doing git-svn fetch will take some time depending on how many commits were done in the svn repository.

  • If the svn repository doesn't contain a branches or tags section for the package, then you can omit the --branches or --tags options.

After fetching from the svn repository, it's now time to import the contents of the orig source.

$ git-symbolic-ref HEAD refs/heads/upstream
$ git-rm --cached -r .
$ git-commit --allow-empty -m 'initial upstream branch'
$ git-checkout -f master
$ git-merge upstream
$ git-import-orig --pristine-tar --no-dch <path_to_orig_tarball>

Note: Using the --pristine-tar option is optional.

Now merge the branches and tags from svn into the master branch.

$ git-merge \
      $(find .git/refs/remotes -type f ! -regex '.*trunk$' ! -regex '.*trunk@[0-9]*$' \
      ! -regex '.*\/alioth\/.*' -print0 | \
      xargs -0 -r echo | sed 's|\.git/refs/||g')

After this step, push the contents of your local repository.

$ git-remote add alioth git+ssh://<username>@git.debian.org/git/pkg-games/<package>.git
$ git-push alioth master
$ git-push alioth upstream
$ git-push alioth pristine-tar
$ git-push alioth --tags

Note: Using a 'pristine-tar' branch is created only if you used the --pristine-tar option above.

This step here is a way to push the tags and branches that were imported from svn (an explanation about this is below). Do this to push all tags and branches that were imported.

$ for REMOTES in `find .git/refs/remotes -type f ! -regex '.*trunk$' \
      ! -regex '.*trunk@[0-9]*$' ! -regex '.*\/alioth\/.*'`; \
      do git-push alioth $(echo $REMOTES | sed 's|\.git/refs/||'); done

Once you have pushed all changes to the shared repository, move the packaging directory from trunk in svn so that no one continues to work on it there.

First create a directory under branches for you package if one doesn't exist already.

$ svn mkdir svn+ssh://svn.debian.org/svn/pkg-games/packages/branches/<package> -m "Creating branches section for <package>"

Finally, move the packaging directory in trunk to the branches section and leave an appropriate message.

$ svn mv svn+ssh://svn.debian.org/svn/pkg-games/packages/trunk/<package> \
      svn+ssh://svn.debian.org/svn/pkg-games/packages/branches/<package>/<package>-svn-save \
      -m "Moving <package> from trunk to branch. Please use git repository instead."

Cloning and pulling

Anyone pulling from a converted repository will have two steps to perform. The first step is the usual cloning.

$ git-clone git+ssh://<username>@git.debian.org/git/pkg-games/<package>.git

And then the next step involves pulling the "remotes" branches and tags that were pushed.

$ cd <package>
$ git-pull origin +refs/remotes/*:refs/remotes/*

Note about tags and branches

Tags and branches are preserved using what's called "git remotes". This means that there are no real git tags and branches visible, but that the SVN trunk, tags and branches are tracked as separate repositories. Downside is that you need to switch to to the remotes individually before you can update them.

All tags and branches are listed under .git/refs/remotes/ and can be checked out, but it is somewhat confusing as all old schemes for using tags and branches are there too...

So to checkout a branch do the following.

$ git-checkout remotes/<some_branch>

And for tags.

$ git-checkout remotes/tags/<some_tag>

mr

The mr command can be used to checkout, update, and commit to multiple repositories. In the pkg-games svn repository, is a packages/trunk/.mrconfig file that configures mr for all the repositories used by the games team.

New git repositories should be added to this .mrconfig file.

To use it, install the mr package, and create a ~/.mrconfig file letting mr know about the pkg-games svn repository. In this example, it will be checked out to ~/src/packages/games, but you can modify to suit. If you already have a pkg-games svn checkout, you can change the directory to point to that.

[src/packages/games]
chain = true
checkout = svn co svn+ssh://svn.debian.org/svn/pkg-games/packages/trunk games

Then run "mr checkout", which will checkout the svn repository into ~/src/packages/games.

A second "mr checkout" (necessary because of bug #447553) will checkout the additional git repositories, into subdirectories of ~/src/packages/games/.

Now you can use mr to act on all configured repositories at once.

Update all repositories:

% mr update

Get status of all checkouts:

% mr status

List all repositories that mr knows about:

% mr list

Or even commit changes in each repository:

% mr commit

If you've added a new git repository in ~/src/packages/games/<foo> and are in that directory, you can have mr automatically add it to the pkg-games .mrconfig file by running "mr register". (Don't forget to commit the file.)

SVN

The SVN repository may be browsed online using a browser via the following link:

[http://svn.debian.org/wsvn/pkg-games/packages/trunk/?rev=0&sc=0].

The base URI of the repository for authorised SVN access is

svn+ssh://svn.debian.org/svn/pkg-games/

To import your package to SVN

    apt-get install svn-buildpackage

    # Import only the files that are touched
    # by the .diff.gz
    svn-inject -o -l 2 <package_dsc_file>.dsc \
    svn+ssh://svn.debian.org/svn/pkg-games/packages/

NOTE: The -l 2 option for svn-inject specifies layout two. You must have svn-buildpackage (>= 0.6.16) to use this option. Another thing to note is [http://bugs.debian.org/411666 Bug 411666] - /usr/bin/svn-inject: initial checkout fails. If your package doesn't exist in the SVN, this bug may affect you. The best thing to do is to use svn-buildpackage (>= 0.6.22).

In very seldom cases (like dead upstream) you would probably want the full source in svn instead just of the contents of the .diff.gz. In that case, remove the -o parameter from the command.

Uploading the Source Tarball

scp <package_name_and_version>.orig.tar.gz \
alioth.debian.org:/var/lib/gforge/chroot/home/groups/pkg-games/htdocs/tarballs/
  • Next, set the property svn-bp:origUrl of the debian folder of the package to point to the corresponding orig tarball

svn co \
svn+ssh://svn.debian.org/svn/pkg-games/packages/trunk/<package_name>
cd <package_name>
svn propset svn-bp:origUrl \
"http://pkg-games.alioth.debian.org/tarballs/<package_name_and_version>.orig.tar.gz" debian
svn ci -N debian -m "origUrl property set for <package_name_and_version>"

To Checkout A Project

    svn co \
    svn+ssh://svn.debian.org/svn/pkg-games/packages/trunk/"packageName"

To build

    # Place the .orig.tar.gz in ../tarballs/

    # Build the package
    svn-buildpackage -rfakeroot

    # or, if you want to make it the build before uploading, and
    # have svn-buildpackage make a svn tag for you
    svn-buildpackage --svn-tag -rfakeroot

    # if you have local changes in your working copy and want to
    # run a test-build, use the `--svn-ignore' switch like so:
    svn-buildpackage --svn-ignore -rfakeroot

    # example for freecol:
    $ svn proplist -v debian
    Properties on 'debian':
      svn-bp:origUrl : http://pkg-games.alioth.debian.org/tarballs/freecol_0.7.2.orig.tar.gz
      mergeWithUpstream : 1

To build using pbuilder

pbuilder is a great tool to check that your package will build in a basic Debian system with the proper build-dependencies. It's also useful in avoiding having to install a package's build-dependencies on your system.

First, you will have to setup pbuilder on your system. The HOWTO at https://wiki.ubuntu.com/PbuilderHowto provides a good guide and some good tips in setting up pbuilder.

Afterwards, it should be just a matter of using the --svn-builder option. For example, to use pdebuild and ignore all .svn directories, use:

svn-buildpackage --svn-dont-clean --svn-builder='pdebuild --debbuildopts "-i\.svn/ -I.svn"'

Alternatively, you can use a simple script as svn-builder. Here's a script taken from the examples from the git-buildpackage package that's been modified to work in the case of svn-buildpackage.

# 
# pbuilder helper from git-buildpackage
# use this as "svn-builder" in ~/.svn-buildpackage.conf
set -e

# pass all options to dpkg-buildpackage:
pdebuild $PBUILDER_OPTS --debbuildopts "-i\.svn/ -I.svn $*"

Create a file called svn-pdebuild under /usr/local/bin and add the contents of the above script into the file. Then you can just run:

svn-buildpackage --svn-builder=svn-pdebuild --svn-dont-clean

You can also specify svn-builder using a configuration file at ~/.svn-buildpackage.conf. Create a file at ~/.svn-buildpackage.conf and add this snippet.

svn-builder=svn-pdebuild
svn-dont-clean

Then just run:

svn-buildpackage

Finally, if you need to specify more options for pdebuild's --debbuildopts option, just add them in like you would normally do for --debbuildopts. Here's an example that works using the method that uses ~/.svn-buildpackage.conf.

svn-buildpackage -sa -us -uc

svn-buildpackage will pass any options it doesn't recognize onto whatever command was specified as svn-builder. If you need to specify options for pbuilder, assign each option to PBUILDER_OPTS.

NOTE: The svn-dont-clean option is used since the Debian Games team SVN won't contain upstream source files.

To release

In order to have a lower threshold for someone to step in to help in for a game, some guide lines should be followed when working with the games maintained in our repository.

Thus, here are a few simple rules that should make our lives a little better.

  • when a package version it is not released, the topmost entry in the debian/changelog file should contain the string "UNRELEASED" instead of unstable or other valid suite name

    # example: boson-base changelog between the released version 0.11-2 and the to be relased 0.11-3

    $ head -n1 boson-base/trunk/debian/changelog
boson-base (0.11-3) UNRELEASED; urgency=low

This can be acomplished if svn-buildpackage is used when handling releases.

Before releasing, please check that your package is lintian clean. Automatic checks are done on our SVN tree by the ["Games/Autobuilder"].

Due to [http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=433536 a bug] in svn-buildpackage (which should be fixed in release 0.6.22), the "tags" directory was not created for packages injected using layout 2, the layout used by our team. If you are using an older version of svn-buildpackage or not at all, please make sure that the tags directory for your package exists under the "tags" directory in SVN:

svn mkdir svn+ssh://svn.debian.org/svn/pkg-games/packages/tags/<package>

Then you will be able to tag a release.

    # make a final build for a package
    $ svn-buildpackage --svn-tag

If you're package is accepted into the Debian archive already, you can save some time by just tagging the release.

svn-buildpackage --svn-tag-only --svn-noautodch

The --svn-noautodch will prevent the changelog from being tagged UNRELEASED.

Possible problems

git-import-orig dch error

When using git-import-orig, there is a chance you'll receive an error similar to this one.

dch: fatal error at line 963:
New version specified (1.4.8.dfsg1-1) is less than
the current version number (1.4.8.dfsg1-1)!  Use -b to force.
dch returned 25
Dch failed.
Import of ../../build-area/ogre_1.4.8.dfsg1.orig.tar.gz failed

This will happen if you omit the --no-dch option while running git-import-orig and the version in the changelog is at the version of the orig tarball that's being imported.

git-import-orig pristine-tar error

It's possible you may receive an error such as this.

pristine-tar: more than one ref matches "upstream":
5f4736c83ea06e9b479a35f25d997b73713306b9 refs/heads/upstream
a8c20d823bc28bb8f6d127e301390bed06e49169 refs/remotes/origin/upstream
/usr/bin/pristine-tar returned 255
Couldn't run '/usr/bin/pristine-tar'

This is caused by git-import-orig (or any of the git-buildpackage tools) supplying the parameter "upstream" to use when running pristine-tar commit instead of using the default refs/heads/upstream or supplying a tag object instead and you cloned a git repository. You will have to omit the --pristine-tar option from the git-buildpackage tools and use pristine-tar manually. See [http://bugs.debian.org/481806].

General Rules

  • If a package your working on is for a particular version that has not been uploaded to the archive, you should either append ~unrelease0 at the end of the version, or use 'UNRELEASED' as the suite in the changelog.

  • packages/ in the svn repo is for free games only. For games which should go to non-free, please use non-free/package/.
  • the SVN properties of the debian/ directory should point to the location where the orig.tar.gz file(s) can be fetched.

See Also