Differences between revisions 110 and 111
Revision 110 as of 2015-12-06 15:23:04
Size: 18379
Editor: GuillemJover
Comment: Use https for hadrons.org URLs
Revision 111 as of 2015-12-06 15:23:48
Size: 18381
Editor: GuillemJover
Comment: Fix typo
Deletions are marked like this. Additions are marked like this.
Line 138: Line 138:
There's some important updates in https://git.hadrons.org/cgit/debian/git-hooks.git, which have already been sent to the Alioth admins. There are some important updates in https://git.hadrons.org/cgit/debian/git-hooks.git, which have already been sent to the Alioth admins.

Translation(s): English - ?Italiano


Using Git on Alioth

Creating a repository

Collab Maint project

Instead of registering a separate project(s) for packages, consider using the Collab Maint project; see ?Alioth/PackagingProject for more information. Once granted access to that project (DD have access by default, non-DD must request it) you can start hosting Git project immediately. The simplest way is to run the create-remote-repo command from git-buildpackage from your local development repository:

$ gbp create-remote-repo 

(Note that this currently doesn't set the hooks correctly.)

An alternative way is to log into git.debian.org and create a git repository under /git/collab-maint/. You can use a little script on git.debian.org:

   1 #!/bin/bash
   2 set -u
   3 umask 002
   4 cd /git/collab-maint && ./setup-repository $1 "Packaging for $1"
   5 echo -e "\n: Clone this repo using: git clone ssh://$USER@git.debian.org/git/collab-maint/$1.git"
   6 echo "Or add the remote to an existing repo: git remote add origin ssh://$USER@git.debian.org/git/collab-maint/$1.git"

Be aware that if you create a repository with a plus sign in the name, eg dvd+rw-tools, gitweb will require you to escape the + sign in the URL to %2B, so you'll need to use a URL such as https://anonscm.debian.org/gitweb/?p=collab-maint/dvd%2Brw-tools.git;a=summary to view your git repository. cgit does not have this problem though.

Separate project

For separately registered projects, Alioth will create a repository named /git/<project-group> owned by your project group (provided that you selected Git as the type of repository that you want, see ?Alioth/FAQ#vcs-repos).

You can create multiple repositories in this directory instead:

$ # On git.debian.org via SSH
$ umask 002
$ mkdir <project>.git
$ cd <project>.git
$ git --bare init --shared
$ vim description
$ mv hooks/post-update.sample hooks/post-update
$ chmod a+x hooks/post-update

Initial push

You can do this manually as described here or use git-buildpackage as described at http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.html#GBP.IMPORT.NEW.UPSTREAM

Then you have to put some real content in that empty repository. Either git fetch/pull on git.debian.org or git push from a remote host to git.debian.org. See below for the Git URLs that you can use.

<!> Example: To create local Git repository and populate it with existing source:

$ cd /path/to/sources/<project>
$ git init
$ git config user.name '<Your name>'
$ git config user.email '<Your email>'
$ git config gitweb.owner '<Your name>'
$ git add .
$ git commit -m 'Initial import of <project> version <version>'
$ git tag <version>

<!> Example: To push a project to an alioth collab-maint repository:

$ cd /path/to/<project>.git
$ git remote add alioth ssh://<user>@git.debian.org/git/collab-maint/<project>.git
$ git push alioth <branchname>
$ git push alioth --tags

Accessing repositories

They are accessible by git using sftp, rsync over ssh, or git-server/http for read-only access. Note that https://anonscm.debian.org/cgit/ is only updated by cron once every 6 hours, so be patient when creating a new repository... it'll show up eventually. For instance, you'll be able to check out your branch though sftp with:

$ git clone ssh://<user>@git.debian.org/git/<group>/<project>.git

Anonymous users will enjoy read-only access with following commands. The <group> could be e.g. package name or collab-maint if hosted collaboratively.

$ git clone git://anonscm.debian.org/git/<group>/<project>.git
$ git clone https://anonscm.debian.org/git/<group>/<project>.git

Please use the native git protocol, because it is much more efficient than cloning over https/http. If your firewall restricts access to port 9418 you could clone over https/http. But notice that if the upstream repo on alioth is repacked, you have to download the whole objects again if you clone/fetch over https/http.

Repository maintenance

Git repositories tend to grow quite large quickly. From time to time, you have to repack the repositories to save space and keep optimal performances (by not having too many of files in the objects subdirectory).

$ cd /git/<group>/<project>.git
$ git repack && git gc

Those operations should be always safe (and can be ran via cron).

You can optimize even more by using some options but then people fetching over https/http (bad idea!) might have troubles and they are not safe to run when someone else is using the repository (for example pushing new revisions). Thus don't put those commands in a cron invocation:

$ git repack -a -d && git gc --prune

Setting up hooks

Commit mails with diff

Modify the relevant values in the log below:

$ cd <project>.git
# For sending diff to a mailing list use this:
$ git config --add multimailhook.mailinglist "project-commit@lists.alioth.debian.org"
# For additionally sending diff to the PTS use this:
$ git config --add multimailhook.bcc "<sourcepackage>_cvs@packages.qa.debian.org"
# If you don't have a commit mailing list and want to send the diff only to the PTS, you have to use the PTS like a mailing list:
git config --add multimailhook.mailinglist "<sourcepackage>_cvs@packages.qa.debian.org"
# Create the hook
$ cat >hooks/post-receive <<END
#!/bin/sh
exec /usr/local/bin/git-commit-notice
END
$ chmod a+x hooks/post-receive

Marking closed bugs as pending in BTS

To mark bugs as pending that are marked as closed in the debian/changelog, add /usr/local/bin/git-post-receive-tag-pending to the repo's post-receive hook, eg:

exec pee /usr/local/bin/git-post-receive-tag-pending /usr/local/bin/git-commit-notice

There are some important updates in https://git.hadrons.org/cgit/debian/git-hooks.git, which have already been sent to the Alioth admins.

Sending notices on IRC via KGB bots

kgb-client is installed on git.debian.org. It is configured using a git config property, a configuration file which should be stored in your project's filespace on alioth and is executed as part of your post-receive hook.

Put the following in hooks/post-receive:

exec /home/groups/kgb/bin/kgb-client-trunk

and make sure to configure it with:

git config kgb.conf /home/groups/my-project/kgb-client.conf

In case there are other hooks that need to be run, the standard input needs to be duplicated, as in:

exec pee /home/groups/kgb/bin/kgb-client-trunk /usr/local/bin/git-commit-notice

For more information, on using the KGB bots and having access granted for your project see the KGB on Alioth page.

Using personal Git repositories

Creating personal Git repositories

It is also possible to have personal Git repositories. Just log in on git.debian.org, then

$ mkdir ~/public_git
$ chmod a+xr public_git  # give access to git daemon
$ exit

Consider $PRJ is your project name, $PRJDIR the path to your local repo of the project and 'login' is your alioth user name.

export PRJ=proj                                # name of exported project (e.g.'pbuilder')
export PRJDIR=~/proj                           # path to your current local git repo
export login=youraliothusername                # duh, your alioth user name

Log out to prepare the repository that will be published. Copy over to alioth.

git clone --bare $PRJDIR $PRJ.git              # create the exportable directory
touch $PRJ.git/git-daemon-export-ok            # tell git daemon that it is a public repo
cp $PRJ.git/hooks/post-update{.sample,}        # Copy the hook script
chmod a+x $PRJ.git/hooks/post-update           # fetches over https/http work
(cd $PRJ.git && git update-server-info)        # fetches over https/http work
editor $PRJ.git/description                    # edit description for cgit/gitweb
scp -r $PRJ.git/ $login@git.debian.org:public_git/ # copy the repo to alioth

Check that your repo is visible (may take some time, though, see below) by using a browser to access your new repo:

https://anonscm.debian.org/cgit/users/$login/$PRJ.git

Check that it works in some empty temp dir:

git clone git://anonscm.debian.org/users/$login/$PRJ.git
# alternatively, you could use the (slower) https fetch method
#git clone https://anonscm.debian.org/git/users/$login/$PRJ.git

Note 1: rsync is much faster than scp: rsync -av -e "ssh -l $login" $PRJ.git/ git.debian.org:public_git/$PRJ.git/

Note 2: Using a sshfs mount to create the bare repository results in some errors and, apparently, is slower than creating the repo on the local disc, and then scp-ing it to alioth.

The symlinks used for the cgit/gitweb interface are updated every hour, at 33 minutes past the hour.

They will be available through the following URLs:

$ git clone git://anonscm.debian.org/users/$login/$PRJ.git

Forking Git repositories onto Alioth

There is a script that can fork a public git repo for you and make it available on Alioth. It will do most of the steps described in the previous section. Please study it before you execute it. You can run it on alioth with:

$ /srv/home/users/tpo/bin/clone_onto_alioth git://anonscm.debian.org/some/alioth_project.git

Maintaining the published Git repository up to date

Pushing to that repo is done with:

$ git push ssh://$login@git.debian.org/~$login/public_git/$PRJ.git

Personal Git repository - the morph way

Probably it's only me (hence the 'morph way' ;) ) but I'd like start from scratch, with an empty repository and then start adding content to it. That's how to do it.

This is a real example that I've created from this doc. Login to git.debian.org with your user, then issue:

$ mkdir ~/public_git
## give access to git daemon
$ chmod a+xr public_git
$ cd public_git
## create the project repository directory
$ git init --bare test.git
Initialized empty Git repository in /srv/alioth.debian.org/chroot/home/users/morph/public_git/test.git/
$ cd test.git/
$ cp hooks/post-update.sample hooks/post-update
$ chmod a+x hooks/post-update
$ touch git-daemon-export-ok
$ git update-server-info
## set the description of the project
$ echo 'the project description' > description

work on alioth is done. You can have a look at the empty repo at:

https://anonscm.debian.org/cgit/users/morph/test.git

On our machine, we can clone the repo with:

$ git clone ssh://morph@git.debian.org/git/users/morph/test.git
Initialized empty Git repository in /home/morph/tmp/test/.git/
warning: You appear to have cloned an empty repository.
fatal: The remote end hung up unexpectedly

Don't worry about the "fatal" error: it's the way git says you've cloned an empty repository.

you can now start working on the repository:

$ cd test
$ echo "dummy file" > f
$ git add .
$ git ci -a -m "commit message"
[master (root-commit) 8ba3ab1] commit message
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 f
$ git push origin master
Counting objects: 3, done.
Writing objects: 100% (3/3), 214 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To ssh://morph@git.debian.org/git/users/morph/test.git
 * [new branch]      master -> master

now that the new branch 'master' is setup, you can git push normally without having to specify repo and ref. Now also cgit/gitweb is showing some info :)

Convert a SVN Alioth repository to Git

Here we'd like to provide a sequence of steps to follow to convert a SVN repository to a Git one (supposing they're both on Alioth). We will give a practical example converting reportbug repository. Let's store its URL in a shell variable, so that the example scripts are easier to re-use for other packages.

SVN_URL='svn://svn.debian.org/svn/reportbug/'

First, you have to reconfigure your FusionForge project to indicate that you want to use git (see ?dedicated section in FAQ) so that /git/<project> is created. Once this is done, you can create as many git repository in it as you want.

Ref: http://gitready.com/beginner/2009/02/04/converting-from-svn.html

Create the author file

SVN stores commit authors information as SVN username, while Git uses fullnames and email addresses.

If you convert a repository without an author file, the commits will belong to a fake user made of the <SVN username>@<SVN UUID> as returned by

$ svn info $SVN_URL| grep UUID
Repository UUID: 692dde74-6b68-4300-bcb1-27de2ae967a4

In order to create a complete author file, we need to know everyone that has ever committed to the SVN repository; to achieve this, we can use this script:

$ svn log $SVN_URL | awk -F'|' '/^r[0-9]+/ { print $2 }' | sort -u
 appaji-guest
 bignose-guest
 lawrencc
 morph
 morph-guest
 (no author)
 root

Now we can create a file that contains this information:

...
svn username = fullname <email address>
...
morph = Sandro Tosi <morph@debian.org>
...

that maps SVN usernames to Git ones; manually edit and store that file where you prefer for later usage. If you are converting from a group-maintained package or package set, there might already be one which you can use.

Please note that if git svn clone finds an SVN author that's not in this file, it will exit with an error and won't finish cloning the repository.

Convert the repository

We are now ready to convert the repository. We will use git-svn, a great bidirectional interface between SVN and Git (you can work on Git and commit on Svn, and fetch the work done on SVN).

Let's first create a directory to contains the converted Git repo (if the same <project> name as SVN is not right) and then do the actual conversion:

mkdir pkg
git svn clone $SVN_URL --prefix=svn-import/ --stdlayout --authors-file=<author filename> --no-metadata pkg

Note:

  • we used clone from git svn since it does "init + fetch" in a single pass
  • we added prefix as these branches will be converted later and a common prefix makes this easier
  • --author-file specifies the location of the author file, previously defined
  • --no-metadata will remove the SVN url from every commit messages (something like "git-svn-id: svn://svn.debian.org/svn/pkg/trunk@696 692dde74-6b68-4300-bcb1-27de2ae967a4"); we used it because we want to remove SVN, if you plan to maintain both repositories, then remove this option
  • pkg/ is the destination dir

Convert remote tags and branches to local one

After conversion, the SVN tags and branches are all Git remote branches. We would like to convert them to Git proper tags and to local tags and branches.

Let's start with converting tags. If you would like to sign the tags with your GPG/PGP key, add the option -s to the git tag command:

for branch in `git branch -r`; do
    if [ `echo $branch | egrep "svn-import/tags/.+$"` ]; then
        version=`basename $branch`
        subject=`git log -1 --pretty=format:"%s" $branch`
        GIT_COMMITTER_DATE=`git log -1 --pretty=format:"%ci" $branch` \
            git tag -f -m "$subject" "debian/$version" "$branch^"

        git branch -d -r $branch
    fi
done

where we list all the "tags/*" branches, convert to tag and remove the remote branch.

If it's a one-way migration (SVN will not be used anymore), then SVN compatibility references can be removed:

git branch -d -r svn-import/trunk
git config --remove-section svn-remote.svn
rm -rf .git/svn .git/{logs/,}refs/remotes/svn/

Now we can convert the remaining remote branches to local ones:

git config remote.origin.url .
git config --add remote.origin.fetch +refs/remotes/*:refs/heads/*
git fetch

And finally we can import the upstream source:

git symbolic-ref HEAD refs/heads/upstream
git rm --cached -r .
git commit --allow-empty -m 'initial upstream branch'
git checkout -f master
git merge upstream
git-import-orig ../foo_1.2.3.orig.tar.gz

References:

Prepare the repository for publishing on web

What we have up to now is a Git repository "injected" in a checkout copy. We would like to publish that repository to the web, so other clients can clone on their machine.

We start by creating a new repository:

morph@alioth:~/reportbug$ git clone --bare reportbug reportbug.git
Initialized empty Git repository in /srv/alioth.debian.org/chroot/home/users/morph/reportbug/reportbug.git/

that also allows for a big size reduction:

morph@alioth:~/reportbug$ du -hs reportbug reportbug.git/
35M     reportbug
1.4M    reportbug.git/

Now we need to edit the description of the repository (the one that cgit/gitweb will display):

echo "Reportbug - reports bugs in the Debian distribution" > reportbug.git/description

We are now ready to copy the resulting repository to its final destination:

cp -r reportbug.git /git/reportbug/

Now point your browser to https://anonscm.debian.org/cgit/reportbug/reportbug.git and enjoy! To get a local repository run:

git clone git://anonscm.debian.org/git/reportbug/reportbug.git
git clone ssh://git.debian.org/git/reportbug/reportbug.git

(in the 2 flavors for anonymous and ssh authentication) on your machine.


?CategoryAlioth CategoryGit