Differences between revisions 99 and 100
Revision 99 as of 2014-04-03 21:12:04
Size: 17356
Editor: ?JaviMerino
Comment:
Revision 100 as of 2014-04-03 21:12:50
Size: 17353
Editor: ?JaviMerino
Comment:
Deletions are marked like this. Additions are marked like this.
Line 19: Line 19:
An alternative way is to log into ''git.debian.org'' and create a git repository under `/git/collab-maint/`.

You can use a little script on git.debian.org:
An alternative way is to log into ''git.debian.org'' and create a git repository under `/git/collab-maint/`. You can use a little script on git.debian.org:

Translation(s): English - Italiano


Using Git on Alioth

Creating a repository

Collab Maint project

Instead of registering a separate project(s) for packages, consider using the Collab Maint project; see Alioth/PackagingProject for more information. Once granted access to that project (DD have access by default, non-DD must request it) you can start hosting Git project immediately. The simplest way is to run the create-remote-repo command from git-buildpackage from your local development repository:

$ gbp create-remote-repo 

(Note that this currently doesn't set the hooks correctly.)

An alternative way is to log into git.debian.org and create a git repository under /git/collab-maint/. You can use a little script on git.debian.org:

   1 #!/bin/bash
   2 set -u
   3 umask 002
   4 cd /git/collab-maint && ./setup-repository $1 "Packaging for $1"
   5 echo -e "\n: Clone this repo using: git clone ssh://$USER@git.debian.org/git/collab-maint/$1.git"
   6 echo "Or add the remote to an existing repo: git remote add origin ssh://$USER@git.debian.org/git/collab-maint/$1.git"

Be aware that if you create a repository with a plus sign in the name, eg dvd+rw-tools, Gitweb will require you to escape the + sign in the URL to %2B, so you'll need to use a URL such as http://anonscm.debian.org/gitweb/?p=collab-maint/dvd%2Brw-tools.git;a=summary to view your git repository.

Separate project

For separately registered projects, Alioth will create a repository named /git/<project-group> owned by your project group (provided that you selected Git as the type of repository that you want, see Alioth/FAQ#vcs-repos).

You can create multiple repositories in this directory instead:

$ # On git.debian.org via SSH
$ umask 002
$ mkdir <project>.git
$ cd <project>.git
$ git --bare init --shared
$ vim description
$ mv hooks/post-update.sample hooks/post-update
$ chmod a+x hooks/post-update

Initial push

You can do this manually as described here or use git-buildpackage as described at http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.html#GBP.IMPORT.NEW.UPSTREAM

Then you have to put some real content in that empty repository. Either git fetch/pull on git.debian.org or git push from a remote host to git.debian.org. See below for the Git URLs that you can use.

<!> Example: To create local Git repository and populate it with existing source:

$ cd /path/to/sources/<project>
$ git init
$ git config user.name '<Your name>'
$ git config user.email '<Your email>'
$ git config gitweb.owner '<Your name>'
$ git add .
$ git commit -m 'Initial import of <project> version <version>'
$ git tag <version>

<!> Example: To push a project to an alioth collab-maint repository:

$ cd /path/to/<project>.git
$ git remote add alioth ssh://<user>@git.debian.org/git/collab-maint/<project>.git
$ git push alioth <branchname>
$ git push alioth --tags

Accessing repositories

They are accessible by git using sftp, rsync over ssh, or git-server/http for read-only access. Note that http://git.debian.org is only updated by cron once every 6 hours, so be patient when creating a new repository... it'll show up eventually. For instance, you'll be able to check out your branch though sftp with:

$ git clone ssh://<user>@git.debian.org/git/<group>/<project>.git

Anonymous users will enjoy read-only access with following commands. The <group> could be e.g. package name or collab-maint if hosted collaboratively.

$ git clone git://git.debian.org/git/<group>/<project>.git
$ git clone http://git.debian.org/git/<group>/<project>.git

Please use the native git protocol, because it is much more efficient than cloning over http. If your firewall restricts access to port 9418 you could clone over http. But notice that if the upstream repo on alioth is repacked, you have to download the whole objects again if you clone/fetch over http.

Repository maintenance

Git repositories tend to grow quite large quickly. From time to time, you have to repack the repositories to save space and keep optimal performances (by not having too many of files in the objects subdirectory).

$ cd /git/<group>/<project>.git
$ git repack && git gc

Those operations should be always safe (and can be ran via cron).

You can optimize even more by using some options but then people fetching over HTTP (bad idea!) might have troubles and they are not safe to run when someone else is using the repository (for example pushing new revisions). Thus don't put those commands in a cron invocation:

$ git repack -a -d && git gc --prune

Setting up hooks

Commit mails with diff

Modify the relevant values in the log below:

$ cd <project>.git
# For sending diff to a mailing list use this:
$ git config --add multimailhook.mailinglist "project-commit@lists.alioth.debian.org"
# For additionally sending diff to the PTS use this:
$ git config --add multimailhook.bcc "<sourcepackage>_cvs@packages.qa.debian.org"
# If you don't have a commit mailing list and want to send the diff only to the PTS, you have to use the PTS like a mailing list:
git config --add multimailhook.mailinglist "<sourcepackage>_cvs@packages.qa.debian.org"
# Create the hook
$ cat >hooks/post-receive <<END
#!/bin/sh
exec /usr/local/bin/git-commit-notice
END
$ chmod a+x hooks/post-receive

Sending notices on IRC via KGB bots

kgb-client is installed on git.debian.org, you can use it directly (see its manual page for details). It doesn't use any git config parameter, you must pass everything on the command line and in an associated config file.

If you want to use git-commit-notice and kbg-client in the post-receive hook, you should use "pee" to multiplex the standard input to both programs. Here's an example:

$ cat >hooks/post-receive <<END
#!/bin/sh
exec pee /usr/local/bin/git-commit-notice "kgb-client --repository git --git-reflog - --conf /home/groups/<yourproject>/kgb-client.conf --module <sourcepackage>"
END
$ chmod a+x hooks/post-receive

For more information, cf. the KGB on Alioth page.

Using personal Git repositories

Creating personal Git repositories

It is also possible to have personal Git repositories. Just log in on git.debian.org, then

$ mkdir ~/public_git
$ chmod a+xr public_git  # give access to git daemon
$ exit

Consider $PRJ is your project name, $PRJDIR the path to your local repo of the project and 'login' is your alioth user name.

export PRJ=proj                                # the name exported project (e.g.'pbuilder')
export PRJDIR=~/proj                           # path to your current local git repo
export login=youraliothusername                # duh, your alioth user name

Log out to prepare the repository that will be published. Copy over to alioth.

git clone --bare $PRJDIR $PRJ.git              # create the exportable directory
touch $PRJ.git/git-daemon-export-ok            # tell git daemon that it is a public repo
cp $PRJ.git/hooks/post-update{.sample,}        # Copy the hook script
chmod a+x $PRJ.git/hooks/post-update           # fetches over http work
(cd $PRJ.git && git update-server-info)        # fetches over http work
editor $PRJ.git/description                       # edit description for gitweb
scp -r $PRJ.git/ $login@git.debian.org:public_git/ # copy the repo to alioth

Check that your repo is visible (may take some time, though, see below) by using a browser to access your new repo:

http://git.debian.org/?p=users/$login/$PRJ.git;a=summary

Check that it works in some empty temp dir:

git clone git://git.debian.org/users/$login/$PRJ.git
# alternatively, you could use the (slower) http fetch method
#git clone http://git.debian.org/git/users/$login/$PRJ.git

Note 1: rsync is much faster than scp: rsync -av -e "ssh -l $login" $PRJ.git/ git.debian.org:public_git/$PRJ.git/

Note 2: Using a sshfs mount to create the bare repository results in some errors and, apparently, is slower than creating the repo on the local disc, and then scp-ing it to alioth.

The symlinks used for the gitweb interface are updated every six hours.

They will be available through the following URLs:

$ git clone git://git.debian.org/users/$login/$PRJ.git

Maintaining the published Git repository up to date

Pushing to that repo is done with:

$ git push ssh://$login@git.debian.org/~$login/public_git/$PRJ.git

Personal Git repository - the morph way

Probably it's only me (hence the 'morph way' ;) ) but I'd like start from scratch, with an empty repository and then start adding content to it. That's how to do it.

This is a real example that I've created from this doc. Login to git.debian.org with your user, then issue:

$ mkdir ~/public_git
## give access to git daemon
$ chmod a+xr public_git
$ cd public_git
## create the project repository directory
$ git init --bare test.git
Initialized empty Git repository in /srv/alioth.debian.org/chroot/home/users/morph/public_git/test.git/
$ cd test.git/
$ cp hooks/post-update.sample hooks/post-update
$ chmod a+x hooks/post-update
$ touch git-daemon-export-ok
$ git update-server-info
## set the description of the project
$ echo 'the project description' > description

work on alioth is done. You can have a look at the empty repo at:

http://git.debian.org/?p=users/morph/test.git

On our machine, we can clone the repo with:

$ git clone git+ssh://morph@git.debian.org/git/users/morph/test.git
Initialized empty Git repository in /home/morph/tmp/test/.git/
warning: You appear to have cloned an empty repository.
fatal: The remote end hung up unexpectedly

Don't worry about the "fatal" error: it's the way git says you've cloned an empty repository.

you can now start working on the repository:

$ cd test
$ echo "dummy file" > f
$ git add .
$ git ci -a -m "commit message"
[master (root-commit) 8ba3ab1] commit message
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 f
$ git push origin master
Counting objects: 3, done.
Writing objects: 100% (3/3), 214 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git+ssh://morph@git.debian.org/git/users/morph/test.git
 * [new branch]      master -> master

now that the new branch 'master' is setup, you can git push normally without having to specify repo and ref. Now also gitweb is showing some info :)

Convert a SVN Alioth repository to Git

Here we'd like to provide a sequence of steps to follow to convert a SVN repository to a Git one (supposing they're both on Alioth). We will give a practical example converting reportbug repository. Let's store its URL in a shell variable, so that the example scripts are easier to re-use for other packages.

SVN_URL='svn://svn.debian.org/svn/reportbug/'

First, you have to reconfigure your FusionForge project to indicate that you want to use git (see dedicated section in FAQ) so that /git/<project> is created. Once this is done, you can create as many git repository in it as you want.

Ref: http://gitready.com/beginner/2009/02/04/converting-from-svn.html

Create the author file

SVN stores commit authors information as SVN username, while Git uses fullnames and email addresses.

If you convert a repository without an author file, the commits will belong to a fake user made of the <SVN username>@<SVN UUID> as returned by

$ svn info $SVN_URL| grep UUID
Repository UUID: 692dde74-6b68-4300-bcb1-27de2ae967a4

In order to create a complete author file, we need to know everyone that has ever committed to the SVN repository; to achieve this, we can use this script:

$ svn log $SVN_URL | awk -F'|' '/^r[0-9]+/ { print $2 }' | sort -u
 appaji-guest
 bignose-guest
 lawrencc
 morph
 morph-guest
 (no author)
 root

Now we can create a file that contains this information:

...
svn username = fullname <email address>
...
morph = Sandro Tosi <morph@debian.org>
...

that maps SVN usernames to Git ones; manually edit and store that file where you prefer for later usage. If you are converting from a group-maintained package or package set, there might already be one which you can use.

Please note that if git svn clone finds an SVN author that's not in this file, it will exit with an error and won't finish cloning the repository.

Convert the repository

We are now ready to convert the repository. We will use git-svn, a great bidirectional interface between SVN and Git (you can work on Git and commit on Svn, and fetch the work done on SVN).

Let's first create a directory to contains the converted Git repo (if the same <project> name as SVN is not right) and then do the actual conversion:

mkdir pkg
git svn clone $SVN_URL --prefix=svn-import/ --stdlayout --authors-file=<author filename> --no-metadata pkg

Note:

  • we used clone from git svn since it does "init + fetch" in a single pass
  • we added prefix as these branches will be converted later and a common prefix makes this easier
  • --author-file specifies the location of the author file, previously defined
  • --no-metadata will remove the SVN url from every commit messages (something like "git-svn-id: svn://svn.debian.org/svn/pkg/trunk@696 692dde74-6b68-4300-bcb1-27de2ae967a4"); we used it because we want to remove SVN, if you plan to maintain both repositories, then remove this option
  • pkg/ is the destination dir

Convert remote tags and branches to local one

After conversion, the SVN tags and branches are all Git remote branches. We would like to convert them to Git proper tags and to local tags and branches.

Let's start with converting tags. If you would like to sign the tags with your GPG/PGP key, add the option -s to the git tag command:

for branch in `git branch -r`; do
    if [ `echo $branch | egrep "svn-import/tags/.+$"` ]; then
        version=`basename $branch`
        subject=`git log -1 --pretty=format:"%s" $branch`
        GIT_COMMITTER_DATE=`git log -1 --pretty=format:"%ci" $branch` \
            git tag -f -m "$subject" "debian/$version" "$branch^"

        git branch -d -r $branch
    fi
done

where we list all the "tags/*" branches, convert to tag and remove the remote branch.

If it's a one-way migration (SVN will not be used anymore), then SVN compatibility references can be removed:

git branch -d -r svn-import/trunk
git config --remove-section svn-remote.svn
rm -rf .git/svn .git/{logs/,}refs/remotes/svn/

Now we can convert the remaining remote branches to local ones:

git config remote.origin.url .
git config --add remote.origin.fetch +refs/remotes/*:refs/heads/*
git fetch

And finally we can import the upstream source:

git symbolic-ref HEAD refs/heads/upstream
git rm --cached -r .
git commit --allow-empty -m 'initial upstream branch'
git checkout -f master
git merge upstream
git-import-orig ../foo_1.2.3.orig.tar.gz

References:

Prepare the repository for publishing on web

What we have up to now is a Git repository "injected" in a checkout copy. We would like to publish that repository to the web, so other clients can clone on their machine.

We start by creating a new repository:

morph@alioth:~/reportbug$ git clone --bare reportbug reportbug.git
Initialized empty Git repository in /srv/alioth.debian.org/chroot/home/users/morph/reportbug/reportbug.git/

that also allows for a big size reduction:

morph@alioth:~/reportbug$ du -hs reportbug reportbug.git/
35M     reportbug
1.4M    reportbug.git/

Now we need to edit the description of the repository (the one that gitweb will display):

echo "Reportbug - reports bugs in the Debian distribution" > reportbug.git/description

We are now ready to copy the resulting repository to its final destination:

cp -r reportbug.git /git/reportbug/

Now point your browser to http://git.debian.org/?p=reportbug/reportbug.git and enjoy! To get a local repository run:

git clone git://git.debian.org/git/reportbug/reportbug.git
git clone git+ssh://git.debian.org/git/reportbug/reportbug.git

(in the 2 flavors for anonymous and ssh authentication) on your machine.


CategoryAlioth CategoryGit