Differences between revisions 77 and 78
Revision 77 as of 2014-02-07 12:07:57
Size: 15182
Comment:
Revision 78 as of 2014-02-07 12:08:49
Size: 15186
Comment:
Deletions are marked like this. Additions are marked like this.
Line 38: Line 38:
If you want to obtain *all* of the team packages (have in mind that the team has *a lot* of packages, so this will probably take a little long), do: If you want to obtain ''all'' of the team packages (have in mind that the team has ''a lot'' of packages, so this will probably take a little long), do:

This page describes the current packaging practices for Ruby in Debian. For more general information, see this wiki page.

This page is quite disorganized, and some of it was written as a "plans for Ruby in wheezy page", not as a general documentation for packaging. Please do not hesitate to improve this page. Another effort to document the workflow of packaging ruby-gems for Debian is made by the debian-diaspora Team at: this wiki page , which is focused at beginners and very detailed.

Joining the pkg-ruby-extras team

We always welcome people to help us out. If you want to, please follow the following guidelines.

  • Contact us first on our mailing list or on IRC.

  • Register an alioth account account.

  • Go to our project page and request your addition to the team. Please specify what you plan to work on.

  • Wait until we approve your addition. Note that there’s a delay before getting your account on svn.debian.org, see the Alioth FAQ for details.

  • Read all the docs (sorry).
  • Enjoy!

Using Git

We use git together with mr (Multiple Repository management tool) to track the git repositories for each package.

Before you continue, ensure you have installed the tools used by the team:

apt-get install mr git-buildpackage gem2deb

Obtaining the master repository

git clone <alioth-username>@git.debian.org:/git/pkg-ruby-extras/pkg-ruby-extras.git

To obtain one of the packages, do:

cd pkg-ruby-extras
./checkout $PACKAGE

If you want to obtain all of the team packages (have in mind that the team has a lot of packages, so this will probably take a little long), do:

# look at .mrconfig
mr --force checkout

To clone all team packages with parallel checkouts, use

mr --force -j 5 checkout

To avoid typing your username and your password see this page.

Creating a new git repository and pushing an existing package to git.debian.org

Note: please do not leave empty repositories on alioth; only create a repository there just before pushing your package.

Suppose you want to package the latest 0.7 version of the foo Ruby gem, to obtain a ruby-foo package. See the naming conventions for packages.

On git.debian.org:

cd /git/pkg-ruby-extras
./setup-repository ruby-foo

On your local machine, get a basic .dsc from the library you want to package, using:

gem2deb foo

gem2deb will then try to download the foo gem from rubygems.org, and convert it to a primitive Debian package ruby-foo_0.7-1.dsc. You can also run gem2deb on a local .gem file foo-0.7.gem or a tarball (e.g. from Github).

At this point, please do not make any change to the package generated by gem2deb. It is important to track all the changes in git. Then:

cd where-you-cloned-pkg-ruby-extras.git
./make-mrconfig # will generate the config for your new repo
mr checkout # will clone your new repo
cd ruby-foo

git import-dsc --pristine-tar path-to/ruby-foo_0.7-1.dsc
git tag -d debian/0.7-1 # because the package is not ready yet
git push --all
git push --tags

Then, make the needed changes to the packaging, and then

git push

After upload, tag the package with

git tag debian/0.7-1

And don't forget to

git push --tags

Updating a package to a newer version

If you have a working debian/watch file (check by running uscan --verbose --report), you can simply import a new version with

git import-orig --pristine-tar --uscan

Alternatively, you can fetch the most recent version of a gem, and make a tarball out of it

gem2tgz <gemname>

where <gemname> is the upstream name of the gem, or run gem2tgz on a local .gem file.

Now update your packaging in the master branch. Once you are done, push the branches and the tags to the team repository

git push --all
git push --tags

Known Issues

If mr checkout gives an error like mr: illegal section "[DEFAULT]" in untrusted:

Add .mrconfig path to ~/.mrtrust

If your local username is different from your Alioth username (that's probably the case for non-DD developers with -guest accounts), add the following to your ~/.ssh/config:

Host alioth.debian.org svn.debian.org git.debian.org
User alioth-username

Handling patches on upstream code

If the package you are working on includes upstream patches (i.e. debian/patches/*), you need to make sure that the Git repository is in a sane state after build (i.e. no uncommitted changes to the work tree).

Since 1.16.1, dpkg-source unapplies by default patches that have been applied during --before-build. For earlier versions, it is possible to force this behaviour by adding the unapply-patches option to debian/source/local-options (source: blog post by Raphael Hertzog)

$ echo "unapply-patches" >>debian/source/local-options
$ git add debian/source/local-options
$ git commit -m "Unapply patches after build"

References

Guidelines for Ruby packaging

Single package for all Ruby versions

For Wheezy, Ruby software must not come in separate packages per Ruby interpreter version, as we did until Squeeze (libfoo-ruby1.8, libfoo-ruby1.9.1 etc). We are going to use a single binary package per source package no matter what. In the case of native extensions, those single binary packages will contain the compiled extensions (*.so) for all supported Ruby versions. This is supported by gem2deb, so you should use it.

Naming of ruby packages

The current libfoo-ruby naming, inspired for perl, is not really great. It would be better to move to the python naming, with ruby-foo. Since all packages will have to be modified anyway to switch to gem2deb (see below), it is the perfect time to do that.

The guidelines are the following:

  • Binary packages must normally be named "ruby-foo". If the package is mainly used as an application (not as a library), then it can be named "foo". Known examples are rails, chef, rubygems, puppet.
  • Source packages must have the same name as the "main" binary package. (our infrastructure is better at handling this case)

Renaming existing packages

Existing packages must be renamed to the new scheme. This renaming must be done using the standard Debian practices (i.e. Debian Developers' Reference, section 5.9.3). In our case, this means:

  • when libfoo-ruby gets renamed to ruby-foo, then ruby-foo must declare Provides: libfoo-ruby, Replaces: libfoo-ruby and Breaks: libfoo-ruby

  • when libfoo-ruby1.8 gets renamed to ruby-foo, then ruby-foo must declare Provides: libfoo-ruby1.8, Replaces: libfoo-ruby1.8, and Breaks: libfoo-ruby1.8

  • when libfoo-ruby1.8 and libfoo-ruby1.9.1 are turned into a single pure-Ruby package ruby-foo, then it must declare Provides: libfoo-ruby, libfoo-ruby1.8, libfoo-ruby1.9.1, Replaces: libfoo-ruby, libfoo-ruby1.8, libfoo-ruby1.9.1, and Breaks: libfoo-ruby, libfoo-ruby1.8, libfoo-ruby1.9.1

  • ruby-foo must also provide transitional binary packages for libfoo-ruby, libfoo-ruby1.8 and libruby1.9.1 (when it's the case). You can use the gen-ruby-trans-pkgs script to generate the proper snippet to be added to your debian/control.

  • note that this is only needed for packages that were already released with Squeeze. New packages must not keep any references to packages named as lib*-ruby.
  • after the renamed package enters the archive, please request the removal of the old package.

    • I guess this is not needed when using transitional packages - according to Renaming_a_Package the old source package will be removed automatically. If that's the case, the above bullet point, as well as this one, can be removed.

Removing transitional packages

For packages that have been renamed before the release of Wheezy from the libsomething-ruby* scheme to ruby-something, all transitional packages libsomething-ruby* can be removed for Jessie, after checking that they have no more reverse (build-)dependencies.

For packages that have not yet been renamed by the Wheezy release, the transitional packages must remain in place until after the Jessie release.

Handling of shebangs

  • applications willing to force the use of ruby1.8 should use /usr/bin/ruby1.8 in shebang, and depend on ruby1.8

  • applications willing to use the selected ruby implementation (whatever it is) should use /usr/bin/env ruby, and depend on ruby | ruby-interpreter

gem2deb currently does the following:

  • If the XS-Ruby-Versions field contains 'all', all shebangs of programs installed to /usr/bin are rewritten to /usr/bin/env ruby

  • Otherwise, all shebangs of programs installed to /usr/bin are rewritten to the binary corresponding to the first version listed in XS-Ruby-Versions. For example, is XS-Ruby-Versions is ruby1.8, ruby1.9.1, all shebangs will be rewritten to /usr/bin/ruby1.8

Values of XS-Ruby-Versions

Currently the XS-Ruby-Versions values matches the name of the ruby interpreter, e.g. current supported values are:

  • ruby1.8
  • ruby1.9.1

When more ruby interpreters will be added, the list will be expanded.

gem2deb as the preferred packaging tool for ruby software

gem2deb is a replacement for our current cdbs-based packaging suite. It:

  • does almost everything automatically
  • uses dh (much faster)
  • is easier to adapt to our needs, since there's no dependency on an external tool
  • runs the test suite as part of the build, for each ruby implementation (yes, it's possible to override this). See more about running tests

  • uses a single binary package for native libraries, instead of one binary package for each ruby implementation

To test it:

git clone git://github.com/ln/gem2deb.git
cd gem2deb
rake install
then, cd to some temporary place, and do:
gem2deb rdiscount # native gem
or:
gem2deb i18n # pure-ruby gem

It serves two goals:

  • it can be used by users to generate .debs from gems locally
  • it generates Debian source package from which we can do the packaging work

Prerequisites for migrations:

  • we need to decide on a workflow for packaging (switch to git?)
  • we need to decide on a migration plan to avoid breaking the archive for too long

Success metrics for the transition to gem2deb

  • There are no non-transitional binary packages named lib.*-ruby.* in the archive. No package should Depend or Recommend a package named lib.*-ruby.*.
  • The only packages installing stuff to /usr/lib/ruby/{1.8,1.9.1} are interpreters. Software packaged with gem2deb will install to /usr/lib/ruby/vendor_ruby/
  • Upgrades from squeeze work. ;)

  • The Ruby community stops whining about Debian ;-) (just kidding. that's not really an achievable goal)

Howto: converting a package from ruby-pkg-tools to gem2deb

This section describes how to convert the libfeedparser-ruby package to gem2deb. That package is simple, but still gives a good overview of the process. Please improve this section with your own experience.

  1. Get the libfeedparser-ruby source package. (apt-get source libfeedparser-ruby)
  2. Find the name of the rubygems on rubygems.org. It is ruby-feedparser.
  3. Run gem2deb ruby-feedparser. This generates a basic (but working) source package.
  4. Cd to ruby-feedparser-0.7/
  5. Copy the changelog entries from the libfeedparser-ruby source package to debian/changelog
  6. Generate the templates for transitional packages: run gen-ruby-trans-pkgs libfeedparser-ruby > /tmp/templates. Edit debian/control: the Replaces, Breaks, Provides go to the ruby-feedparser binary package, and the other binary packages need to be added at the end of debian/control.

  7. Fill-in debian/control: description, homepage, build-dependencies, ...
  8. Copy debian/copyright from libfeedparser-ruby, and review it. It is a good idea to use that opportunity to convert it to DEP5.
  9. Edit ruby-feedparser.docs
  10. Find how the test suite needs to be run, and edit debian/ruby-tests.rb accordingly. There are many examples in the packages maintained by the pkg-ruby-extras team.
  11. Build the package, make sure everything works (build in a clean chroot, run lintian, etc, etc).
  12. Import it into the pkg-ruby-extras git repository.
  13. Ask for review and sponsorship.

Requesting Sponsorship

Once your package is ready (or at least, once you think it is), a DD will have to review your changes and upload it to the archive. It is important to understand that reviewing and sponsoring packages is a tedious process, and that most DDs would prefer to do something else instead. So, make sure that you make the best possible use of their time, so they will be happy to sponsor you again. :-)

First, make sure that your package is really in good shape. That means answering at least the following questions:

  • Does my package build fine in a clean chroot? The easiest way to test that is using pbuilder.

  • Is my package lintian-clean? If not, did I explain why it isn’t in debian/changelog? Note that real problems should not be hidden by lintian overrides.
  • Did I check that all usages of rubygems have been patched out?
  • Is my debian/watch file correct? That file will be used by your sponsor to fetch the upstream source, so it is very important that it is correct. Please check that uscan --download-current-version does the right thing.

  • Is the Git repo up-to-date? Did I commit and push my changes?
  • Does my package actually work? Please check in a clean chroot, it’s easy to miss some @require@s.

Once your package is ready, you must:

  • Change the distribution from UNRELEASED to unstable.

  • Request sponsorship by mailing debian-ruby@lists.debian.org. Here is an example email:

Subject: RFS: libfoo-ruby 1.2-1, libbar-ruby 2.0-1

Hi,

The following packages are ready to be uploaded (I also verified the
points listed on http://wiki.debian.org/Teams/Ruby/Packaging#Requesting_Sponsorship).

Could you please sponsor them?

 libfoo-ruby 1.2-1
 libbar-ruby 2.0-1

Thank you!