Differences between revisions 90 and 91
Revision 90 as of 2016-11-11 13:55:08
Size: 16836
Comment: fix name of mrconfig update script
Revision 91 as of 2016-11-15 14:56:01
Size: 16838
Comment: fix quoting
Deletions are marked like this. Additions are marked like this.
Line 270: Line 270:
$ echo gem '<library name>', '<version>' > Gemfile $ echo "gem '<library name>', '<version>'" > Gemfile

This page describes the current packaging practices for Ruby in Debian. For more general information, see this wiki page.

This page is quite disorganized, and some of it was written as a "plans for Ruby in wheezy page", not as a general documentation for packaging. Please do not hesitate to improve this page. Another effort to document the workflow of packaging ruby-gems for Debian is made by the debian-diaspora Team at: this wiki page , which is focused at beginners and very detailed.

Joining the pkg-ruby-extras team

We always welcome people to help us out. If you want to, please follow the following guidelines.

  • Contact us first on our mailing list or on IRC.

  • Register an alioth account account.

  • Go to our project page and request your addition to the team. Please specify what you plan to work on.

  • Wait until we approve your addition. Note that there’s a delay before getting your account on svn.debian.org, see the Alioth FAQ for details.

  • Read all the docs (sorry).
  • Enjoy!

Using Git

We use git together with myrepos (Multiple Repository management tool) to track the git repositories for each package.

Before you continue, ensure you have installed the tools used by the team:

apt-get install myrepos git-buildpackage pristine-tar gem2deb

Git workflow

The following workflow is used for (almost) all non-native packages maintained by the Ruby team:

  • git-buildpackage as a building tool
  • 3 main branches:
    • pristine-tar branch to store tarballs generated by gem2deb or by the gemwatch service

    • upstream branch tracking unpacked upstream tarballs

    • master branch containing upstream + the debian/ repository (with patches unapplied).

  • possibly other branches for backports or stable/testing updates, with specific debian/gbp.conf configuration
    • wheezy-backports replacing master for backports
    • master-<codename>, upstream-<codename> for updates targetted to the particular version <codename>

Obtaining the master repository

git clone <alioth-username>@git.debian.org:/git/pkg-ruby-extras/pkg-ruby-extras.git

If you are new to alioth/waiting for ssh keys to be added, try https url

git clone https://anonscm.debian.org/git/pkg-ruby-extras/pkg-ruby-extras.git

To obtain one of the packages, do:

cd pkg-ruby-extras
./checkout $PACKAGE

If you want to obtain all of the team packages (have in mind that the team has a lot of packages, so this will probably take a little long), do:

# look at .mrconfig
mr --force checkout

To clone all team packages with parallel checkouts, use

mr --force -j 5 checkout

To avoid typing your username and your password see this ?page.

Creating a new Git repository and pushing an existing package to git.debian.org

Note: please do not leave empty repositories on alioth; only create a repository there just before pushing your package.

Suppose you want to package the latest 0.7 version of the foo Ruby gem, to obtain a ruby-foo package. See the naming conventions for packages.

On git.debian.org:

cd /git/pkg-ruby-extras
./setup-repository ruby-foo

On your local machine, get a basic .dsc from the library you want to package, using:

gem2deb foo

gem2deb will then try to download the foo gem from rubygems.org, and convert it to a primitive Debian package ruby-foo_0.7-1.dsc. You can also run gem2deb on a local .gem file foo-0.7.gem or a tarball (e.g. from Github).

At this point, please do not make any change to the package generated by gem2deb. It is important to track all the changes in git. Then:

cd where-you-cloned-pkg-ruby-extras.git
./update-mrconfig # will generate the config for your new repo
./checkout ruby-foo # will clone your new repo
cd ruby-foo

gbp import-dsc --pristine-tar path-to/ruby-foo_0.7-1.dsc
git tag -d debian/0.7-1 # because the package is not ready yet
git push --all
git push --tags

Then, make the needed changes to the packaging, and then

git push

After upload, tag the package with

git tag debian/0.7-1

And don't forget to

git push --tags

Known Issues

If mr checkout gives an error like mr: illegal section "[DEFAULT]" in untrusted:

Add .mrconfig path to ~/.mrtrust

If your local username is different from your Alioth username (that's probably the case for non-DD developers with -guest accounts), add the following to your ~/.ssh/config:

Host alioth.debian.org svn.debian.org git.debian.org
User alioth-username

Handling patches on upstream code

If the package you are working on includes upstream patches (i.e. debian/patches/*), you need to make sure that the Git repository is in a sane state after build (i.e. no uncommitted changes to the work tree).

Since 1.16.1, dpkg-source unapplies by default patches that have been applied during --before-build. For earlier versions, it is possible to force this behaviour by adding the unapply-patches option to debian/source/local-options (source: blog post by Raphael Hertzog)

$ echo "unapply-patches" >>debian/source/local-options
$ git add debian/source/local-options
$ git commit -m "Unapply patches after build"

References

Guidelines for Ruby packaging

gem2deb as the preferred packaging tool for ruby software

gem2deb is the current packaging tool for Debian Ruby packages. It:

  • does almost everything automatically
  • uses dh
  • is easier to adapt to our needs, since there's no dependency on an external tool
  • runs the test suite as part of the build, for each ruby implementation (yes, it's possible to override this). See more about running tests

  • uses a single binary package for native libraries, instead of one binary package for each ruby implementation

It serves two goals:

  • it can be used by users to generate .debs from gems locally
  • it generates Debian source package from which we can do the packaging work

gem2deb can be considered to some extend as the reference implementation of the Debian Ruby policy.

Single package for all Ruby versions

Since Wheezy, Ruby software must not come in separate packages per Ruby interpreter version, as we did until Squeeze (libfoo-ruby1.8, libfoo-ruby1.9.1 etc). We use a single binary package per source package no matter what. In the case of native extensions, those single binary packages will contain the compiled extensions (*.so) for all supported Ruby versions. This is supported by gem2deb, so you should use it.

Naming of ruby packages

The libfoo-ruby naming is deprecated.

The guidelines are the following:

  • Binary packages must normally be named "ruby-foo". If the package is mainly used as an application (not as a library), then it can be named "foo". Known examples are rails, chef, bundler, puppet.
  • Source packages must have the same name as the "main" binary package. (our infrastructure is better at handling this case)

Handling of shebangs

Ruby applications should use /usr/bin/ruby, and depend on ruby | ruby-interpreter.

gem2deb currently does the following:

  • If the XS-Ruby-Versions field contains 'all', all shebangs of programs installed to /usr/bin are rewritten to /usr/bin/ruby

  • If several versions of the interpreter were supported by gem2deb (as it was the case for Wheezy), all shebangs of programs installed to /usr/bin would be rewritten to the binary corresponding to the first version listed in XS-Ruby-Versions. For example, is XS-Ruby-Versions is ruby2.1, ruby2.2, all shebangs would be rewritten to /usr/bin/ruby2.1

Values of XS-Ruby-Versions

The XS-Ruby-Versions values match the name of the ruby interpreter, or all if the package is supposed to work with every supported version of the interpreter.

For Wheezy, supported values were:

  • all
  • ruby1.8
  • ruby1.9.1

For Jessie, only one version of the interpreter is included. The valid values are:

  • ruby2.1
  • all

For Stretch, only one version of the interpreter will be included. The valid values are:

  • ruby2.2 (temporary in unstable)
  • ruby2.3
  • all

When more ruby interpreters will be added, the list will be expanded.

Requesting Sponsorship

Once your package is ready (or at least, once you think it is), a DD will have to review your changes and upload it to the archive. It is important to understand that reviewing and sponsoring packages is a tedious process, and that most DDs would prefer to do something else instead. So, make sure that you make the best possible use of their time, so they will be happy to sponsor you again. :-)

First, make sure that your package is really in good shape. That means answering at least the following questions:

  • Does my package build fine in a clean chroot? The easiest way to test that is using pbuilder.

  • Is my package lintian-clean? If not, did I explain why it isn’t in debian/changelog? Note that real problems should not be hidden by lintian overrides.
  • Is my debian/watch file correct? That file will be used by your sponsor to fetch the upstream source, so it is very important that it is correct. Please check that uscan --download-current-version does the right thing.

  • Is the Git repo up-to-date? Did I commit and push my changes and tags?
  • Does my package actually work? Please check in a clean chroot, it’s easy to miss some dependencies.
  • Is the test suite enabled? are all the tests passing?

Once your package is ready, you must:

  • Change the distribution from UNRELEASED to unstable.

  • Request sponsorship by mailing debian-ruby@lists.debian.org. Here is an example email:

Subject: RFS: ruby-foo 1.2-1, ruby-bar 2.0-1

Hi,

The following packages are ready to be uploaded (I also verified the
points listed on http://wiki.debian.org/Teams/Ruby/Packaging#Requesting_Sponsorship).

Could you please sponsor them?

 ruby-foo 1.2-1
 ruby-bar 2.0-1

Thank you!

Tasks for team-maintained Ruby packages

Updating a package to a newer version

If you have a working debian/watch file (check by running uscan --verbose --report), you can simply import a new version with

gbp import-orig --pristine-tar --uscan

Alternatively, you can fetch the most recent version of a gem, and make a tarball out of it

gem2tgz <gemname>

where <gemname> is the upstream name of the gem, or run gem2tgz on a local .gem file. Then import the created tarball in the Git repository with gbp import-orig --pristine-tar.

Now update your packaging in the master branch. Once you are done, push the branches and the tags to the team repository

git push --all
git push --tags

Install the package and see if all dependencies are satisfied for rubygems integration by creating a Gemfile for the library and running bundle install --local.

$ echo "gem '<library name>', '<version>'" > Gemfile
$ bundle install --local

See steps of updating rails apps like diaspora, gitlab and redmine

Howto: converting a package from ruby-pkg-tools to gem2deb

This section describes how to convert the libfeedparser-ruby package to gem2deb. That package is simple, but still gives a good overview of the process. Please improve this section with your own experience.

  1. Get the libfeedparser-ruby source package. (apt-get source libfeedparser-ruby)
  2. Find the name of the rubygems on rubygems.org. It is ruby-feedparser.
  3. Run gem2deb ruby-feedparser. This generates a basic (but working) source package.
  4. Cd to ruby-feedparser-0.7/
  5. Copy the changelog entries from the libfeedparser-ruby source package to debian/changelog
  6. Generate the templates for transitional packages: run gen-ruby-trans-pkgs libfeedparser-ruby > /tmp/templates. Edit debian/control: the Replaces, Breaks, Provides go to the ruby-feedparser binary package, and the other binary packages need to be added at the end of debian/control.

  7. Fill-in debian/control: description, homepage, build-dependencies, ...
  8. Copy debian/copyright from libfeedparser-ruby, and review it. It is a good idea to use that opportunity to convert it to DEP5.
  9. Edit ruby-feedparser.docs
  10. Find how the test suite needs to be run, and edit debian/ruby-tests.rb accordingly. There are many examples in the packages maintained by the pkg-ruby-extras team.
  11. Build the package, make sure everything works (build in a clean chroot, run lintian, etc, etc).
  12. Import it into the pkg-ruby-extras git repository.
  13. Ask for review and sponsorship.

Renaming existing packages using the old naming convention

Existing packages must be renamed to the new scheme. This renaming must be done using the standard Debian practices (i.e. Debian Developers' Reference, section 5.9.3). In our case, this means:

  • when libfoo-ruby gets renamed to ruby-foo, then ruby-foo must declare Provides: libfoo-ruby, Replaces: libfoo-ruby and Breaks: libfoo-ruby

  • when libfoo-ruby1.8 gets renamed to ruby-foo, then ruby-foo must declare Provides: libfoo-ruby1.8, Replaces: libfoo-ruby1.8, and Breaks: libfoo-ruby1.8

  • when libfoo-ruby1.8 and libfoo-ruby1.9.1 are turned into a single pure-Ruby package ruby-foo, then it must declare Provides: libfoo-ruby, libfoo-ruby1.8, libfoo-ruby1.9.1, Replaces: libfoo-ruby, libfoo-ruby1.8, libfoo-ruby1.9.1, and Breaks: libfoo-ruby, libfoo-ruby1.8, libfoo-ruby1.9.1

  • ruby-foo must also provide transitional binary packages for libfoo-ruby, libfoo-ruby1.8 and libruby1.9.1 (when it's the case). You can use the gen-ruby-trans-pkgs script to generate the proper snippet to be added to your debian/control.

  • note that this is only needed for packages that were already released with Squeeze. New packages must not keep any references to packages named as lib*-ruby.
  • According to Renaming_a_Package the old source package will be removed automatically after the renamed package enters the archive. If that doesn't happen after a reasonable amount of time, please request the removal of the old package.

Removing transitional packages

For packages that have been renamed before the release of Wheezy from the libsomething-ruby* scheme to ruby-something, all transitional packages libsomething-ruby* can be removed for Jessie, after checking that they have no more reverse (build-)dependencies.

For packages that have not yet been renamed by the Wheezy release, the transitional packages must remain in place until after the Jessie release.

Packages with transitional packages to be dropped have bugs reported with the usertag ruby-policy-remove-transitional-packages.

Enabling autopkgtest

Since version 0.9, gem2deb has gained support for autopkgtest. See this message to the debian-ruby mailing list for more details on how to activate and configure the test suite for autopkgtest.

Check and fix location of rubygems-integration files

Some packages do not have a new version but still need to be reuploaded, because the rubygems metadata are installed in an obsolete directory (e.g /usr/share/rubygems-integration/1.8). Tracking and fixing packaging with files in /usr/share/rubygems-integration/{1.*,2.0} is needed before Jessie freeze. Use apt-file to find those packages, and if no new upstream version is available, refresh the packaging and reupload.


PackageManagement CategoryPackaging