Another effort to document the workflow of packaging ruby-gems for Debian is made by the debian-diaspora Team at: this wiki page, which is targeting beginners and very detailed.
- Joining the Debian Ruby team
- Using Git
- Guidelines for Ruby packaging
- Tasks for team-maintained Ruby packages
- Requesting Sponsorship
Joining the Debian Ruby team
We always welcome people to help us out. If you want to, please follow the following guidelines.
Contact us first on our mailing list or on IRC - #debian-ruby on irc.oftc.net (If you are using Matrix or XMPP, there are bridges to IRC which you can use to join IRC channels from your matrix or xmpp clients).
- Send contributions via merge requests on salsa. One or more of the existing team members will review your contributions.
- once you have made enough contributions and you are confident that you will stay around, request to be added as team member on salsa.
- Read all the docs (sorry).
We use git together with myrepos (Multiple Repository management tool) to track the git repositories for each package.
Before you continue, ensure you have installed the tools used by the team:
apt-get install myrepos git-buildpackage pristine-tar gem2deb
The following workflow is used for (almost) all non-native packages maintained by the Ruby team:
- git-buildpackage as a building tool
- 3 main branches:
pristine-tar branch to store tarballs generated by gem2deb or by the gemwatch service
upstream branch tracking unpacked upstream tarballs
master branch containing upstream + the debian/ repository (with patches unapplied).
- possibly other branches for backports or stable/testing updates, with specific debian/gbp.conf configuration
- wheezy-backports replacing master for backports
master-<codename>, upstream-<codename> for updates targetted to the particular version <codename>
Obtaining the master repository
git clone email@example.com:ruby-team/meta.git ruby-team
To obtain one of the packages, do:
cd ruby-team ./checkout $PACKAGE
If you want to obtain all of the team packages (have in mind that the team has a lot of packages, so this will probably take a little long), do:
# look at .mrconfig mr --force checkout
To clone all team packages with parallel checkouts, use
mr --force -j 5 checkout
To avoid typing your username and your password see this ?page.
If you don't want to use mr, you can clone individual repos:
gbp clone --pristine-tar firstname.lastname@example.org:ruby-team/<pkg-name>.git
Creating a new Git repository and pushing an existing package to salsa.debian.org
Note: please do not leave empty repositories on salsa; only create a repository there just before pushing your package.
Suppose you want to package the latest 0.7 version of the foo Ruby gem, to obtain a ruby-foo package. See the naming conventions for packages.
On your local workstation:
cd ruby-team ./setup-project
Packaging new gems from scratch
Note: See Teams/Ruby/Packaging/gem2deb if you are new to ruby packaging
On your local machine, get a basic .dsc from the library you want to package, using:
gem2deb will then try to download the foo gem from rubygems.org, and convert it to a primitive Debian package ruby-foo_0.7-1.dsc. You can also run gem2deb on a local .gem file foo-0.7.gem or a tarball (e.g. from Github).
At this point, please do not make any change to the package generated by gem2deb. It is important to track all the changes in git. Then:
cd where-you-cloned-pkg-ruby-extras.git ./update-mrconfig # will generate the config for your new repo ./checkout ruby-foo # will clone your new repo cd ruby-foo gbp import-dsc --pristine-tar path-to/ruby-foo_0.7-1.dsc git tag -d debian/0.7-1 # because the package is not ready yet git push --all git push --tags
Then, make the needed changes to the packaging, and then
After upload, tag the package with
git tag debian/0.7-1
And don't forget to
git push --tags
If mr checkout gives an error like mr: illegal section "[DEFAULT]" in untrusted:
Add .mrconfig path to ~/.mrtrust
If your local username is different from your Alioth username (that's probably the case for non-DD developers with -guest accounts), add the following to your ~/.ssh/config:
Host alioth.debian.org svn.debian.org git.debian.org User alioth-username
Guidelines for Ruby packaging
gem2deb as the preferred packaging tool for ruby software
gem2deb is the current packaging tool for Debian Ruby packages. It:
- does almost everything automatically
- uses dh
- is easier to adapt to our needs, since there's no dependency on an external tool
runs the test suite as part of the build, for each ruby implementation (yes, it's possible to override this). See more about running tests
- uses a single binary package for native libraries, instead of one binary package for each ruby implementation
It serves two goals:
- it can be used by users to generate .debs from gems locally
- it generates Debian source package from which we can do the packaging work
gem2deb can be considered to some extent as the reference implementation of the Debian Ruby policy. See Teams/Ruby/Packaging/gem2deb for using gem2deb to create new gem packages.
Switching to github tar and other changes to orig-source
Guidelines for Packaging Ruby on Rails Applications
Multiple gemspec files
You can create debian/gemspec as a symbolic link to the gemspec file you want to use. You can also set DH_RUBY_GEMSPEC variable in debian/rules. For autopkgtest, you should remove the Testsuite field in debian/control and create debian/tests manually.
Example packages: ruby-concurrent, ruby-flipper, ruby-elasticsearch
Handling Rails Engines
See Teams/Ruby/Packaging/RailsEngines for packaging such gems.
Tasks for team-maintained Ruby packages
Updating a package to a newer version
First make sure you got a local repository from salsa which has the pristine-tar branch:
gbp clone --pristine-tar email@example.com:ruby-team/<pkg-name>.git
If you have a working debian/watch file (check by running uscan --verbose --report), you can simply import a new version with
gbp import-orig --pristine-tar --uscan
If you want a different version than latest version, you may have to download it manually using a web browser / wget or use uscan --verbose --download-version option and run gbp import-orig --pristine-tar separately.
Alternatively, you can fetch the most recent version of a gem, and make a tarball out of it
where <gemname> is the upstream name of the gem, or run gem2tgz on a local .gem file.
You can also get a specific version and then create the tgz by doing
gem fetch -v <gemname> gem2tgz <gemname>.gem
Then import the created tarball in the Git repository with gbp import-orig --pristine-tar.
Option 1 (gbp dch):
You can use the following command,
gbp dch -a
Verify the version in changelog is updated, if the version is not updated, update it manually (if there is an UNRELEASED entry when running this command, the upstream version won't be updated automatically).
Option 2 (dch):
Or use dch for it, it will give you a template:
dch -v <new-upstream-version>-1
Insert this as the first entry:
* New upstream Release
Then try to build the package like usual and make corrections and changes which are needed.
- And add the important changes you made for the new release (removing a patch, inserting a new one, etc). Don't insert changes made by upstream in debian/changelog.
Once you are done, change UNRELEASED to unstable (or experimental in case of breaking changes) and push the branches and the tags to the team repository
git push -u --all --follow-tags
Install the package and see if all dependencies are satisfied for rubygems integration by creating a Gemfile for the library and running bundle install --local.
$ echo "gem '<library name>', '<version>'" > Gemfile $ bundle install --local
Updating packages with API breaking changes
Read this transitions primer if you are new to handling transitions.
At least https://SemVer.org compliance should be assumed (even though many upstreams don't follow it and breaking changes are introduced in even security updates, for example rack 2.0.7 -> 2.0.8 is not compatible) as a compromise (as assuming every change breaks is not practical). This means major version updates of upstream with public API (when version >= 1.0) and minor updates of upstream without a public API (when version < 1.0). For example ruby-method-source update from 0.8 to 1.0 and pry update from 0.12 to 0.13, both are breaking changes (sometimes we may be lucky and nothing breaks, but we cannot assume that when upstream explicitly gives a warning by appropriate version bumps).
When uploading such API breaking changes these steps must be followed,
Verify autopkgtests of all reverse dependencies and rebuilds of all reverse build dependencies are passing. Use Packaging/ruby-team-meta-build; or upload to experimental and watch the britney pseudo-excuses for experimental
- There are two paths you can take for uploading to unstable,
- Ideally all breaking packages should be fixed before uploading to unstable. Fixed packages should be uploaded along with the package that introduced API breakage.
- Upload to experimental and file bug reports with severity important against all failed packages. It is a good idea to document the list of packages that failed in debian/changelog. Give sufficient time for the maintainers (at least a week for simple fixes or more for complex changes) to fix the broken packages.
- When all the broken packages are fixed or the provided time is passed, upload to unstable along with all the fixed packages. Add Breaks for all the packages that failed. The bugs against unfixed packages should be raised to serious.
Exceptions (be careful with minor updates also/minor updates broke API in the past): rails, ruby-rack, ruby-doorkeeper, ruby-devise, ruby-graphql, ruby-grape
Note: Bigger transitions like ruby or rails may need collaboration from the whole team and new versions may be uploaded to unstable and broken packages may be fixed later. Try to coordinate with the team on the mailing list.
Once your package is ready (or at least, once you think it is), a DD will have to review your changes and upload it to the archive. It is important to understand that reviewing and sponsoring packages is a tedious process, and that most DDs would prefer to do something else instead. So, make sure that you make the best possible use of their time, so they will be happy to sponsor you again.
First, make sure that your package is really in good shape. That means answering at least the following questions:
Does my package build fine in a clean chroot? The easiest way to test that is using sbuild or pbuilder.
- Is my package lintian-clean? If not, did I explain why it isn’t in debian/changelog? Note that real problems should not be hidden by lintian overrides.
Is my debian/watch file correct? That file will be used by your sponsor to fetch the upstream source, so it is very important that it is correct. Please check that uscan --download-current-version does the right thing.
- Is the Git repo up-to-date? Did I commit and push my changes and tags?
- Does my package actually work? Please check in a clean chroot, it’s easy to miss some dependencies.
- Is the test suite enabled? are all the tests passing?
- Is autopkgtest passing in a clean chroot? ($ autopkgtest -B ../foo.deb -- schroot unstable-amd64-sbuild)
Are you updating an existing package? Did you check if it is a breaking change? If it is a breaking change as per https://semver.org, follow Teams/Ruby/Packaging#Updating_packages_with_API_breaking_changes
Once your package is ready, you must:
Change the distribution from UNRELEASED to unstable.
Request sponsorship by mailing firstname.lastname@example.org. Here is an example email:
Subject: RFS: ruby-foo 1.2-1, ruby-bar 2.0-1 Hi, The following packages are ready to be uploaded (I also verified the points listed on http://wiki.debian.org/Teams/Ruby/Packaging#Requesting_Sponsorship). <Describe the reason behind updating the package.> Could you please sponsor them? ruby-foo 1.2-1 ruby-bar 2.0-1 Thank you!