WORK IN PROGRESS!!
Contents
Using the tool gem2deb along with debhelper is the recommended way to build Debian packages from Ruby gems. It automates almost all the steps and provides a standard and consistent way of building packages.
Prerequisites
Go through Joining Debian Ruby Team and get yourself added to the alioth group.
- Install the packages gem2deb, quilt, git-buildpackage using the command
Note: $ is used to denote terminal prompt and is not part of the command
$ sudo apt-get install gem2deb quilt git-buildpackage
Packaging Conventions
Various packaging conventions that are followed by the Debian Ruby Team can be found here
Steps involved in packaging
Preparing your environment
0. Use Debian Ruby team's helper script
Debian Ruby team have created a setup script that installs and configures the basic tools required for packaging Ruby gems. To use it, follow the steps
- (a). Clone the Team's master repository
$ git clone <alioth username>@git.debian.org:/git/pkg-ruby-extras/pkg-ruby-extras.git
- (b). Change to the cloned repo
$ cd pkg-ruby-extras
- (c). Run the setup script and follow the instructions. This may take time depending upon your connectivity.
$ ./setup
1. Add the following to ~/.bashrc (or ~/.zshrc depending on your shell)
export DEBEMAIL=your@email.domain export DEBFULLNAME='Your Name' alias lintian='lintian -iIEcv --pedantic --color auto'
2. For the changes to take effect, run the following command (change .bashrc to one specific to your shell)
$ source ~/.bashrc
3. Add the following lines to ~/.quiltrc
QUILT_PATCHES=debian/patches QUILT_NO_DIFF_INDEX=1 QUILT_NO_DIFF_TIMESTAMPS=1 QUILT_REFRESH_ARGS="-p ab" QUILT_DIFF_ARGS="--color=auto"
4. Add the following lines to ~/.ssh/config, replacing username-guest with your actual salsa username:
Host git.debian.org salsa.debian.org User username-guest
Identifying the gem and filing ITP
Most of the gems (exceptions mainly include rails-assets-* gems) use Rubygems as a central repository for distribution. A Rubygems page contain many basic information like Runtime dependencies, Development dependencies, License, Authors, Homepage link etc.
Checking if someone is already working on the package
Before starting packaging, we have to make sure that no one else is currently working on the package, so that your effort will not be wasted for duplication. For that, make use of the Work-Needing and Prospective Packages page and search for the package name under ITP option. If any results turn up, it means someone else is currently working on the package and you should probably contact and co-ordinate with them before continuing.
Filing ITP
When you are intending to prepare a package for Debian, it is customary that you file an ITP (stands for 'Intend To Package') bug informing the Debian world about your work. As explained earlier, this will help to avoid duplication of work.
To file an ITP, you can either use reportbug tool or send a mail to submit@bugs.debian.org with a specific format
- The subject of the mail should be like as follows
ITP: ruby-flowdock -- A gem that offers flexible authentication solution for Rails with Warden
Here, ruby-flowdock is the package name, followed by a space, two dashes and a one line description of the package.
- The content of the mail should be similar to following
package: wnpp Severity: wishlist Owner: 'Your Name' <yourid@example.domain> *Package Name : ruby-flowdock Version : 0.7.1 Upstream Author : Antti Pitkänen (Author name/s of the Gem). *URL : http://github.com/flowdock/flowdock-api (Link to the homepage or git repo of the Gem) *License : Expat *Description : Gem for using Flowdock's API <Description and why you are packaging the gem>
The last line of the content, that is an extended description and why you are packaging the gem (if it is a dependency of some other software, if it is related to some other bug etc) is optional and can be skipped.
The preferred way is to use the reportbug program which provides a template document that you can simply edit and use.
Initial package creation using gem2deb
Note: In this tutorial, it is assumed that you create a directory with your gem's name (example: flowdock) and is performing the following steps while inside that directory. This directory will be called 'root directory' in this tutorial.
As told earlier, gem2deb program is the Debian Ruby team's preferred way of doing packaging. To use it, initiate the following command on a terminal
$ gem2deb <gem name>
Example : $ gem2deb flowdock
This command will
- Fetch the gem from rubygems page
- Create source tarball from the gem
- Create Debian source package from the tarball
- Build the package
Note: If your build gave any errors like Unmet build dependencies, install those packages and try gem2deb again.
Note: In case you get an error like "Tests failed" during build stage, ignore for now (and continue pressing Y, if asked)
If you check the contents of the current directory, you can find
a folder ruby-flowdock-<version number>
a flowdock_<version>.tar.gz file
- a ruby-flowdock-version.orig.tar.gz file which is a symlink to the previous tarball.
If the build was succesful, you will also find
- a .dsc file
- a .changes file
- a .deb file and a .debian.tar.xz file which are the results of build.
Using git for packaging
Debian Ruby team follows a git based packaging workflow making use of ?gbp and alioth repos. So, you will be doing all your packaging inside a git repo with three branches
- master - where the source code + packaging related stuff is stored
- upstream - where the upstream source code is stored
- pristine-tar - containing delta files that are needed for package regeneration from git
You make all your package specific changes to the master branch.
1. To convert your package to a git-based directory structure, you can make use of the tool http://tracker.debian.org/git-buildpackage and issue the following command
$ gbp import-dsc --author-is-committer --pristine-tar <path to the dsc file created by gem2deb>
This will create a folder named ruby-<package name>, which will be referred as source directory in this tutorial.
2. Change to the source directory and try the following command to make sure all three branches have been created
$ git branch
3. If you give the following command you can find the tags applied to the repo
$ git tag
4. Here, we have to delete the debian/<version>-<revision> tag as that is used to denote the state of the repository when a package enters Debian archive. Since we are only starting work with this package and may make many changes before it is uploaded to debian, that tag should be deleted and recreated when the package actually gets accepted to debian. To delete the tag, do
$ git tag -d debian/<version>-<revision>
Editing basic package information
You will have to edit the information regarding package in the files like control, copyright, changelog etc inside the debian directory. Even though new versions of gem2deb automatically does most of the job for you (except copyright file), it is necessary that you manually verify the files for perfection.
To understand what files under debian directory are and how they work, refer Debian New Maintainer's Guide Chapter 4 and 5
Building
You can build your package using the toold ?debuild. To use that, make sure you are in the source directory and execute the following command
$ debuild
This command will
- Build the package using dpkg-buildpackage
- Run lintian on the changes file generated (it will be outside the source directory, that is inside the root directory)
- Sign the changes and dsc file using debsign
Lintian
Lintian dissects Debian packages and reports bugs and policy violations. It contains automated checks for many aspects of Debian policy as well as some checks for common errors. Lintian can be used to find out problems about your package. Even though it will be automatically run during debuild, you can manually run it using the command
$ lintian <path to .changes file generated by debuild>
As a package maintainer, it is your duty to make sure that your package builds fine and is lintian-happy. Some warnings from lintian that are normal and may be skipped can be found ?here.
After you make significant change to the package, run debuild again to see if everything is working fine.
Running tests and identifying failures
In Debian packages of Ruby gems, tests are automatically run using special scripts with ruby-tests.{rake|rb|yaml} which are present inside the debian subdirectory. New versions of gem2deb automatically identifies the test directory structure and generates either of these file to automatically run tests during package building.
Tests sometimes fail while running and need to be fixed so that the building happens without any issues in a clean chroot. To know about various scenarios associated to running tests and the common test failures, visit Tests.
To solve Test Failures, you may have to edit the upstream code and generate patches that should be applied before the buildin begins. In Debian Ruby team, the recommended way of creating patches is using the tool quilt.
Using quilt
As described earlier, quilt is a tool to make changes to upstream source code and store the changes as patch files that may get applied before the package building begins. This is recommended because it maintains the patches as separate files and the upstream code can be maintained pristine. The steps to patch a file (for eg: spec/spec_helper.rb) are
1. Run the command to create a new patch and put it on top of the patch series. The name of the patch should probably be sensible and descriptive about what the patch do
$ quilt new remove-bundler.patch
2. Add the files that you intend to edit to the patch using
$ quilt add spec/spec_helper.rb
3. Now, open the file using any text editor and make necessary changes and save the file.
4. Refresh the patch so that the changes you applied gets added to it by
$ quilt refresh
- (i). It is recommended that your patch have a DEP-3 header. To generate a DEP-3 header for the patch, execute the command
$ quilt header -e --dep3
- and fill the fields appropriately. Save and close the file.
5. To revert the changes present in patch, do
$ quilt pop
6. Check the directory debian/patches to find the patch file along with a series file. The series file specifies the order the patches are to be applied.
Testing in a clean chroot environment
In order to imitate Debian's buildd system, it is recommended to use sbuild for building packages in clean chroot. If you use the Debian Ruby Team's helper script to setup your environment, you will have sbuild properly configured to work.
To use sbuild, change to the source directory and execute the command
$ sbuild
sbuild does the following tasks :-
1. Build the package in a clean chroot environment where the minimal package set + Build Depends packages are installed. It makes sure that the build uses Debian packages of ruby gems instead of the ones installed using some other method.
2. Run lintian on the package and displays its output
If sbuild reports errors, try to fix them modifying files inside debian subdirectory. For example, if sbuild fails to run tests because of gems missing, you probably wouldn't have added the corresponding packages to Build-Depends in debian/control file. Similarly, make sure the package builds fine in a clean chroot and lintian is happy with the result.
Note: In the logs of sbuild, the files that are part of the package are listed. Do make sure the package's gemspec file is included in that. If it is not, check the log for the step "Install Rubygems integration metadata" where the possible error will be listed. A common error is that gemspec file uses the git command for working and it will fail if the directory is not a git repository (which is the case in sbuild). So, the gemspec file has to be patched using quilt to fix that.
Uploading to alioth
Debian Ruby team maintains an alioth repo where the team-maintained packages are stored. Any member of the Alioth group of team is able to push to these repos. The steps involve
1. Login to the server using the command
$ ssh git.debian.org
2. Change to the Debian Ruby team directory
$ cd /git/pkg-ruby-extras/
3. Create a repository for your package
$ ./setup-repository <package name>
example: ./setup-repository ruby-flowdock
4. Log off from the server by pressing Ctrl-D or the command
$ logout
5. Add the alioth repo as remote to your local packaging repository
$ git remote add origin <your-username>@git.debian.org:/git/pkg-ruby-extras/<package name>.git
6. Make sure you have committed all your changes to the local repository and push to the remote repository
$ git push origin --all
7. Push the tags as well
$ git push origin --tags
Once you pushed all your packaging work to the alioth repo, you need some Debian Developer who have official uploading permission to sponsor your package.
Requesting sponsorship
Upload permission on Debian is limited to Debian Developers and Debian Maintainers (only specific packages). So, your package has to be sponsored by a Debian Developer. To request for sponsorship, send a mail to the team mailing list at debian-ruby@lists.debian.org with the following properties
- Subject of the mail should be as follows
RFS: <package name>
Example: RFS: ruby-flowdock
- Content of the mail should be similar to the following (the OPTIONAL tag and the content between them, as the name suggests, are optional so that the DD get an idea about the package and its necessity)
Hi, I've prepared the Debian package of the Ruby gem <gem name>. The package was tested on sbuild and was successfully built. It is also lintian-clean. I've uploaded the package to the salsa repo which may be found at https://anonscm.debian.org/git/pkg-ruby-extras/<package name>.git <OPTIONAL> Add a brief description about why you packaged it - like the package being dependency of another package, part of a long dependency chain etc. </OPTIONAL> Please consider to review and upload it. <Name>
An example RFS mail is as follows:
Hi, I've prepared the Debian package of the Ruby gem flowdock. The package was tested on sbuild and was successfully built. It is also lintian-clean. I've uploaded the package to the salsa repo which may be found at https://anonscm.debian.org/git/pkg-ruby-extras/ruby-flowdock.git The package is a member of the Runtime dependency chain of GitLab, and is thus required for completing the packaging of GitLab. Please consider to review and upload it. Balasankar C