Differences between revisions 15 and 16
Revision 15 as of 2018-04-07 17:00:37
Size: 13505
Editor: ElanaHashman
Comment: Updates for salsa and clj-helper
Revision 16 as of 2019-07-09 19:38:10
Size: 13681
Comment: Leiningen has been packaged; begin "Leiningen Helper" section as a stub
Deletions are marked like this. Additions are marked like this.
Line 12: Line 12:
Clojure packages must be built using [[http://pkg-java.alioth.debian.org/docs/tutorial.html|JavaHelper]] and [[Java/MavenRepoHelper|MavenRepoHelper]] or [[Java/MavenDebianHelper|MavenDebianHelper]] until [[Clojure/Leiningen|Leiningen]] is packaged and we've developed a "Leiningen Helper" for Clojure packaging. Clojure packages must be built using [[http://pkg-java.alioth.debian.org/docs/tutorial.html|JavaHelper]] and [[Java/MavenRepoHelper|MavenRepoHelper]] or [[Java/MavenDebianHelper|MavenDebianHelper]] until --([[Clojure/Leiningen|Leiningen]] is packaged and)-- we've developed a "Leiningen Helper" for Clojure packaging.
Line 319: Line 319:
= Leiningen Helper =
Dh-leiningen needs your help! Here are some links to workarounds used in existing packages that the future `dh-leiningen` could make obsolete:
 

Translation(s): English


Clojure Packaging Tutorial

Introduction

Clojure packages must be built using JavaHelper and MavenRepoHelper or MavenDebianHelper until Leiningen is packaged and we've developed a "Leiningen Helper" for Clojure packaging.

Clojure package names

Typically, we use packages' upstream names as Debian names, adding a -clojure suffix. For example, the cgrand/parsley project (on GitHub) has a source package parsley-clojure and binary package libparsley-clojure (on Salsa).

This does not cover all possible Clojure package names. For more information on our detailed naming policy, see the Clojure Packaging Reference.

Helpful tools

You can use the clj-helper application to help generate packaging for upstream source packages. A number of helpful prompts walk the user through gathering all the information required for packaging.

clj-helper is aware of existing Debian packaging, so you can also use this to update packaging; it will autocomplete any fields it can find in the existing debian/control and debian/copyright files.

You will also need git-buildpackage and pristine-tar installed.

Creating a Clojure package from source

An upstream Clojure source package that builds with Leiningen might have the following directory structure:

.
├── CHANGELOG.md
├── LICENSE
├── project.clj
├── README.md
├── resources
├── src
│   └── gibson
│       └── core.clj
└── test
    └── gibson
        └── core_test.clj

With a project.clj file that looks like this:

   1 (defproject gibson "0.1.0"
   2   :description "Utilities for hacking the Gibson"
   3   :url "https://anonscm.debian.org/git/users/ehashman/gibson.git"
   4   :license {:name "Eclipse Public License"
   5             :url "http://www.eclipse.org/legal/epl-v10.html"}
   6   :dependencies [[org.clojure/clojure "1.8.0"]
   7                  [org.clojure/tools.nrepl "0.2.12"]])

Creating the package's git repository

Since the majority of Clojure software uses git for source control, you can preserve the upstream git tree in our package history. (If you're updating a repository that includes all upstream source in a single commit, you can use gbp import-orig, but I prefer to preserve upstream history where possible.) Here's how to set up the repository.

First, clone the upstream source:

$ git clone https://salsa.debian.org/ehashman/gibson.git

Then set up a corresponding repository on Salsa. You can request that someone from the Clojure Team with "Master" permissions or higher completes this step for you.

Then you should be able to access the bare repository https://salsa.debian.org/clojure-team/gibson-clojure.git [not a real repo]

You'll need to set that up as a remote repository:

$ git remote add salsa https://salsa.debian.org/clojure-team/gibson-clojure.git

Now you can set up the three required branches:

  • master: Where the debian packaging lives. You will build off this branch.

  • upstream: Where the upstream source tree lives.

  • pristine-tar: Where the "pristine" tarball files live. Once a package is uploaded to the archive, you must use the same original source tarball to upload future packaging changes for the same upstream version. Hence, you will commit that tarball here so anyone can upload a package update against the original "pristine" tar.

The workflow you will use to create the branches depends on whether or not upstream tags their releases.

When upstream tags their releases

Add a new upstream/VERSION tag that matches the existing upstream tag:

$ git tag                    # See all available upstream tags
gibson-0.1.0
$ git checkout gibson-0.1.0  # Check out our target version tag
$ git tag upstream/0.1.0     # Make a new tag in the right format
$ git checkout -b upstream   # Make a new 'upstream' branch tracking the tag
$ git branch -D master       # Delete 'master' branch; you won't need it anymore

Now the upstream branch is fully set up! We can push these to salsa:

$ git push salsa upstream
$ git push salsa upstream/0.1.0

Next, you'll download the "pristine" original tar and create the pristine-tar branch. You will need the pristine-tar package installed, in order for us to commit the original tarball we downloaded earlier to git.

Download the original tarball (.tar.gz file) corresponding to the upstream tag we're packaging (gibson-0.1.0 in this case). Put it in the directory above our source directory with the name gibson-clojure_0.1.0.orig.tar.gz.

You can now commit the tarball to git with the pristine-tar command:

$ pristine-tar commit ../gibson-clojure_0.1.0.orig.tar.gz

Now the pristine-tar branch is done! Push it to salsa:

$ git push salsa pristine-tar

You'll create the master branch in the next section.

When upstream does NOT tag their releases

You can file a bug against upstream requesting that they tag their releases. But sometimes, the maintainers do not respond. In this case, you must create your own upstream tags and pristine tarball.

You'll need to add a new upstream/VERSION tag that matches the commit with the version release. If you're not sure what commit this is, you can look at the git log of the project.clj file. There should only be one commit in the git history where the version is set to 0.1.0; elsewhere, Clojure developers usually commit against version SNAPSHOTs (e.g. "0.1.1-SNAPSHOT").

Say you've found the right commit, and its ID is abc123def. You can now make a tag:

$ git checkout abc123def     # Check out our target commit
$ git tag upstream/0.1.0     # Make a new tag in the right format
$ git checkout -b upstream   # Make a new 'upstream' branch tracking the tag
$ git branch -D master       # Delete 'master' branch; you won't need it anymore

Now the upstream branch is fully set up! Push these to salsa:

$ git push salsa upstream
$ git push salsa upstream/0.1.0

You'll create the pristine-tar and master branches in the next section.

Debian files

Now we're ready to generate our Debian packaging! To start, we'll need a branch to work off. Let's create one:

$ git checkout upstream   # We'll start on the 'upstream' branch
$ git checkout -b master  # Create a new 'master' branch based on 'upstream' and check it out

You can use clj-helper to generate packaging the packaging. clj-helper supports Clojure software that builds with Leiningen (the default) and with maven (use the --maven flag). (For more detailed information on compilation and use, see the README.) Since our package builds with Leiningen, here is a sample run for the package above:

$ clj-helper
  Enter the source package's name: [gibson]

  Enter the source package's homepage: []
  https://anonscm.debian.org/git/users/ehashman/gibson.git
  Enter the year this release is copyrighted: []
  2017
  Enter the upstream author's name: []
  Elana Hashman
  Enter the upstream author's email: []
  debian@hashman.ca
  Enter the upstream license, in abbreviated form: [EPL-1.0]

  Enter the names of any dependencies, separated by commas: []
  tools.nrepl
  Enter the package maintainer(s): [Debian Clojure Maintainers <pkg-clojure-maintainers@lists.alioth.debian.org>]

  Enter the package uploader(s): [Elana Hashman <debian@hashman.ca>]

  Enter the project description, ending with a blank line:
  Utilities for hacking the Gibson
   Some long description of the package.

  Generating pom...
  Wrote /export/scratch/ehashman/debian/gibson/pom.xml

  Moving pom to debian/pom.xml...
  Now you can create your Debian changelog with `dch --create`.

  Once you have committed these changes to version control, you may build your package with `gbp buildpackage -uc -us`.
$ dch --create
  <edit your changelog entry here>

Note that new packages should always be released to Debian unstable (or experimental), but until you're ready to tag and upload your package, you should leave the target distro set to "UNRELEASED".

This created a debian directory and some files! Here they are:

debian/
├── changelog
├── compat
├── control
├── copyright
├── libgibson-clojure.classpath
├── libgibson-clojure.doc-base
├── libgibson-clojure.docs
├── libgibson-clojure.poms
├── pom.xml
├── rules
└── source
    └── format

These are all the files you need to build the package.

Take a look over all these newly created files before you commit them. For instance, clj-helper isn't smart enough to figure out your dependencies for you, so you may want to specify minimum versions in debian/control.

You may also choose to change the packaging license from the default generated, add additional docs to the docs and doc-base files, and update the pom.xml if it includes any information you do not want to include in your packaging (plugins, false SCM info, etc.)

Once you are happy with all these files, you should commit them to the master branch:

$ git add debian/                      # Add all the debian/* files
$ git commit -am "Package for Debian"  # Commit them (using a message of your choice)
$ git push salsa master                # Push to salsa

Building the package

Now your are ready to build your package!

You will need git-buildpackage installed in order to build packages, in addition to the rest of the build dependencies specified in debian/control.

When upstream tags their releases

To build the package, run

$ gbp buildpackage -uc -us --git-pristine-tar  # Build package, don't sign source or changes

When upstream does NOT tag their releases

You should run

$ gbp buildpackage -uc -us --git-pristine-tar-commit

to ensure that we generate a pristine tar from the upstream source and that we commit it to the pristine-tar branch during our build. You'll only need to run this once; after your pristine tar is committed, you can build using the command above in the "When upstream tags their releases" section.

Your built package

Assuming all goes well, your packaging files can all be found in the directory above the source directory, looking something like:

.
├── gibson
│   └── ... (source files)
├── gibson-clojure_0.1.0-1.debian.tar.xz
├── gibson-clojure_0.1.0-1.dsc
├── gibson-clojure_0.1.0-1_amd64.build
├── gibson-clojure_0.1.0-1_amd64.buildinfo
├── gibson-clojure_0.1.0-1_amd64.changes
├── gibson-clojure_0.1.0.orig.tar.gz
└── libgibson-clojure_0.1.0-1_all.deb

Yay! You have a package!

When things go wrong

Here are some tips for when you run into build issues.

Issue: GBP refuses to build my package!

Maybe you received an error like this one:

gbp:error: You have uncommitted changes in your source tree:
gbp:error: On branch master
...

GBP will have printed a list of the files that are not committed to the source tree.

If these are build artifacts, you should first remove them by running

$ debian/rules clean

If they are changes to your packaging, you may choose to commit them, or to ignore them for your current build by passing the following flag to gbp:

$ gbp buildpackage -uc -us --git-ignore-new

Issue: No files were listed but it still won't build!

Commonly, upstream .gitignore files on Clojure projects ignore all pom.xml changes. Ensure that you haven't forgotten to commit or reset any changes in your POM.

Issue: My build failed because I'm missing dependencies!

You should make sure you install anything missing. dkpg-checkbuilddeps will let you know which ones are missing:

dpkg-checkbuilddeps: error: Unmet build dependencies: libtools-nrepl-clojure (>= 0.2.12)
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
debuild: fatal error at line 1116:
dpkg-buildpackage -rfakeroot -us -uc -i -I failed
gbp:error: 'debuild -i -I -uc -us' failed: it exited with 29

If these dependencies aren't available on your current OS/distro, you may opt to create a special environment for your package builds, to ensure they have access to all the dependencies you need. For example, you could create an LXC or a VM running Debian sid (unstable). You could even get creative and make a Dockerfile with a debian:unstable base.

Leiningen Helper

Dh-leiningen needs your help! Here are some links to workarounds used in existing packages that the future dh-leiningen could make obsolete:


CategoryClojure