Differences between revisions 7 and 8
Revision 7 as of 2017-05-23 17:12:52
Size: 11404
Editor: ElanaHashman
Comment: Add build section
Revision 8 as of 2017-05-23 17:25:41
Size: 13366
Editor: ElanaHashman
Comment: Added some common build issues Q&A
Deletions are marked like this. Additions are marked like this.
Line 204: Line 204:

Note that new packages should always be released to Debian `unstable` (or `experimental`).
Line 280: Line 282:
== When things go wrong ==

Here are some tips for when you run into build issues.

==== Issue: GBP refuses to build my package! ====

Maybe you received an error like this one:

{{{
gbp:error: You have uncommitted changes in your source tree:
gbp:error: On branch master
...
}}}

GBP will have printed a list of the files that are not committed to the source tree.

If these are build artifacts, you should first remove them by running

{{{
$ debian/rules clean
}}}

If they are changes to your packaging, you may choose to commit them, or to ignore them for your current build by passing the following flag to `gbp`:

{{{
$ gbp buildpackage -uc -us --git-ignore-new
}}}

==== Issue: No files were listed but it still won't build! ====

Commonly, upstream `.gitignore` files on Clojure projects ignore all `pom.xml` changes. Ensure that you haven't forgotten to commit or reset any changes in your POM.

==== Issue: My build failed because I'm missing dependencies! ====

You should make sure you install anything missing. `dkpg-checkbuilddeps` will let you know which ones are missing:

{{{
dpkg-checkbuilddeps: error: Unmet build dependencies: libtools-nrepl-clojure (>= 0.2.12)
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
debuild: fatal error at line 1116:
dpkg-buildpackage -rfakeroot -us -uc -i -I failed
gbp:error: 'debuild -i -I -uc -us' failed: it exited with 29
}}}

If these dependencies aren't available on your current OS/distro, you may opt to create a special environment for your package builds, to ensure they have access to all the dependencies you need. For example, you could create an LXC or a VM running Debian sid (unstable). You could even get creative and make a Dockerfile with a `debian:unstable` base.

Translation(s): English


Clojure Packaging Guide

Introduction

Clojure packages must be built using JavaHelper and MavenRepoHelper until Leiningen is packaged and we've developed a "Leiningen Helper" for Clojure packaging.

Understanding Clojure packages names

When naming Clojure software packages, the namespace is dropped from the package name, any non-alphabetic characters are replaced with dashes, and the package name ends in -clojure. Binary packages are differentiated from the source packages with the lib prefix (for library code). For example,

  • [net.cgrand/parsley "0.9.3"] has source package parsley-clojure and binary package libparsley-clojure

  • [org.clojure/tools.nrepl "0.2.12"] has source package tools-nrepl-clojure and binary package libtools-nrepl-clojure

If the package has a namespace conflict, you may include it in the package name. For example,

  • [com.hypirion/io "0.3.1"] has source package com-hypirion-io-clojure and binary package libcom-hypirion-io-clojure because the name libio-clojure is too easily confused with commons-io, the namespace clojure.java.io, and other similar packages

Multiple packages can be built from the same source package: for instance, if you have a source package that produces an application and library package. In this case, the name of the application follows standard Debian policy.

  • [leiningen "2.7.1"] has source package leiningen, application binary package leiningen, and library binary package libleiningen-clojure

Helpful tools

You can use the clj-helper application to help generate packaging for upstream source packages. A number of helpful prompts walk the user through gathering all the information required for packaging.

clj-helper is aware of existing Debian packaging, so you can also use this to update packaging; it will autocomplete any fields it can find in the existing debian/control and debian/copyright files.

Creating a Clojure package from source

An upstream Clojure source package might have the following directory structure:

.
├── CHANGELOG.md
├── LICENSE
├── project.clj
├── README.md
├── resources
├── src
│   └── gibson
│       └── core.clj
└── test
    └── gibson
        └── core_test.clj

With a project.clj file that looks like this:

   1 (defproject gibson "0.1.0"
   2   :description "Utilities for hacking the Gibson"
   3   :url "https://anonscm.debian.org/git/users/ehashman/gibson.git"
   4   :license {:name "Eclipse Public License"
   5             :url "http://www.eclipse.org/legal/epl-v10.html"}
   6   :dependencies [[org.clojure/clojure "1.8.0"]
   7                  [org.clojure/tools.nrepl "0.2.12"]])

Creating the package's git repository

Since the majority of Clojure software uses git for source control, you can preserve the upstream git tree in our package history. Here's how to set up the repository.

First, clone the upstream source:

$ git clone https://anonscm.debian.org/git/users/ehashman/gibson.git

And then set up a corresponding repository on Alioth:

ehashman@alioth:/git/pkg-clojure$ ./setup-repository gibson-clojure

You should now be able to browse to the Debian git tree at https://anonscm.debian.org/git/pkg-clojure/gibson-clojure.git [not a real repo]

You'll need to set that up as a remote repository:

$ git remote add alioth https://anonscm.debian.org/git/pkg-clojure/gibson-clojure.git

Now you can set up the three required branches:

  • master: Where the debian packaging lives. You will build off this branch.

  • upstream: Where the upstream source tree lives.

  • pristine-tar: Where the "pristine" tarball files live. Once a package is uploaded to the archive, you must use the same original source tarball to upload future packaging changes for the same upstream version. Hence, you will commit that tarball here so anyone can upload a package update against the original "pristine" tar.

The workflow you will use to create the branches depends on whether or not upstream tags their releases.

When upstream tags their releases

Add a new upstream/VERSION tag that matches the existing upstream tag:

$ git tag                    # See all available upstream tags
gibson-0.1.0
$ git checkout gibson-0.1.0  # Check out our target version tag
$ git tag upstream/0.1.0     # Make a new tag in the right format
$ git checkout -b upstream   # Make a new 'upstream' branch tracking the tag
$ git branch -D master       # Delete 'master' branch; you won't need it anymore

Now the upstream branch is fully set up! We can push these to alioth:

$ git push alioth upstream
$ git push alioth upstream/0.1.0

Next, you'll download the "pristine" original tar and create the pristine-tar branch. You will need the pristine-tar package installed, in order for us to commit the original tarball we downloaded earlier to git.

Download the original tarball (.tar.gz file) corresponding to the upstream tag we're packaging (gibson-0.1.0 in this case). Put it in the directory above our source directory with the name gibson-clojure_0.1.0.orig.tar.gz.

You can now commit the tarball to git with the pristine-tar command:

$ pristine-tar commit ../gibson-clojure_0.1.0.orig.tar.gz

Now the pristine-tar branch is done! Push it to alioth:

$ git push alioth pristine-tar

You'll create the master branch in the next section.

When upstream does NOT tag their releases

You can file a bug against upstream requesting that they tag their releases. But sometimes, the maintainers do not respond. In this case, you must create your own upstream tags and pristine tarball.

You'll need to add a new upstream/VERSION tag that matches the commit with the version release. If you're not sure what commit this is, you can look at the git log of the project.clj file. There should only be one commit in the git history where the version is set to 0.1.0; elsewhere, Clojure developers usually commit against version SNAPSHOTs (e.g. "0.1.1-SNAPSHOT").

Say you've found the right commit, and its ID is abc123def. You can now make a tag:

$ git checkout abc123def     # Check out our target commit
$ git tag upstream/0.1.0     # Make a new tag in the right format
$ git checkout -b upstream   # Make a new 'upstream' branch tracking the tag
$ git branch -D master       # Delete 'master' branch; you won't need it anymore

Now the upstream branch is fully set up! Push these to alioth:

$ git push alioth upstream
$ git push alioth upstream/0.1.0

You'll create the pristine-tar and master branches in the next section.

Debian files

Now we're ready to generate our Debian packaging! To start, we'll need a branch to work off. Let's create one:

$ git checkout upstream   # We'll start on the 'upstream' branch
$ git checkout -b master  # Create a new 'master' branch based on 'upstream' and check it out

You can use clj-helper to generate packaging the packaging. (For more detailed information on compilation and use, see the README.) Here is a sample run for the package above:

$ clj-helper
  Enter the source package's name: [gibson]

  Enter the source package's homepage: []
  https://anonscm.debian.org/git/users/ehashman/gibson.git
  Enter the year this release is copyrighted: []
  2017
  Enter the upstream author's name: []
  Elana Hashman
  Enter the upstream author's email: []
  debian@hashman.ca
  Enter the upstream license, in abbreviated form: [EPL-1.0]

  Enter the names of any dependencies, separated by commas: []
  tools.nrepl
  Enter the package maintainer(s): [Debian Clojure Maintainers <pkg-clojure-maintainers@lists.alioth.debian.org>]

  Enter the package uploader(s): [Elana Hashman <debian@hashman.ca>]

  Enter the project description, ending with a blank line:
  Utilities for hacking the Gibson
   Some long description of the package.

  Generating pom...
  Wrote /export/scratch/ehashman/debian/gibson/pom.xml

  Moving pom to debian/pom.xml...
  Now you can create your Debian changelog with `dch --create`.

  Once you have committed these changes to version control, you may build your package with `gbp buildpackage -uc -us`.
$ dch --create
  <edit your changelog entry here>

Note that new packages should always be released to Debian unstable (or experimental).

This created a debian directory and some files! Here they are:

debian/
├── changelog
├── compat
├── control
├── copyright
├── libgibson-clojure.classpath
├── libgibson-clojure.doc-base
├── libgibson-clojure.docs
├── libgibson-clojure.poms
├── pom.xml
├── rules
└── source
    └── format

These are all the files you need to build the package.

Take a look over all these newly created files before you commit them. For instance, clj-helper isn't smart enough to figure out your dependencies for you, so you may want to specify minimum versions in debian/control.

You may also choose to change the packaging license from the default generated, add additional docs to the docs and doc-base files, and update the pom.xml if it includes any information you do not want to include in your packaging (plugins, false SCM info, etc.)

Once you are happy with all these files, you should commit them to the master branch:

$ git add debian/                      # Add all the debian/* files
$ git commit -am "Package for Debian"  # Commit them (using a message of your choice)
$ git push alioth master               # Push to alioth

Building the package

Now your are ready to build your package!

You will need git-buildpackage installed in order to build packages, in addition to the rest of the build dependencies specified in debian/control.

When upstream tags their releases

To build the package, run

$ gbp buildpackage -uc -us  # Build package, don't sign source or changes

When upstream does NOT tag their releases

You should run

$ gbp buildpackage -uc -us --git-pristine-tar --git-pristine-tar-commit

to ensure that we generate a pristine tar from the upstream source and that we commit it to the pristine-tar branch during our build.

Your built package

Assuming all goes well, your packaging files can all be found in the directory above the source directory, looking something like:

.
├── gibson
│   └── ... (source files)
├── gibson-clojure_0.1.0-1.debian.tar.xz
├── gibson-clojure_0.1.0-1.dsc
├── gibson-clojure_0.1.0-1_amd64.build
├── gibson-clojure_0.1.0-1_amd64.buildinfo
├── gibson-clojure_0.1.0-1_amd64.changes
├── gibson-clojure_0.1.0.orig.tar.gz
└── libgibson-clojure_0.1.0-1_all.deb

Yay! You have a package!

When things go wrong

Here are some tips for when you run into build issues.

Issue: GBP refuses to build my package!

Maybe you received an error like this one:

gbp:error: You have uncommitted changes in your source tree:
gbp:error: On branch master
...

GBP will have printed a list of the files that are not committed to the source tree.

If these are build artifacts, you should first remove them by running

$ debian/rules clean

If they are changes to your packaging, you may choose to commit them, or to ignore them for your current build by passing the following flag to gbp:

$ gbp buildpackage -uc -us --git-ignore-new

Issue: No files were listed but it still won't build!

Commonly, upstream .gitignore files on Clojure projects ignore all pom.xml changes. Ensure that you haven't forgotten to commit or reset any changes in your POM.

Issue: My build failed because I'm missing dependencies!

You should make sure you install anything missing. dkpg-checkbuilddeps will let you know which ones are missing:

dpkg-checkbuilddeps: error: Unmet build dependencies: libtools-nrepl-clojure (>= 0.2.12)
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
debuild: fatal error at line 1116:
dpkg-buildpackage -rfakeroot -us -uc -i -I failed
gbp:error: 'debuild -i -I -uc -us' failed: it exited with 29

If these dependencies aren't available on your current OS/distro, you may opt to create a special environment for your package builds, to ensure they have access to all the dependencies you need. For example, you could create an LXC or a VM running Debian sid (unstable). You could even get creative and make a Dockerfile with a debian:unstable base.


CategoryClojure