Clojure CLI Tools in Debian

This page is intended to help the (re-)packaging of clj (and maybe other tools?).

[ Original Debian GSoC 2021 Project ]

[ Accepted Proposal ]

Relevant ITPs

1. The following ITPs should be fixed:

2. Consider also closing the following RFPs:


We plan to collaborate with upstream during the packaging process, to ensure the package's quality. From what we have already heard from them in the debian-clojure list, they are listening :)

Packaging `clj`

Upstream repo: https://github.com/clojure/brew-install.git

The main branch is named after the current Clojure version (as of 2021.06.11, it is 1.10.3).

git checkout 1.10.3

As of 2021.06.11, HEAD --> 1.10.3, and HEAD --> 577cb35b0a2475ec5befb93087873770a3ec0c4c.

In order to correctly plan how to package clj, we first need to study the upstream dependencies.

Dependency Analysis

Strategy

  1. Check which packages are already in Debian (some of them already are).
  2. Identify leaf nodes (all).
  3. Identify leaf nodes to work on (probably only a subset of them).
  4. Work on org.clojure leaf nodes.

  5. Work on non- org.clojure leaf nodes.

Conclusions

  1. The only first-level dependency not already in Debian is org.clojure:tools.deps.alpha.

  2. There are two org.clojure leaf nodes: libtools-gitlibs-clojure and libtools-cli-clojure.

Course of action

Initial proposal (modified - see below):

  1. Package org.clojure:tools.gitlibs.

  2. Consider updating libtools-cli-clojure (already in Debian).

  3. Consider packaging org.clojure:data.xml.

  4. Consider packaging com.cognitec.aws:* packages

Current Status

Revised proposal, after feedback from pollo, ehashman 891136 and Alex Miller:

  1. update org.clojure:data.xml to 0.2.0-beta6. (./) (already in experimental)

    • lein uses 0.2.0-beta5 ==> open PR upstream so lein uses 0.2.0-beta6.

    • lein has many outdated dependencies: maybe also consider opening other PRs upstream so lein uses latest dependencies of other libraries already in Debian?

  2. package org.clojure:tools.gitlibs. (waiting for review)

  3. consider updating libjsch-agent-proxy-java to (TBD: 891136 mentions 0.0.9). (no longer needed by org.clojure:tools.gitlibs)

  4. consider updating jgit to (TBD: 891136 mentions 4.10.0). (no longer needed by org.clojure:tools.gitlibs)

  5. consider updating libtools-cli-clojure (already in Debian) (not needed for now - maybe later?).

  6. consider packaging com.cognitec.aws:* packages. (probably not needed for clojure.tools.deps.alpha?)

  7. package org.clojure:tools.deps.alpha. (in progress)

  8. package clj. (TODO)

Detailed Dependency Analysis

`clj` dependencies

`clj` dependencies (raw tree)

According to mvn dependency:tree, we have the following tree:

commit 577cb35b0a2475ec5befb93087873770a3ec0c4c (HEAD -> 1.10.3)

% mvn dependency:tree
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------------< org.clojure:clojure-tools >----------------------
[INFO] Building clojure-tools 1.10.3.855
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ clojure-tools ---
[INFO] org.clojure:clojure-tools:jar:1.10.3.855
[INFO] +- org.clojure:clojure:jar:1.10.3:compile
[INFO] |  +- org.clojure:spec.alpha:jar:0.2.194:compile
[INFO] |  \- org.clojure:core.specs.alpha:jar:0.2.56:compile
[INFO] +- org.clojure:tools.deps.alpha:jar:0.11.922:compile
[INFO] |  +- org.apache.maven.resolver:maven-resolver-api:jar:1.7.0:compile
[INFO] |  +- org.apache.maven.resolver:maven-resolver-spi:jar:1.7.0:compile
[INFO] |  +- org.apache.maven.resolver:maven-resolver-impl:jar:1.7.0:compile
[INFO] |  |  +- org.apache.maven.resolver:maven-resolver-named-locks:jar:1.7.0:compile
[INFO] |  |  +- org.apache.commons:commons-lang3:jar:3.8.1:compile
[INFO] |  |  \- javax.annotation:javax.annotation-api:jar:1.3.2:compile
[INFO] |  +- org.apache.maven.resolver:maven-resolver-util:jar:1.7.0:compile
[INFO] |  +- org.apache.maven.resolver:maven-resolver-connector-basic:jar:1.7.0:compile
[INFO] |  +- org.apache.maven.resolver:maven-resolver-transport-file:jar:1.7.0:compile
[INFO] |  +- org.apache.maven.resolver:maven-resolver-transport-http:jar:1.7.0:compile
[INFO] |  |  +- org.apache.httpcomponents:httpclient:jar:4.5.13:compile
[INFO] |  |  |  \- commons-codec:commons-codec:jar:1.11:compile
[INFO] |  |  +- org.apache.httpcomponents:httpcore:jar:4.4.14:compile
[INFO] |  |  \- org.slf4j:jcl-over-slf4j:jar:1.7.30:runtime
[INFO] |  +- org.apache.maven:maven-resolver-provider:jar:3.8.1:compile
[INFO] |  |  +- org.apache.maven:maven-model:jar:3.8.1:compile
[INFO] |  |  +- org.apache.maven:maven-model-builder:jar:3.8.1:compile
[INFO] |  |  |  \- org.codehaus.plexus:plexus-interpolation:jar:1.25:compile
[INFO] |  |  +- org.apache.maven:maven-repository-metadata:jar:3.8.1:compile
[INFO] |  |  \- org.codehaus.plexus:plexus-utils:jar:3.2.1:compile
[INFO] |  +- org.apache.maven:maven-core:jar:3.8.1:compile
[INFO] |  |  +- org.apache.maven:maven-settings:jar:3.8.1:compile
[INFO] |  |  +- org.apache.maven:maven-settings-builder:jar:3.8.1:compile
[INFO] |  |  |  \- org.sonatype.plexus:plexus-sec-dispatcher:jar:1.4:compile
[INFO] |  |  |     \- org.sonatype.plexus:plexus-cipher:jar:1.4:compile
[INFO] |  |  +- org.apache.maven:maven-builder-support:jar:3.8.1:compile
[INFO] |  |  +- org.apache.maven:maven-artifact:jar:3.8.1:compile
[INFO] |  |  +- org.apache.maven:maven-plugin-api:jar:3.8.1:compile
[INFO] |  |  +- org.apache.maven.shared:maven-shared-utils:jar:3.2.1:compile
[INFO] |  |  |  \- commons-io:commons-io:jar:2.5:compile
[INFO] |  |  +- org.eclipse.sisu:org.eclipse.sisu.plexus:jar:0.3.4:compile
[INFO] |  |  |  \- javax.enterprise:cdi-api:jar:1.0:compile
[INFO] |  |  |     \- javax.annotation:jsr250-api:jar:1.0:compile
[INFO] |  |  +- org.eclipse.sisu:org.eclipse.sisu.inject:jar:0.3.4:compile
[INFO] |  |  +- com.google.inject:guice:jar:no_aop:4.2.1:compile
[INFO] |  |  |  +- aopalliance:aopalliance:jar:1.0:compile
[INFO] |  |  |  \- com.google.guava:guava:jar:25.1-android:compile
[INFO] |  |  |     +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO] |  |  |     +- org.checkerframework:checker-compat-qual:jar:2.0.0:compile
[INFO] |  |  |     +- com.google.errorprone:error_prone_annotations:jar:2.1.3:compile
[INFO] |  |  |     +- com.google.j2objc:j2objc-annotations:jar:1.1:compile
[INFO] |  |  |     \- org.codehaus.mojo:animal-sniffer-annotations:jar:1.14:compile
[INFO] |  |  +- org.codehaus.plexus:plexus-classworlds:jar:2.6.0:compile
[INFO] |  |  \- org.codehaus.plexus:plexus-component-annotations:jar:2.1.0:compile
[INFO] |  +- org.clojure:data.xml:jar:0.2.0-alpha6:compile
[INFO] |  |  \- org.clojure:data.codec:jar:0.1.0:compile
[INFO] |  +- org.clojure:tools.gitlibs:jar:2.3.167:compile
[INFO] |  +- org.clojure:tools.cli:jar:1.0.206:compile
[INFO] |  +- com.cognitect.aws:api:jar:0.8.505:compile
[INFO] |  |  +- org.clojure:data.json:jar:1.0.0:compile
[INFO] |  |  +- org.clojure:tools.logging:jar:1.1.0:compile
[INFO] |  |  +- com.cognitect:http-client:jar:0.1.106:compile
[INFO] |  |  |  +- org.eclipse.jetty:jetty-client:jar:9.4.36.v20210114:compile
[INFO] |  |  |  |  \- org.eclipse.jetty:jetty-io:jar:9.4.36.v20210114:compile
[INFO] |  |  |  +- org.eclipse.jetty:jetty-http:jar:9.4.36.v20210114:compile
[INFO] |  |  |  \- org.eclipse.jetty:jetty-util:jar:9.4.36.v20210114:compile
[INFO] |  |  \- org.clojure:core.async:jar:1.3.610:compile
[INFO] |  |     \- org.clojure:tools.analyzer.jvm:jar:1.1.0:compile
[INFO] |  |        +- org.clojure:tools.analyzer:jar:1.0.0:compile
[INFO] |  |        +- org.clojure:core.memoize:jar:1.0.236:compile
[INFO] |  |        |  \- org.clojure:core.cache:jar:1.0.207:compile
[INFO] |  |        |     \- org.clojure:data.priority-map:jar:1.0.0:compile
[INFO] |  |        +- org.ow2.asm:asm:jar:5.2:compile
[INFO] |  |        \- org.clojure:tools.reader:jar:1.3.2:compile
[INFO] |  +- com.cognitect.aws:endpoints:jar:1.1.11.969:compile
[INFO] |  +- com.cognitect.aws:s3:jar:811.2.858.0:compile
[INFO] |  \- javax.inject:javax.inject:jar:1:compile
[INFO] \- org.slf4j:slf4j-nop:jar:1.7.30:compile
[INFO]    \- org.slf4j:slf4j-api:jar:1.7.30:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.599 s
[INFO] Finished at: 2021-06-09T11:07:08-03:00
[INFO] ------------------------------------------------------------------------

`clj` dependencies (full tree, listed)

`clj` dependencies (first-level)

* org.clojure:clojure:jar:1.10.3:compile (has 2 dependencies, already in Debian as clojure)

* org.clojure:tools.deps.alpha:jar:0.11.922:compile (16 dependencies, not yet in Debian)

* org.slf4j:slf4j-nop:jar:1.7.30:compile (has 1 dependency, already in Debian as libslf4j)

`clj` dependencies (second-level) - `org.clojure:tools.deps.alpha` dependencies

`clj` dependencies (third-level) - `data-xml-clojure` dependencies (raw tree)

UPDATE: I've just packaged this library into experimental. Debian PTS

UPDATE: The following dependency has a fatal flaw: I did it on master (and not the release tag). Thanks @pollo for noting this! I'm missing one dependency: [org.clojure/data.codec] (not currently on Debian. As this dependency was later replaced upstream, I have two options: a) Also package data.codec (which will eventually be removed), or b) patching the release with the dependency switch. As the first option would imply useless work, I choose the second one.

commit da84d4f3a9151f94a3846d752c9ec1fb8e195764 (HEAD -> master)

% lein deps :tree
...
Possibly confusing dependencies found:
[org.clojure/clojurescript "1.10.439"] -> [com.google.javascript/closure-compiler-unshaded "v20180805"] -> [args4j "2.0.26"]
 overrides
[figwheel-sidecar "0.5.17"] -> [figwheel "0.5.17" :exclusions [org.clojure/tools.reader]] -> [org.clojure/clojurescript "1.10.238" :exclusions [org.apache.ant/ant]] -> [com.google.javascript/closure-compiler-unshaded "v20180204"] -> [args4j "2.33"]
 and
[figwheel-sidecar "0.5.17"] -> [org.clojure/clojurescript "1.10.238" :exclusions [org.apache.ant/ant]] -> [com.google.javascript/closure-compiler-unshaded "v20180204"] -> [args4j "2.33"]

Consider using these exclusions:
[figwheel-sidecar "0.5.17" :exclusions [args4j]]

[org.clojure/clojurescript "1.10.439"] -> [com.google.javascript/closure-compiler-unshaded "v20180805"] -> [com.google.errorprone/error_prone_annotations "2.0.18"]
 overrides
[org.clojure/clojurescript "1.10.439"] -> [com.google.javascript/closure-compiler-unshaded "v20180805"] -> [com.google.guava/guava "25.1-jre"] -> [com.google.errorprone/error_prone_annotations "2.1.3"]

Consider using these exclusions:
[org.clojure/clojurescript "1.10.439" :exclusions [com.google.errorprone/error_prone_annotations]]

[org.clojure/clojurescript "1.10.439"] -> [com.google.javascript/closure-compiler-unshaded "v20180805"] -> [com.google.code.findbugs/jsr305 "3.0.1"]
 overrides
[org.clojure/clojurescript "1.10.439"] -> [com.google.javascript/closure-compiler-unshaded "v20180805"] -> [com.google.guava/guava "25.1-jre"] -> [com.google.code.findbugs/jsr305 "3.0.2"]

Consider using these exclusions:
[org.clojure/clojurescript "1.10.439" :exclusions [com.google.code.findbugs/jsr305]]

 [binaryage/devtools "0.9.10"]
   [binaryage/env-config "0.2.2"]
 [clojure-complete "0.2.5" :exclusions [[org.clojure/clojure]]]
 [com.cemerick/piggieback "0.2.2"]
 [figwheel-sidecar "0.5.17"]
   [clj-stacktrace "0.2.8"]
   [co.deps/ring-etag-middleware "0.2.0"]
   [com.stuartsierra/component "0.3.2"]
     [com.stuartsierra/dependency "0.2.0"]
   [figwheel "0.5.17" :exclusions [[org.clojure/tools.reader]]]
   [hawk "0.2.11" :exclusions [[org.clojure/clojure]]]
     [net.incongru.watchservice/barbary-watchservice "1.0"]
       [net.java.dev.jna/jna "3.2.2"]
   [http-kit "2.3.0"]
   [org.clojure/core.async "0.4.474" :exclusions [[org.clojure/tools.reader]]]
     [org.clojure/tools.analyzer.jvm "0.7.0"]
       [org.clojure/core.memoize "0.5.9"]
         [org.clojure/core.cache "0.6.5"]
           [org.clojure/data.priority-map "0.0.7"]
       [org.clojure/tools.analyzer "0.6.9"]
       [org.ow2.asm/asm-all "4.2"]
   [ring-cors "0.1.12" :exclusions [[ring/ring-core] [org.clojure/clojure]]]
   [ring/ring-core "1.6.3" :exclusions [[org.clojure/tools.reader] [org.clojure/clojure]]]
     [clj-time "0.11.0"]
       [joda-time "2.8.2"]
     [commons-fileupload "1.3.3"]
     [commons-io "2.5"]
     [crypto-equality "1.0.0"]
     [crypto-random "1.2.0"]
     [ring/ring-codec "1.0.1"]
   [simple-lein-profile-merge "0.1.4"]
   [strictly-specking-standalone "0.1.1"]
     [net.cgrand/parsley "0.9.3" :exclusions [[org.clojure/clojure]]]
       [net.cgrand/regex "1.1.0"]
     [net.cgrand/sjacket "0.1.1" :exclusions [[org.clojure/clojure] [net.cgrand/parsley]]]
   [suspendable "0.1.1" :exclusions [[org.clojure/clojure] [com.stuartsierra/component]]]
 [nrepl "0.6.0" :exclusions [[org.clojure/clojure]]]
 [org.clojure/clojure "1.10.0-beta8"]
   [org.clojure/core.specs.alpha "0.2.44"]
   [org.clojure/spec.alpha "0.2.176"]
 [org.clojure/clojurescript "1.10.439"]
   [com.cognitect/transit-clj "0.8.309" :exclusions [[org.clojure/clojure]]]
     [com.cognitect/transit-java "0.8.332"]
       [com.fasterxml.jackson.core/jackson-core "2.8.7"]
       [commons-codec "1.10"]
       [org.msgpack/msgpack "0.6.12"]
         [com.googlecode.json-simple/json-simple "1.1.1" :exclusions [[junit]]]
         [org.javassist/javassist "3.18.1-GA"]
   [com.google.javascript/closure-compiler-unshaded "v20180805"]
     [args4j "2.0.26"]
     [com.google.code.findbugs/jsr305 "3.0.1"]
     [com.google.code.gson/gson "2.7"]
     [com.google.errorprone/error_prone_annotations "2.0.18"]
     [com.google.guava/guava "25.1-jre"]
       [com.google.j2objc/j2objc-annotations "1.1"]
       [org.checkerframework/checker-qual "2.0.0"]
       [org.codehaus.mojo/animal-sniffer-annotations "1.14"]
     [com.google.javascript/closure-compiler-externs "v20180805"]
     [com.google.jsinterop/jsinterop-annotations "1.0.0"]
     [com.google.protobuf/protobuf-java "3.0.2"]
   [org.clojure/data.json "0.2.6"]
   [org.clojure/google-closure-library "0.0-20170809-b9c14c6b"]
     [org.clojure/google-closure-library-third-party "0.0-20170809-b9c14c6b"]
   [org.clojure/tools.reader "1.3.0"]
   [org.mozilla/rhino "1.7R5"]
 [org.clojure/test.check "0.9.0"]
 [org.clojure/tools.nrepl "0.2.13"]

`clj` dependencies (third-level) - `com.cognitec.aws:*` dependencies

`lein` dependencies (raw tree, upstream)

commit e270102447fb9ca1f33591d39b72b15d06cdc0ef (HEAD -> master)

% lein deps :tree
 [clojure-complete "0.2.5" :exclusions [[org.clojure/clojure]]]
 [commons-io "2.8.0"]
 [commons-lang "2.6"]
 [leiningen-core "2.9.5"]
   [clj-commons/pomegranate "1.2.0" :exclusions [[org.slf4j/jcl-over-slf4j]]]
     [org.apache.maven.resolver/maven-resolver-api "1.3.3"]
     [org.apache.maven.resolver/maven-resolver-connector-basic "1.3.3"]
     [org.apache.maven.resolver/maven-resolver-impl "1.3.3"]
     [org.apache.maven.resolver/maven-resolver-spi "1.3.3"]
     [org.apache.maven.resolver/maven-resolver-transport-file "1.3.3"]
     [org.apache.maven.resolver/maven-resolver-transport-http "1.3.3"]
     [org.apache.maven.resolver/maven-resolver-transport-wagon "1.3.3"]
     [org.apache.maven.resolver/maven-resolver-util "1.3.3"]
     [org.apache.maven.wagon/wagon-http "3.3.2"]
       [org.apache.maven.wagon/wagon-http-shared "3.3.2"]
         [org.jsoup/jsoup "1.11.3"]
     [org.apache.maven.wagon/wagon-provider-api "3.3.2"]
     [org.apache.maven/maven-resolver-provider "3.6.1"]
       [javax.inject "1"]
       [org.apache.maven/maven-model-builder "3.6.1"]
         [org.apache.maven/maven-artifact "3.6.1"]
           [org.apache.commons/commons-lang3 "3.8.1"]
         [org.apache.maven/maven-builder-support "3.6.1"]
         [org.codehaus.plexus/plexus-component-annotations "1.7.1" :exclusions [[junit]]]
         [org.codehaus.plexus/plexus-interpolation "1.25"]
       [org.apache.maven/maven-model "3.6.1"]
       [org.apache.maven/maven-repository-metadata "3.6.1"]
       [org.codehaus.plexus/plexus-utils "3.2.0"]
   [com.hypirion/io "0.3.1"]
   [org.clojure/clojure "1.10.1"]
     [org.clojure/core.specs.alpha "0.2.44"]
     [org.clojure/spec.alpha "0.2.176"]
   [org.clojure/tools.macro "0.1.5"]
   [org.flatland/classlojure "0.7.1"]
   [org.slf4j/slf4j-nop "1.7.25"]
     [org.slf4j/slf4j-api "1.7.25"]
   [robert/hooke "1.3.0"]
 [net.cgrand/parsley "0.9.3" :exclusions [[org.clojure/clojure]]]
 [nrepl "0.8.3"]
 [org.apache.httpcomponents/httpclient "4.5.13"]
   [commons-codec "1.11"]
   [commons-logging "1.2"]
 [org.apache.httpcomponents/httpcore "4.4.13"]
 [org.clojars.trptcolin/sjacket "0.1.1.1" :exclusions [[org.clojure/clojure]]]
   [net.cgrand/regex "1.1.0"]
 [org.clojure/data.xml "0.2.0-alpha5"]
   [org.clojure/data.codec "0.1.0"]
 [scout "0.1.1"]
 [stencil "0.5.0" :exclusions [[org.clojure/core.cache]]]
   [quoin "0.1.2"]
 [timofreiberg/bultitude "0.3.0" :exclusions [[org.clojure/clojure]]]
   [org.tcrawley/dynapath "1.0.0"]