Wiki

• FrontPage
• RecentChanges
• FindPage
• HelpContents
• GoPackaging

This page is about packaging golang.org libraries in Debian.

Table of contents:

Facts

• The go compiler (gc) needs to have the source code of a package available when compiling code that uses it. E.g. if your code uses github.com/mstap/godebiancontrol, gc needs $GOPATH/src/github.com/mstap/godebiancontrol (can that be fixed?)

• Perl uses libXX-perl for package names (e.g. libnet-inet6glue-perl for Net::INET6Glue), while Ruby switched to ruby-xx (e.g. ruby-tioga). TODO: Is there some rationale behind either decision? Which is “better”?

• Both Perl and Ruby use /usr/lib/{perl,ruby} and store source as well as shared objects there.

Package naming

For github.com/mstap/godebiancontrol (which contains "go" already), the resulting Debian package name is golang-godebiancontrol-dev.

We use the -dev suffix to keep the namespace clean in case go supports shared libraries, which would then be shipped as golang-godebiancontrol.

Where to store go src/pkg data?

Go libraries (not binaries!) are present in Debian only for the purpose of building binary packages. They should not be used directly for Go development. Instead, the well-documented and platform independent way of using “go get” should be used.

Why should I use “go get” instead of apt-get to install Go libraries?

• By using “go get” you are forced to set up the environment variable $GOPATH which you need for other “go” commands (e.g. “go build”, “go test”, etc.).

• Debian packages install files to system locations, which users cannot modify. That means that users would not be able to upgrade packages with the canonical “go get -u <package>”. Even worse, when using sudo to forcibly modify the system files, it still would not work since no VCS information is contained in the Debian packages.

I want to build a Go binary Debian package

• libraries are installed to /usr/lib/gocode
• When building Go Debian packages, GOPATH is set to “/usr/lib/gocode”.

Making gc find the system go libraries

The first possibility that came to my mind was adding export GOPATH=/usr/lib/gocode to /etc/profile, but there are two problems with that:

1. /etc/profile (and /etc/profile.d/*.sh) is not executed by every shell, e.g. not by zsh

2. Debian Policy actually forbids modifying it and says that programs need to work without environment variables in a reasonable way (see “Environment variables” in http://www.debian.org/doc/debian-policy/ch-opersys.html)

Therefore, I think the best way is to patch our version of go to always add /usr/lib/gocode to c.GOPATH in src/pkg/go/build/build.go.

Multi-Arch/cross-compiling

golang-go is currently available for amd64 (linux_amd64), armel (linux_arm), armhf (linux_arm), i386 (linux_386).

Note that the identifier Go uses is “linux_arm” for both armel and armhf while the files differ:

Binary files /tmp/armhf/usr/lib/go/pkg/linux_arm/unicode.a and /tmp/armel/usr/lib/go/pkg/linux_arm/unicode.a differ

Multi-arch in itself doesn’t make sense: gc produces static binaries, so to run a program that was compiled for i386 on amd64, you don’t need any libraries.

For cross-compiling, it doesn’t matter which pre-compiled packages you have installed (e.g. linux_amd64) because the .go source files are shipped, too. go build will just re-compile the libraries then:

$ echo $GOPATH
/usr/lib/gocode:/tmp/src/golang
$ GOARCH=386 CGO_ENABLED=0 PATH=~/go-i386/go/bin:$PATH go build -v
github.com/mstap/godebiancontrol
debtest
$ file debtest
debtest: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped

TODO

• In the long term, we should request a section “golang”, see 390609 for an example. This step should follow after there are a number of libraries.

• lintian check for missing Built-Using

Example binary + library packaging

debian/rules:

#!/usr/bin/make -f
#export DH_VERBOSE=1

# GOPKG is the upstream path which you would normally “go get”.
# Using it allows us to build applications without patching locations.
GOPKG := code.google.com/p/codesearch

# Temporary working directory to which the source will be copied, then
# compiled.
TMPGOPATH = $(CURDIR)/debian/tmp/usr/lib/gocode

override_dh_auto_install:
        mkdir -p ${TMPGOPATH}/src/${GOPKG}
        # Copy all .go files to /usr/lib/gocode (we compile and ship).
        find . -path ./debian -prune -o -type f -name "*.go" -exec tar cf - {} + | (cd "${TMPGOPATH}/src/${GOPKG}" && tar xvf -)
        # Ensure that GOPATH is clean: It should only contain the temporary
        # /usr/lib/gocode containing the package we want to install and the
        # /usr/lib/gocode of the system we are building on. It should
        # specifically NOT contain the user’s local ~/gocode.
        GOPATH=${TMPGOPATH}:/usr/lib/gocode go install -v ${GOPKG}/...
        
        # see also http://lists.debian.org/debian-devel/2011/03/msg00852.html
        # The Built-Using field makes sure the required sources are kept around
        # until all binaries that were built from the sources are deleted.
        # Necessary since Go links statically and thus there is no dependency
        # on the library in the resulting package.
        deps=`dpkg-query -f='$${source:Package} (= $${source:Version}), ' -W $$(perl -MParse::DebControl -E 'my $$p = Parse::DebControl->new(); my $$data = $$p->parse_file("./debian/control"); my @deps = grep { /^\s*golang-/ } split(",", $$data->[0]->{"Build-Depends"}); say join(" ", @deps)')`; \
        for pkg in `dh_listpackages`; do \
                echo -n "misc:Built-Using=$$deps" >> debian/$$pkg.substvars; \
        done

%:
        dh $@ --parallel

debian/control:

Source: codesearch
Section: utils
Priority: extra
Maintainer: Michael Stapelberg <stapelberg@debian.org>
Build-Depends: debhelper (>= 8.0.0), golang-go, libparse-debcontrol-perl
Standards-Version: 3.9.4
Homepage: https://code.google.com/p/codesearch/

Package: codesearch
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Built-Using: ${misc:Built-Using}
Description: regular expression search over large bodies of source code
 Code Search is a tool for indexing and then performing regular expression
 searches over large bodies of source code. It is a set of command-line
 programs written in Go.
 .
 For background and an overview of the commands, see Regular Expression
 Matching with a Trigram Index:
 http://swtch.com/~rsc/regexp/regexp4.html

Package: golang-codesearch-dev
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Built-Using: ${misc:Built-Using}
Description: regular expression search over large bodies of source code
 Code Search is a tool for indexing and then performing regular expression
 searches over large bodies of source code. It is a set of command-line
 programs written in Go.
 .
 For background and an overview of the commands, see Regular Expression
 Matching with a Trigram Index:
 http://swtch.com/~rsc/regexp/regexp4.html
 .
 This package contains the source and pre-compiled package files.

debian/codesearch.install:

debian/tmp/usr/lib/gocode/bin/* /usr/bin

debian/golang-codesearch-dev.install:

usr/lib/gocode/src
usr/lib/gocode/pkg

For a binary-only package, use just the first .install file, for a library-only package, use just the second .install file.

Resulting package file lists

$ dpkg -c ../codesearch_0.01+hg20121021-1_amd64.deb
drwxr-xr-x root/root         0 2013-01-01 13:48 ./
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/bin/
-rwxr-xr-x root/root   1385040 2013-01-01 13:48 ./usr/bin/cgrep
-rwxr-xr-x root/root   1389728 2013-01-01 13:48 ./usr/bin/cindex
-rwxr-xr-x root/root   1520272 2013-01-01 13:48 ./usr/bin/csearch
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/share/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/share/doc/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/share/doc/codesearch/
-rw-r--r-- root/root       189 2013-01-01 12:39 ./usr/share/doc/codesearch/changelog.Debian.gz
-rw-r--r-- root/root       415 2012-10-21 15:24 ./usr/share/doc/codesearch/README
-rw-r--r-- root/root      2512 2013-01-01 12:54 ./usr/share/doc/codesearch/copyright

$ dpkg -c ../golang-codesearch_0.01+hg20121021-1_amd64.deb
drwxr-xr-x root/root         0 2013-01-01 13:48 ./
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/pkg/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/p/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/p/codesearch/
-rw-r--r-- root/root    620548 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/p/codesearch/index.a
-rw-r--r-- root/root    313606 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/p/codesearch/regexp.a
-rw-r--r-- root/root     24074 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/p/codesearch/sparse.a
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/
-rw-r--r-- root/root      1751 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/read_test.go
-rw-r--r-- root/root      3451 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/write_test.go
-rw-r--r-- root/root       887 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/mmap_windows.go
-rw-r--r-- root/root     10584 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/read.go
-rw-r--r-- root/root      2198 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/merge_test.go
-rw-r--r-- root/root       670 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/mmap_linux.go
-rw-r--r-- root/root     20511 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/regexp.go
-rw-r--r-- root/root      7712 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/merge.go
-rw-r--r-- root/root      2835 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/regexp_test.go
-rw-r--r-- root/root     14683 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/write.go
-rw-r--r-- root/root       794 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/mmap_bsd.go
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/cgrep/
-rw-r--r-- root/root      1511 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/cgrep/cgrep.go
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/cindex/
-rw-r--r-- root/root      3626 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/cindex/cindex.go
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/csearch/
-rw-r--r-- root/root      3244 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/csearch/csearch.go
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/sparse/
-rw-r--r-- root/root      1696 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/sparse/set.go
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/
-rw-r--r-- root/root     10398 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/match.go
-rw-r--r-- root/root      5700 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/utf.go
-rw-r--r-- root/root      4848 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/copy.go
-rw-r--r-- root/root      1504 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/regexp.go
-rw-r--r-- root/root      5201 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/regexp_test.go
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/share/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/share/doc/
drwxr-xr-x root/root         0 2013-01-01 13:48 ./usr/share/doc/golang-codesearch/
-rw-r--r-- root/root       189 2013-01-01 12:39 ./usr/share/doc/golang-codesearch/changelog.Debian.gz
-rw-r--r-- root/root      2512 2013-01-01 12:54 ./usr/share/doc/golang-codesearch/copyright

Things to ask on the Go mailing lists

• Is it reasonable to ship the static libraries (pkg/) along with their source?

  • At least at the moment we should ship static libraries + their source: http://thread.gmane.org/gmane.comp.lang.go.general/81527/focus=81531 and http://thread.gmane.org/gmane.comp.lang.go.general/81527/focus=81687

• Would it be possible to switch to GNU triplets (x86_64-gnu-linux) instead of linux_arm, which doesn’t even properly differentiate armel and armhf?
  • No clear answer (yet?)
• Is the full source code of libraries required? (think of C and its header files)

  • Currently yes, this is tracked at http://golang.org/issue/2775

• What compiler should a distribution chose? gc, gccgo, both? Are there specific reasons/gotchas for any of the options?

  • Both should be present, but we need to decide for one with which we compile everything. One cannot mix gccgo-compiled code and gc-compiled code: http://thread.gmane.org/gmane.comp.lang.go.general/81527/focus=81687