This page is about packaging golang.org libraries in Debian.
Table of contents:
Contents
Facts
The go compiler (gc) needs to have the source code of a package available when compiling code that uses it. E.g. if your code uses github.com/mstap/godebiancontrol, gc needs $GOPATH/src/github.com/mstap/godebiancontrol (can that be fixed?)
Perl uses libXX-perl for package names (e.g. libnet-inet6glue-perl for Net::INET6Glue), while Ruby switched to ruby-xx (e.g. ruby-tioga). TODO: Is there some rationale behind either decision? Which is “better”?
- Both Perl and Ruby use /usr/lib/{perl,ruby} and store source as well as shared objects there.
Package naming
For github.com/mstap/godebiancontrol (which contains "go" already), the resulting Debian package name is golang-godebiancontrol-dev.
We use the -dev suffix to keep the namespace clean in case go supports shared libraries, which would then be shipped as golang-godebiancontrol.
Where to store go src/pkg data?
Go libraries (not binaries!) are present in Debian only for the purpose of building binary packages. They should not be used directly for Go development. Instead, the well-documented and platform independent way of using “go get” should be used.
Why should I use “go get” instead of apt-get to install Go libraries?
- By using “go get” you are forced to set up the environment variable $GOPATH which you need for other “go” commands (e.g. “go build”, “go test”, etc.).
Debian packages install files to system locations, which users cannot modify. That means that users would not be able to upgrade packages with the canonical “go get -u <package>”. Even worse, when using sudo to forcibly modify the system files, it still would not work since no VCS information is contained in the Debian packages.
I want to build a Go binary Debian package
- libraries are installed to /usr/lib/gocode
- When building Go Debian packages, GOPATH is set to “/usr/lib/gocode”.
Making gc find the system go libraries
The first possibility that came to my mind was adding export GOPATH=/usr/lib/gocode to /etc/profile, but there are two problems with that:
- /etc/profile (and /etc/profile.d/*.sh) is not executed by every shell, e.g. not by zsh
Debian Policy actually forbids modifying it and says that programs need to work without environment variables in a reasonable way (see “Environment variables” in http://www.debian.org/doc/debian-policy/ch-opersys.html)
Therefore, I think the best way is to patch our version of go to always add /usr/lib/gocode to c.GOPATH in src/pkg/go/build/build.go.
Multi-Arch/cross-compiling
golang-go is currently available for amd64 (linux_amd64), armel (linux_arm), armhf (linux_arm), i386 (linux_386).
Note that the identifier Go uses is “linux_arm” for both armel and armhf while the files differ:
Binary files /tmp/armhf/usr/lib/go/pkg/linux_arm/unicode.a and /tmp/armel/usr/lib/go/pkg/linux_arm/unicode.a differ
Multi-arch in itself doesn’t make sense: gc produces static binaries, so to run a program that was compiled for i386 on amd64, you don’t need any libraries.
For cross-compiling, it doesn’t matter which pre-compiled packages you have installed (e.g. linux_amd64) because the .go source files are shipped, too. go build will just re-compile the libraries then:
$ echo $GOPATH /usr/lib/gocode:/tmp/src/golang $ GOARCH=386 CGO_ENABLED=0 PATH=~/go-i386/go/bin:$PATH go build -v github.com/mstap/godebiancontrol debtest $ file debtest debtest: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
TODO
In the long term, we should request a section “golang”, see 390609 for an example. This step should follow after there are a number of libraries.
- lintian check for missing Built-Using
Example binary + library packaging
debian/rules:
#!/usr/bin/make -f
#export DH_VERBOSE=1
# GOPKG is the upstream path which you would normally “go get”.
# Using it allows us to build applications without patching locations.
GOPKG := code.google.com/p/codesearch
# Temporary working directory to which the source will be copied, then
# compiled.
TMPGOPATH = $(CURDIR)/debian/tmp/usr/lib/gocode
override_dh_auto_install:
mkdir -p ${TMPGOPATH}/src/${GOPKG}
# Copy all .go files to /usr/lib/gocode (we compile and ship).
find . -path ./debian -prune -o -type f -name "*.go" -exec tar cf - {} + | (cd "${TMPGOPATH}/src/${GOPKG}" && tar xvf -)
# Ensure that GOPATH is clean: It should only contain the temporary
# /usr/lib/gocode containing the package we want to install and the
# /usr/lib/gocode of the system we are building on. It should
# specifically NOT contain the user’s local ~/gocode.
GOPATH=${TMPGOPATH}:/usr/lib/gocode go install -v ${GOPKG}/...
# see also http://lists.debian.org/debian-devel/2011/03/msg00852.html
# The Built-Using field makes sure the required sources are kept around
# until all binaries that were built from the sources are deleted.
# Necessary since Go links statically and thus there is no dependency
# on the library in the resulting package.
deps=`dpkg-query -f='$${source:Package} (= $${source:Version}), ' -W $$(perl -MParse::DebControl -E 'my $$p = Parse::DebControl->new(); my $$data = $$p->parse_file("./debian/control"); my @deps = grep { /^\s*golang-/ } split(",", $$data->[0]->{"Build-Depends"}); say join(" ", @deps)')`; \
for pkg in `dh_listpackages`; do \
echo -n "misc:Built-Using=$$deps" >> debian/$$pkg.substvars; \
done
%:
dh $@ --paralleldebian/control:
Source: codesearch
Section: utils
Priority: extra
Maintainer: Michael Stapelberg <stapelberg@debian.org>
Build-Depends: debhelper (>= 8.0.0), golang-go, libparse-debcontrol-perl
Standards-Version: 3.9.4
Homepage: https://code.google.com/p/codesearch/
Package: codesearch
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Built-Using: ${misc:Built-Using}
Description: regular expression search over large bodies of source code
Code Search is a tool for indexing and then performing regular expression
searches over large bodies of source code. It is a set of command-line
programs written in Go.
.
For background and an overview of the commands, see Regular Expression
Matching with a Trigram Index:
http://swtch.com/~rsc/regexp/regexp4.html
Package: golang-codesearch-dev
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Built-Using: ${misc:Built-Using}
Description: regular expression search over large bodies of source code
Code Search is a tool for indexing and then performing regular expression
searches over large bodies of source code. It is a set of command-line
programs written in Go.
.
For background and an overview of the commands, see Regular Expression
Matching with a Trigram Index:
http://swtch.com/~rsc/regexp/regexp4.html
.
This package contains the source and pre-compiled package files.debian/codesearch.install:
debian/tmp/usr/lib/gocode/bin/* /usr/bin
debian/golang-codesearch-dev.install:
usr/lib/gocode/src usr/lib/gocode/pkg
For a binary-only package, use just the first .install file, for a library-only package, use just the second .install file.
Resulting package file lists
$ dpkg -c ../codesearch_0.01+hg20121021-1_amd64.deb drwxr-xr-x root/root 0 2013-01-01 13:48 ./ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/bin/ -rwxr-xr-x root/root 1385040 2013-01-01 13:48 ./usr/bin/cgrep -rwxr-xr-x root/root 1389728 2013-01-01 13:48 ./usr/bin/cindex -rwxr-xr-x root/root 1520272 2013-01-01 13:48 ./usr/bin/csearch drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/share/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/share/doc/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/share/doc/codesearch/ -rw-r--r-- root/root 189 2013-01-01 12:39 ./usr/share/doc/codesearch/changelog.Debian.gz -rw-r--r-- root/root 415 2012-10-21 15:24 ./usr/share/doc/codesearch/README -rw-r--r-- root/root 2512 2013-01-01 12:54 ./usr/share/doc/codesearch/copyright $ dpkg -c ../golang-codesearch_0.01+hg20121021-1_amd64.deb drwxr-xr-x root/root 0 2013-01-01 13:48 ./ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/pkg/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/p/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/p/codesearch/ -rw-r--r-- root/root 620548 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/p/codesearch/index.a -rw-r--r-- root/root 313606 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/p/codesearch/regexp.a -rw-r--r-- root/root 24074 2013-01-01 13:48 ./usr/lib/gocode/pkg/linux_amd64/code.google.com/p/codesearch/sparse.a drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/ -rw-r--r-- root/root 1751 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/read_test.go -rw-r--r-- root/root 3451 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/write_test.go -rw-r--r-- root/root 887 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/mmap_windows.go -rw-r--r-- root/root 10584 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/read.go -rw-r--r-- root/root 2198 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/merge_test.go -rw-r--r-- root/root 670 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/mmap_linux.go -rw-r--r-- root/root 20511 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/regexp.go -rw-r--r-- root/root 7712 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/merge.go -rw-r--r-- root/root 2835 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/regexp_test.go -rw-r--r-- root/root 14683 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/write.go -rw-r--r-- root/root 794 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/index/mmap_bsd.go drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/cgrep/ -rw-r--r-- root/root 1511 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/cgrep/cgrep.go drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/cindex/ -rw-r--r-- root/root 3626 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/cindex/cindex.go drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/csearch/ -rw-r--r-- root/root 3244 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/cmd/csearch/csearch.go drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/sparse/ -rw-r--r-- root/root 1696 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/sparse/set.go drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/ -rw-r--r-- root/root 10398 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/match.go -rw-r--r-- root/root 5700 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/utf.go -rw-r--r-- root/root 4848 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/copy.go -rw-r--r-- root/root 1504 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/regexp.go -rw-r--r-- root/root 5201 2012-10-21 15:24 ./usr/lib/gocode/src/code.google.com/p/codesearch/regexp/regexp_test.go drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/share/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/share/doc/ drwxr-xr-x root/root 0 2013-01-01 13:48 ./usr/share/doc/golang-codesearch/ -rw-r--r-- root/root 189 2013-01-01 12:39 ./usr/share/doc/golang-codesearch/changelog.Debian.gz -rw-r--r-- root/root 2512 2013-01-01 12:54 ./usr/share/doc/golang-codesearch/copyright
Things to ask on the Go mailing lists
- Is it reasonable to ship the static libraries (pkg/) along with their source?
At least at the moment we should ship static libraries + their source: http://thread.gmane.org/gmane.comp.lang.go.general/81527/focus=81531 and http://thread.gmane.org/gmane.comp.lang.go.general/81527/focus=81687
- Would it be possible to switch to GNU triplets (x86_64-gnu-linux) instead of linux_arm, which doesn’t even properly differentiate armel and armhf?
- No clear answer (yet?)
- Is the full source code of libraries required? (think of C and its header files)
Currently yes, this is tracked at http://golang.org/issue/2775
- What compiler should a distribution chose? gc, gccgo, both? Are there specific reasons/gotchas for any of the options?
Both should be present, but we need to decide for one with which we compile everything. One cannot mix gccgo-compiled code and gc-compiled code: http://thread.gmane.org/gmane.comp.lang.go.general/81527/focus=81687