Status: brain-dump
Last-Updated: 2024-09-01
Summary
Some of our dependency handling is suboptimal, as it requires manually tracking them where tools could automatically inject and handles these dependencies without needing to duplicate information and keep it up-to-date.
We are currently tracking lots of dependencies manually, which could be generated automatically by the tooling, or overridden if wrong.
This would ideally be handled by a new generalized and extensible tool that generates dependencies, for example dpkg-gendepends.
Static Linking
We currently handle this by populating substvars that need to be kept in sync with both the build dependencies, and whether the depended on packages contains a license that requires tracking.
Whether the static linking needs tracking for license or security/rebuilding purposes depends on whether the user of the code is using it.
Some packages are inherently always to be statically linked such as Golang or Rust -dev packages. Others have a dual nature, such as C static libraries shipped with the -dev package where the .so symlink is provided. In this case how the package is used depends on the user, which is the one that would need to be annotated (well its build dependency). This could be done through new syntax, for example Build-Depends: libfoo-dev {static}, libbar-dev. Although this potential new syntax would need to be carefully considered, and perhaps make very generic, as we are running out of bracket-like characters in dependency fields.
This information is currently either generated via substvars manually in debian/rules via hand-crafted dpkg-query calls, or via a debhelper helper like dh-builtusing or a language-specific helper such as dh-golang. Depending on the substvars usage pattern, this might be problematic, as substvars (currently) cannot express all package names as part of their names (such as pkgname:Something.
To be able to fully automate its generation in a way that incurs less duplication and localizes the knowledge of what is being provided and how it is being used into the respective packages, we need the license information for the provider in a location that is vendor-neutral and can be guaranteed to exist (so not _/usr/share/doc/pkg/_), this would imply either a field in the binary package DEBIAN/control or a new binary package control file, so that they can be accessed from the dpkg database.
If the package has a dual nature, then the depending package would need a way to specify how it is using the depended on package (dynamic or static linking, with dynamic being the default).
License compliance
We only need this for licenses that require it. The field use is Built-Using.
The information is machine parseable in debian/copyright, which would ideally be installed as is into binary packages, but for now it can be extracted and injected into new field.
Code embedding
This should be used when embedding code, that has no license requirements to keep the used code around as part of a release or the archive. The field used is Static-Built-Using.
To be able to fully and automatically track statically linked libraries we could append -Wl,-Map=%.dpkg-linkmap to LDFLAGS and then parse the LOAD entries. There are several drawbacks with this approach:
The linker -Map flag does not generate parent directories, so something like -Wl,-Map=debian/.build/dpkg/static-libs/%.dpkg-linkmap cannot be used as ld will then fail to create the map file.
The upstream build system might already be using the -Wl,-Map flag, so ours would override it and might break the build. (This would need to be opt-in, or part of a dpkg-build-api level then be able to opt-out.)
- The upstream build system might not be taking LDFLAGS into account, although this is a common occurrence and the packaging might already be assigning LDFLAGS into CFLAGS or other language-specific flags.
pkg-config files
Generate automatic dependencies for -dev packages from .pc files. There's a branch in dpkg.
.so linking
Generate automatic dependencies for libfoo-dev to libfooN from -so symlinks to .so.N. There is a branch in dpkg.
External dependency generators?
Perhaps if the framework is extensible and modular, this could make it possible to hook additional dependency generators from other "helpers" into the tool to handle this in a generic way. An example of this are the various language-specific dependency generators, or the request to integrate the gir dependency generator into dpkg somehow (see 548463).
Several of the existing tools in the form of dh_ helpers including debhelper itself could be candidates for modularization.