Translation(s): српски


This is the homepage outlining the progress being made on packaging Tensorflow for debian. Any information gleamed from various threads in the debian-ai mailing list will be made available here.

Tensorflow packaging

The Salsa repo is here: https://salsa.debian.org/deeplearning-team/tensorflow The package is classic quilt (3.0) source, so patches outside the debian dir should be managed in quilt. There are upstream, pristine-tar and master branches, which I believe is the conventional git-buildpackage layout.

Background

TensorFlow is an important package for Debian due to its importance as one of the top 2 Machine Learning frameworks alongside PyTorch. Packaging work is ongoing since the build system bazel was uploaded in Oct 2020. This work is being done under the Debian Deep Learning team banner, co-ordinated on the debian-ai mailing list.

Because bazel is new (and has some ideas antithetical to debian packaging) there is no debhelper support so packaging is relatively hard work, and includes developing workflows and patterns for debian builds of bazel-using packages.

History

Before bazel was packaged in debian it was very difficult to build TensorFlow. Mo Zhou <cdluminate@gmail.com> worked around it by using an alternate build system and succeeded in uploading a package to experimental [https://tracker.debian.org/news/1015398/accepted-tensorflow-1101dfsg-a2-source-into-experimental/|experimental]] However it was removed the following year due to the maintenance burden for a fast-moving package being too high. The Debian-Med team along with Google managed to package Bazel as bazel-bootstrap in October 2020, enabling a more conventional packaging effort to proceed.

Current Status

Wookey <wookey@debian.org> with help from debian-bazel, debian-ai and debian-med/science teams, and Francis Murtagh has recently (1st half of 2021) been working on getting it in a state suitable for upload.

Version 2.3.1 is being packaged initially. We will move to 2.4 as soon as 2.3.1 is basically working.

An initial version was uploaded to NEW (targeting experimental) on 10th of June 2021, and accepted into the archive on 9th of Feb 2022.

The current packaging produces three packages: libtensorflow-cc2 (C++ library) libtensorflow-framework2 (framework library) libtensorflow-dev (headers, pkgconfig files)

A tensorflowlite package is currently in progress.

The c-library version does not yet build because it depends on unpackaged things: google-storageapi and various aws-* packages The python bindings have not yet been packaged but is a high priority.

https://tracker.debian.org/pkg/tensorflow

Contact

Conversations are taking place under the debian-ai@lists.debian.org mailing list and it is best to keep TensorFlow related discussions there if possible.

Also the debian-ai IRC channel on the OFTC IRC network: * #debian-ai: https://lists.debian.org/debian-ai/

To Do

Things still to do:

Bazel issues

Open Questions

Current open questions arising from mailing list conversations:

Commands to attempt build

# Commands to attempt to build packages locally if useful:
DEBEMAIL="firstname.lastname@domain.com"
>DEBFULLNAME="Firstname Lastname"
export DEBEMAIL DEBFULLNAME
git clone https://salsa.debian.org/deeplearning-team/tensorflow.git salsa
wget http://wookware.org/files/tensorflow_2.3.1.orig.tar.xz
cd salsa
# Install the build-depends of tensorflow
mk-build-deps --install --tool 'apt-get --yes --no-install-recommends' --remove ./debian/control
rm -rf tensorflow-build-deps_*.buildinfo
rm -rf tensorflow-build-deps_*.changes
debuild -sa -b -uc -us --jobs=8 || true

Adding Patches to Source

TensorFlow packaging uses Quilt for patches rather than Git. A very useful tutorial on its usage can be found here: https://raphaelhertzog.com/2012/08/08/how-to-use-quilt-to-manage-patches-in-debian-packages/

Entry into Experimental

Wookey has managed to get Tensorflow through the NEW queue and into Experimental: https://tracker.debian.org/news/1301497/accepted-tensorflow-231-1-source-amd64-into-experimental-experimental/

Building Tensorflow Lite

Tensorflow Lite has not yet been packaged, figuring out how to add it as a target for Bazel is the issue, there is however the possibility of building it with CMake from v2.4 onwards:
https://www.tensorflow.org/lite/guide/build_cmake

UPDATE: Tensorflow Lite packaging branch has been added to Salsa Repo: https://salsa.debian.org/deeplearning-team/tensorflow/-/tree/experimental/tflite_packaging/debian

--config=native_arch_linux
TF_SET_ANDROID_WORKSPACE=False

 cc_binary(
         name = "libtensorflow_lite_all.so",
         linkshared = 1,
         deps = [
                 "//tensorflow/lite:framework",
                 "//tensorflow/lite/kernels:builtin_ops",
         ],
 )

For more info on this experimental branch contact: Francis Murtagh <francis.murtagh@arm.com> - Any feedback is welcome.

UPDATE: Packaging fails when building Tensorflow and ?TfLite together, the build log can be found here: http://www.wookware.org/files/tensorflow_2.3.1-2_amd64-2021-09-14T23:00:37Z.build

ERROR: /home/wookey/packages/tensorflow/salsa/tensorflow/core/kernels/BUILD:4659:18: Couldn't build file tensorflow/core/kernels/_objs/conv_ops/conv_ops_fused_double.pic.o: C++ compilation of rule '//tensorflow/core/kernels:conv_ops' failed (Exit 1): gcc failed: error executing command 
  (cd /tmp/.cache/bazel/_bazel_wookey/5a73853b1764682f4fdcfb56b63560fb/execroot/org_tensorflow && \
  exec env - \
    PATH=/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11 \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/usr/bin/python3 \
    PYTHON_LIB_PATH=/usr/lib/python3/dist-packages \
    TF2_BEHAVIOR=1 \
    TF_CONFIGURE_IOS=0 \
    TF_ENABLE_XLA=1 \

UPDATE Nov 2023: A separate attempt at packaging was added to Salsa as tensorflowlite: https://salsa.debian.org/deeplearning-team/tensorflowlite

# TODO:
# * Get tensorflow-lite.so building for runtime package
# * package fft2d
#     https://github.com/petewarden/OouraFFT
#     mentioned in:
#     tensorflow/lite/tools/cmake/modules/fft2d.cmake
#
# lintian:
#
#E: tensorflowlite source: license-problem-non-free-img-lenna [tensorflow/core/lib/bmp/testdata/lena.bmp]
#E: tensorflowlite source: license-problem-non-free-img-lenna [tensorflow/core/lib/gif/testdata/lena.gif]
#E: tensorflowlite source: license-problem-non-free-img-lenna [tensorflow/core/lib/png/testdata/lena_gray.png]
#E: tensorflowlite source: license-problem-non-free-img-lenna [tensorflow/core/lib/png/testdata/lena_palette.png]
#E: tensorflowlite source: license-problem-non-free-img-lenna [tensorflow/core/lib/png/testdata/lena_palette_trns.png]
#E: tensorflowlite source: license-problem-non-free-img-lenna [tensorflow/core/lib/png/testdata/lena_rgba.png]
#E: libtensorflowlite-dev: non-standard-toplevel-dir [debian/]
#E: tensorflowlite source: source-is-missing [tensorflow/core/grappler/costs/graph_properties_testdata/large_graph.pbtxt.html]
#E: tensorflowlite source: version-substvar-for-external-package Depends ${binary:Version} libtensorflowlite-dev -> libtensorflowlite [debian/control:30]

Packages depending on Tensorflow

Package

Info

Chiron

https://github.com/haotianteng/chiron

Tensorboard

https://lists.debian.org/debian-ai/2020/11/msg00019.html

NanoVar

https://github.com/cytham/nanovar/blob/master/requirements.txt

ArmNN Delegate

https://arm-software.github.io/armnn/21.05/md_delegate__build_guide_native.xhtml