Differences between revisions 21 and 22
Revision 21 as of 2012-07-09 11:55:05
Size: 3693
Editor: picca
Comment: add an interesting link
Revision 22 as of 2012-07-10 12:18:26
Size: 4397
Editor: ?Baptiste Carvello
Comment:
Deletions are marked like this. Additions are marked like this.
Line 78: Line 78:
A curiosity regarding the python extensions: they get imported using a wrapper around {{{__import__}}} called boost.python.import_ext.
This is defined in {{{cctbx_sources/boost_adaptbx/boost/python.py}}}. This wrapper does 3 things:

 * it checks for a libstdc++ mismatch with the python binary (useless in Debian)
 * it beautifies the import exception messages (nice to have, but unimportant)
 * it calls dlopen with RTLD_GLOBAL. People in the Boost community seem to think that this is necessary for correct operation in corner cases.

So we probably have to keep using the wrapper and have all our python packages with extensions depend on boost_adaptbx (and possibly some parts of libtbx).

cctbx packaging

peoples interested by this packaging effort

  • Baptiste Carvello
  • picca

  • Radostan Riedel

you can find the git repository here cctbx repo

the current ITP is here 679905

upstream

the main website is there http://cctbx.sourceforge.net/

build dependencies

mmdb

679982

gpp4

679988

clipper

679990

bundled

  • libann-dev
  • libboost-python-dev
  • libfftw3-dev
  • python-setuptools (>= 0.6.10)

  • python-all-dev
  • debhelper (>= 7)

  • python-all-dev (>= 2.3)

  • python (>= 2.3.5-11)

  • libboost-thread-dev
  • scons
  • python-support
  • libtool
  • coreutils

package organisation

As explain by the upstream

python module organisation generated using this script cctbx-depends.png

  • Red boxes : python modules,
  • green boxes are others (C++ or external).
  • Black arrows are run-time dependancies
  • Red arrows are build-time dependancies
  • Green arrows are optional dependancies.

libraries generated

  • libann.so
  • libboost_python.so
  • libboost_thread.so
  • libcbf.a
  • libccp4io.a
  • libcctbx_sgtbx_asu.so
  • libcctbx.so
  • libclipper.so
  • libiotbx_mtz.so
  • libiotbx_pdb.so
  • libmmtbx_masks.so
  • libomptbx.so
  • librstbx.so
  • libscitbx_boost_python.so
  • libscitbx_minpack.so
  • libscitbx_slatec.so
  • libsmtbx_refinement_constraints.so
  • libspotfinder.so

is it all ?, API/ABI stability ?

python modules/extensions

A curiosity regarding the python extensions: they get imported using a wrapper around __import__ called boost.python.import_ext. This is defined in cctbx_sources/boost_adaptbx/boost/python.py. This wrapper does 3 things:

  • it checks for a libstdc++ mismatch with the python binary (useless in Debian)
  • it beautifies the import exception messages (nice to have, but unimportant)
  • it calls dlopen with RTLD_GLOBAL. People in the Boost community seem to think that this is necessary for correct operation in corner cases.

So we probably have to keep using the wrapper and have all our python packages with extensions depend on boost_adaptbx (and possibly some parts of libtbx).

...

TODO

  • solve the API/ABI stability problems for libs
  • identify the bundeled libraries (boost ...)
  • what should be do with this future problem (could you elaborate ?)

  • license check of the remaining files
  • package missing build-dep
  • repack to remove all bundled libraries, it should save some space.
  • remove the cctbx bundled in objcryst-fox once done
  • take care of the pickle object libtbx_env
  • run all tests at the end of the build (python setup.py check ?)
  • what about the documentation ?
  • provide pkg-config files for public shared libraries (patch for upstream)

Build system

  • Modify configure.py to support "--use-system-libraries"
    • skip shlibs build which are already available in Debian.
    • link with the system libraries.
    • use pkg-config as much as possible for library detection
  • version the shared libraries using scons
  • support a --prefix option
  • the install target should support DESTDIR
  • can we customise the build with the traditionals CFLAGS, CXXFLAGS, CPPFLAGS and LDFLAGS ?
    • we need -O2 and -g for debugging symbols.
  • add a setup.py for python modules (autogenerated ?)
  • add a clean target ?

Problems

  • upstream is linking additional symbols into libcbf. libcbf is built statically so we can don't need libcbf-dev
    • can be backported into the cbf upstream ?
    • check with the cbf Debian maintainer
  • upstreams libann is build with with self_includes which makes it incompatible with libann in Debian.
    • can be backported into the ann upstream ?
    • check with the ann Debian maintainer
  • fftw3tbx is missing a header file.
  • useless file in the current source package cctbx_sources/lapack_fem/xerbla.f.orig
  • what should we do with all the dispatcher in the bin directory ?

http://cci.lbl.gov/~hohn/scitbx-tour.html#Installation