Differences between revisions 1 and 173 (spanning 172 versions)
Revision 1 as of 2010-10-02 15:29:21
Size: 1799
Comment: Issues to package sage
Revision 173 as of 2015-04-13 16:23:33
Size: 11590
Editor: ?JeromeBenoit
Comment: surf-alggeo is now in the NEW queue
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
Some people within [[DebianScience]] would like to package again Sage for Debian. An old version of Sage got packaged long time ago by Tim Abbott, but, because of lack of maintainance, it became totally unusable and was removed from Debian. Some people within [[DebianScience]] would like to package again Sage for Debian. An old version of Sage got packaged long time ago by Tim Abbott, but, because of lack of maintainance, it became totally unusable and was removed from Debian. The status pages ([[http://people.debian.org/~thansen/debian-sage-status.html|master branch]], [[http://people.debian.org/~thansen/debian-sage-dev-status.html|develop branch]]) give an overview about package versions in Sage and Debian. If you want to help out, pick one of the open tasks or issues below and get going. Or contact us via the [[http://lists.debian.org/debian-science/|mailing list]] or in #debian-science on irc.oftc.net.
Line 3: Line 3:
Here is the current list of open issues:
 * flintqs / SIMPQS issue. FlintQS should probably be removed as a SPKG in Sage, but it may require work (qsieve). Bill Hart may be able to comment further (William Stein).
 * opencdk / gnutls. William Stein thinks it is only needed to "provide a secure SSL mode for the Sage notebook". So we can probably use the Debian version of gnutls, but that will require assessment.
 * Sage does not yet support scipy 0.8 because it depends on numpy-1.4 and "Sage is not going there yet". Strangely, Debian sid currently has scipy 0.7.2 (good), but includes numpy 1.4.1 (ut-oh). So that will probably create issues.
 * This pickles patch is still not integrated into Python upstream, but it is needed for Sage: http://bugs.python.org/issue7689.
 * This python2.6 issue probably affects sid: http://bugs.python.org/issue7491 which breaks stuff, witness: http://github.com/cschwan/sage-on-gentoo/issues#issue/1
 * pexpect issue (Debian pkg: python-pexpect): Debian ships version 2.3, but Sage requires version 2.0 (though one of the necessary patches is already in 2.4). Plotting in the notebook is likely to break with newer versions that are in Debian; and >2.0 is reported to be slow.
 * ecl is OK unless compiled with Unicode support (http://github.com/cschwan/sage-on-gentoo/issues/closed#issue/2). Debian may do that, so it could be an issue.
 * Unpackaged dependencies:
  * ratpoints
  * cliquer
 * Dependencies that need to be updated:
  * libflint
  * libfplll0
  * gfan
  * lcalc
  * m4ri
= Steps to get an up-to-date Sage standard suite in Debian =

== Building Sage with Debian packages ==

We have a git repository ([[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagemath.git;a=summary|sagemath]]) to document the steps we currently do to experiment with building Sage against Debian packages. Important elements are the build dependencies of Sage, patches for Sage and a script called "pruner" that uses automake to determine if the necessary packages are installed (see also [[https://lists.debian.org/debian-science/2015/02/msg00024.html|this mail]]).

It is recommended to do these steps one by one:

||{{{git clone --recursive url/to/sagemath.git}}}|| Recursive git clone to also get sage which is included as a git submodule.||
||{{{sudo mk-build-deps --install debian/control}}}|| Install build dependencies that are in unstable (requires Debian unstable). ||
||{{{quilt push -a}}} || Apply patches to Sage. ||
||{{{debian/rules prune}}} || Run the pruner script. ||
||{{{debian/rules build}}} || Build sage and run the test suite (includes pruning if necessary, but not patching). ||

The output of the pruner tells you in the end which dependencies were not found on the machine. These will be downloaded and built in the build step, before Sage itself is built. In the moment the pruner should output something like the following, and it is for now recommended to use these three packages from Sage.
{{{
libgap will not come from debian!
singular will not come from debian!
jmol will not come from debian!
95 system packages will be used
}}}

The following commands clean everything up properly.
||{{{debian/rules clean}}}||Clean up after prune and build.||
||{{{quilt pop -a}}}||Unapply patches.||
||{{{cd sage && git clean -dfx}}}||Clean sage folder (git does not show untracked files from the built due to .gitignore).||
||{{{rm -rf ~/.local/lib/python2.7/site-packages/sage*}}}||Remove sage python library that is installed during the build process.||

Finally, there are still dependencies that cannot yet be satisfied with packages from unstable. This changes frequently. For now (branch sage-6.5 in the sagemath git repository) the following packages should be installed from git:
 * [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/cddlib.git;a=summary|cddlib]]
 * [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/rubiks.git;a=summary|rubiks]]
 * [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagenb.git;a=summary|sagenb]]
 * [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/singular.git;a=summary|singular]] (use the tag debian/3-1-6+dfsg-1, Sage will use its own singular but this is needed anyway)

Some Debian packages have to be rebuilt to include patches:
 * [[https://packages.qa.debian.org/p/pari.html|pari]] with [[http://git.sagemath.org/sage.git/plain/build/pkgs/pari/patches/cb_pari_err_handle.patch|cb_pari_err_handle.patch]] (see DebianBug:777237)
 * [[https://packages.qa.debian.org/m/maxima.html|maxima]] with the patch attached to DebianBug:779804
 * [[https://packages.qa.debian.org/e/ecl.html|ecl]] with the workaround described in DebianBug:777473

With these changes the compilation of Sage 6.5 should finish and one can concentrate on fixing failing doctests.

For compiling a development version of Sage 6.6 (branch master) one should also update the following Debian packages:
 * [[https://packages.qa.debian.org/g/gap.html|gap]] to version 4.7.7 from experimental
 * [[https://packages.qa.debian.org/c/cython.html|cython]] to version 0.22

Here the compilation currently stops due to an incompatible pari version.

== Open tasks ==


||<tablestyle="border:1pt solid"> '''Package''' ||<style="width: 70pt"> '''Bug / Link''' || '''Assigned''' ||'''Comments''' ||
||<-4 style="border-top:1pt solid; border-bottom:1pt solid"> '''Packages that need a new maintainer and an upload to Debian:''' ||
|| jmol || || || There is a request for help, see DebianBug:719330 ||
|| libgap || || || Can maybe be replaced by a wrapper for GAP. ||
|| pexpect-2.0 || || || (Debian pkg: python-pexpect): Debian ships version 2.4, but Sage requires version 2.0 (though one of the necessary patches is already in 2.4). Plotting in the notebook is likely to break with newer versions that are in Debian; and >2.0 is reported to be slow. There is a new upstream version so Sage might update, see [[http://trac.sagemath.org/sage_trac/ticket/10295|this sage bug]]. ||
|| [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/singular.git;a=summary|singular]] || [[https://packages.qa.debian.org/s/singular.html|PTS]] || || singular 3.1.6 is packaged in debian-science's git and Sage >= 6.2 has also 3.1.6; a package for singular 4.0.1 is in the NEW queue, see DebianBug:764475. ||
||<-4 style="border-top:1pt solid; border-bottom:1pt solid"> '''Sage components that need a new maintainer and an upload to Debian:''' ||
|| [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagenb.git;a=summary|sagenb]] || || || the dependencies should be in Debian now (DebianBug:691509, DebianBug:703541, DebianBug:703531, DebianBug:703533, DebianBug:703534, DebianBug:703535 and DebianBug:703536) ||
|| sagetex || || || ||
||<-4 style="border-top:1pt solid; border-bottom:1pt solid"> '''Softwares that need to be packaged:''' ||
|| 4ti2 || || || Singular 4 depends on [[http://www.4ti2.de/|4ti2]]; a package is in the NEW queue, see DebianBug:769997 ||
|| normaliz || || || Singular 4 depends on [[http://www.home.uni-osnabrueck.de/wbruns/normaliz/|normaliz]]; an ITP has been filled to package normaliz 2.12.2, see DebianBug:776019 ||
|| surf-alggeo || || || Singular 4 depends on [[http://surf.sourceforge.net/|surf]]; a package is in the NEW queue, see DebianBug:777132 ||
||<-4 style="border-top:1pt solid; border-bottom:1pt solid"> '''Packages that are (hopefully) not needed:''' ||
|| cephes || || || The Cephes Mathematical Library is only needed for systems where the libc doesn't provide appropriate mathematical functions (cygwin and FreeBSD) and so is not required for Debian.||
|| iconv || || || is part of the libc ||
|| [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/mpir.git;a=summary|mpir]] || DebianBug:708391 || || mpir is supposed to be ABI-compatible with [[http://packages.qa.debian.org/g/gmp.html|gmp]]. A lot of math packages use gmp and we hope we can have a purely gmp based Sage in Debian. Combining the two will most likely lead to conflicts, using mpir we would need an extra mpir version of many packages. ||

















== Other known or suspected issues ==

 * ECL has a strange bug in debian (see DebianBug:777473).
 * sage expects to use both ecl and maxima, together, but maxima in debian is using gcl and has no support for ecl ; Snark has fixes for that, but some things (like the above ECL problem) need to clear out before they can be pushed in debian. (see DebianBug:779804)
 * pari in sage's latest has a patch and uses a header file which aren't in the debian package, so one currently needs to modify a few things by hand to get going (see DebianBug:777237)


== Information bits that are no issues ==

 * Sage upstream would like to get rid of flintqs since some recent versions of flint ship with a quadratic sieve (see [[http://trac.sagemath.org/sage_trac/ticket/11792|this ticket]], but it hasn't been done yet). This [[https://groups.google.com/d/msg/flint-devel/lmRdEPV6el8/JNbZ4eTy_n0J|mail of William Hart]] tells more about the state of the quadratic sieve code in both flint and flint2. flintqs was uploaded to Debian for the meantime (but blocked in unstable).


== New build system for Sage (the distribution) ==

this is slightly out of date, also, Sage (the community) is not interested in using packages that another distribution provides.

The Sage library is getting an autotools build system in the course of a [[SummerOfCode2013/Projects#Get_Sage_ready_for_Debian|Google Summer of Code project]]. With the new build system, it can be built against system libraries.
There are tarballs with the code at [[http://tool.em.cs.uni-frankfurt.de/~felix/sage/]]. (There is also a [[https://github.com/felix-salfelder/sage/tree/WIP512|git repository]] but the tarballs are recommended). Note that this is work in progress and problems are expected, but we are happy about testers. The transition of this work into Sage is coordinated via a bunch of [[http://trac.sagemath.org/query?reporter=felixs|Sage tickets]]. We have a Debian package [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagelib.git;a=summary|sagelib]] that currently builds metapackages that help installing the build dependencies of Sage. To test building Sage against Debian packages, one should consult the [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagelib.git;a=blob;f=debian/control|control file]]. The dependencies of the metapackage sagelib-builddeps-git still need special treatment and can't be just installed from Debian.

The following Debian packages should be installed from the git repositories:
 * [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagenb.git;a=summary|sagenb]]
 * [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/singular.git;a=summary|singular]] (branch master-3-1-5, build with DEB_BUILD_OPTIONS=nodoc)

For the remaining packages where the Debian package is missing or is missing features required by Sage, it is currently most convenient to use the versions from Sage. This is done by configuring with

{{{
../configure \
--disable-all \
--enable-{boehm_gc,cython,ecl,flintqs,genus2reduction,maxima,pari,pexpect,pynac,rubiks,sagelib,sagetex,tachyon} \
--with-gmp
}}}

== New build system for Sage (the library) ==

There is some work in progress trying to make sagelib work on top of any software distribution. we use autotools to sort out dependencies, run time paths and provide standard build/installation routines. unlike the first approach, this is heading towards a fork and will likely not play well with the current upstream development process. Experimental tarballs are available at [[http://tool.em.cs.uni-frankfurt.de/~felix/sagelib/]], there is no public git repo yet.


== Old sagemath package ==

The old, buggy sagemath Debian package source code is available at http://snapshot.debian.org/package/sagemath/3.0.5dfsg-5.1/

== Links ==
 * [[http://people.debian.org/~thansen/debian-sage-status.html|Sage in Debian status page (master branch)]]
 * [[http://people.debian.org/~thansen/debian-sage-dev-status.html|Sage in Debian status page (develop branch)]]
 * [[http://wiki.debian.org/JulienPuydt|Graph of dependencies between Sage packages]]
 * [[http://fedoraproject.org/wiki/SIGs/SciTech/SAGE|Sage in Fedora]]

Some people within DebianScience would like to package again Sage for Debian. An old version of Sage got packaged long time ago by Tim Abbott, but, because of lack of maintainance, it became totally unusable and was removed from Debian. The status pages (master branch, develop branch) give an overview about package versions in Sage and Debian. If you want to help out, pick one of the open tasks or issues below and get going. Or contact us via the mailing list or in #debian-science on irc.oftc.net.

Steps to get an up-to-date Sage standard suite in Debian

Building Sage with Debian packages

We have a git repository (sagemath) to document the steps we currently do to experiment with building Sage against Debian packages. Important elements are the build dependencies of Sage, patches for Sage and a script called "pruner" that uses automake to determine if the necessary packages are installed (see also this mail).

It is recommended to do these steps one by one:

git clone --recursive url/to/sagemath.git

Recursive git clone to also get sage which is included as a git submodule.

sudo mk-build-deps --install debian/control

Install build dependencies that are in unstable (requires Debian unstable).

quilt push -a

Apply patches to Sage.

debian/rules prune

Run the pruner script.

debian/rules build

Build sage and run the test suite (includes pruning if necessary, but not patching).

The output of the pruner tells you in the end which dependencies were not found on the machine. These will be downloaded and built in the build step, before Sage itself is built. In the moment the pruner should output something like the following, and it is for now recommended to use these three packages from Sage.

libgap will not come from debian!
singular will not come from debian!
jmol will not come from debian!
95 system packages will be used 

The following commands clean everything up properly.

debian/rules clean

Clean up after prune and build.

quilt pop -a

Unapply patches.

cd sage && git clean -dfx

Clean sage folder (git does not show untracked files from the built due to .gitignore).

rm -rf ~/.local/lib/python2.7/site-packages/sage*

Remove sage python library that is installed during the build process.

Finally, there are still dependencies that cannot yet be satisfied with packages from unstable. This changes frequently. For now (branch sage-6.5 in the sagemath git repository) the following packages should be installed from git:

Some Debian packages have to be rebuilt to include patches:

With these changes the compilation of Sage 6.5 should finish and one can concentrate on fixing failing doctests.

For compiling a development version of Sage 6.6 (branch master) one should also update the following Debian packages:

  • gap to version 4.7.7 from experimental

  • cython to version 0.22

Here the compilation currently stops due to an incompatible pari version.

Open tasks

Package

Bug / Link

Assigned

Comments

Packages that need a new maintainer and an upload to Debian:

jmol

There is a request for help, see 719330

libgap

Can maybe be replaced by a wrapper for GAP.

pexpect-2.0

(Debian pkg: python-pexpect): Debian ships version 2.4, but Sage requires version 2.0 (though one of the necessary patches is already in 2.4). Plotting in the notebook is likely to break with newer versions that are in Debian; and >2.0 is reported to be slow. There is a new upstream version so Sage might update, see this sage bug.

singular

PTS

singular 3.1.6 is packaged in debian-science's git and Sage >= 6.2 has also 3.1.6; a package for singular 4.0.1 is in the NEW queue, see 764475.

Sage components that need a new maintainer and an upload to Debian:

sagenb

the dependencies should be in Debian now (691509, 703541, 703531, 703533, 703534, 703535 and 703536)

sagetex

Softwares that need to be packaged:

4ti2

Singular 4 depends on 4ti2; a package is in the NEW queue, see 769997

normaliz

Singular 4 depends on normaliz; an ITP has been filled to package normaliz 2.12.2, see 776019

surf-alggeo

Singular 4 depends on surf; a package is in the NEW queue, see 777132

Packages that are (hopefully) not needed:

cephes

The Cephes Mathematical Library is only needed for systems where the libc doesn't provide appropriate mathematical functions (cygwin and FreeBSD) and so is not required for Debian.

iconv

is part of the libc

mpir

708391

mpir is supposed to be ABI-compatible with gmp. A lot of math packages use gmp and we hope we can have a purely gmp based Sage in Debian. Combining the two will most likely lead to conflicts, using mpir we would need an extra mpir version of many packages.

Other known or suspected issues

  • ECL has a strange bug in debian (see 777473).

  • sage expects to use both ecl and maxima, together, but maxima in debian is using gcl and has no support for ecl ; Snark has fixes for that, but some things (like the above ECL problem) need to clear out before they can be pushed in debian. (see 779804)

  • pari in sage's latest has a patch and uses a header file which aren't in the debian package, so one currently needs to modify a few things by hand to get going (see 777237)

Information bits that are no issues

  • Sage upstream would like to get rid of flintqs since some recent versions of flint ship with a quadratic sieve (see this ticket, but it hasn't been done yet). This mail of William Hart tells more about the state of the quadratic sieve code in both flint and flint2. flintqs was uploaded to Debian for the meantime (but blocked in unstable).

New build system for Sage (the distribution)

this is slightly out of date, also, Sage (the community) is not interested in using packages that another distribution provides.

The Sage library is getting an autotools build system in the course of a Google Summer of Code project. With the new build system, it can be built against system libraries. There are tarballs with the code at http://tool.em.cs.uni-frankfurt.de/~felix/sage/. (There is also a git repository but the tarballs are recommended). Note that this is work in progress and problems are expected, but we are happy about testers. The transition of this work into Sage is coordinated via a bunch of Sage tickets. We have a Debian package sagelib that currently builds metapackages that help installing the build dependencies of Sage. To test building Sage against Debian packages, one should consult the control file. The dependencies of the metapackage sagelib-builddeps-git still need special treatment and can't be just installed from Debian.

The following Debian packages should be installed from the git repositories:

  • sagenb

  • singular (branch master-3-1-5, build with DEB_BUILD_OPTIONS=nodoc)

For the remaining packages where the Debian package is missing or is missing features required by Sage, it is currently most convenient to use the versions from Sage. This is done by configuring with

../configure \
--disable-all \
--enable-{boehm_gc,cython,ecl,flintqs,genus2reduction,maxima,pari,pexpect,pynac,rubiks,sagelib,sagetex,tachyon} \
--with-gmp

New build system for Sage (the library)

There is some work in progress trying to make sagelib work on top of any software distribution. we use autotools to sort out dependencies, run time paths and provide standard build/installation routines. unlike the first approach, this is heading towards a fork and will likely not play well with the current upstream development process. Experimental tarballs are available at http://tool.em.cs.uni-frankfurt.de/~felix/sagelib/, there is no public git repo yet.

Old sagemath package

The old, buggy sagemath Debian package source code is available at http://snapshot.debian.org/package/sagemath/3.0.5dfsg-5.1/