|
Size: 8643
Comment: normaliz is being packaged by Jerome Benoit <calculus@rezozer.net>
|
Size: 9325
Comment: Delete info about our mini-dinstall repo.
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| Some people within [[DebianScience]] would like to package again Sage for Debian. An old version of Sage got packaged long time ago by Tim Abbott, but, because of lack of maintainance, it became totally unusable and was removed from Debian. The status pages ([[http://people.debian.org/~thansen/debian-sage-status.html|master branch]], [[http://people.debian.org/~thansen/debian-sage-dev-status.html|develop branch]]) give an overview about package versions in Sage and Debian. If you want to help out, pick one of the open tasks or issues below and get going. Or contact us via the [[http://lists.debian.org/debian-science/|mailing list]] or in #debian-science on irc.oftc.net. | Some people within [[DebianScience]] have again packaged [[https://packages.debian.org/source/sid/sagemath|Sage for Debian]]. This is one of the most complex packages in Debian and will need continual maintenance - we are always looking for new contributors! If you want to help out, read below to learn how to build the package from source, then pick one of the [[#Open_tasks|open tasks]] and get going. You can contact us via the [[https://lists.alioth.debian.org/mailman/listinfo/debian-science-sagemath|mailing list]] or in #debian-science on irc.oftc.net. |
| Line 3: | Line 3: |
| = Steps to get an up-to-date Sage standard suite in Debian = | <<TableOfContents()>> |
| Line 5: | Line 5: |
| == New build system for Sage (the distribution) == | = Build from source = |
| Line 7: | Line 7: |
| this is slightly out of date, also, Sage (the community) is not interested in using packages that another distribution provides. | Our packages in the Debian official archives should be buildable using the [[https://www.debian.org/doc/manuals/maint-guide/build.en.html|normal Debian build processes]]. This is for users that wish to build from source, as well as build bots. |
| Line 9: | Line 9: |
| The Sage library is getting an autotools build system in the course of a [[SummerOfCode2013/Projects#Get_Sage_ready_for_Debian|Google Summer of Code project]]. With the new build system, it can be built against system libraries. There are tarballs with the code at [[http://tool.em.cs.uni-frankfurt.de/~felix/sage/]]. (There is also a [[https://github.com/felix-salfelder/sage/tree/WIP512|git repository]] but the tarballs are recommended). Note that this is work in progress and problems are expected, but we are happy about testers. The transition of this work into Sage is coordinated via a bunch of [[http://trac.sagemath.org/query?reporter=felixs|Sage tickets]]. We have a Debian package [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagelib.git;a=summary|sagelib]] that currently builds metapackages that help installing the build dependencies of Sage. To test building Sage against Debian packages, one should consult the [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagelib.git;a=blob;f=debian/control|control file]]. The dependencies of the metapackage sagelib-builddeps-git still need special treatment and can't be just installed from Debian. |
For maintainers, our development workflow is to build from [[https://salsa.debian.org/science-team/sagemath|our git repository]]. Important elements are the build dependencies of Sage, patches for Sage and a script called "pruner" that uses autoconf to determine if the necessary packages are installed. |
| Line 12: | Line 11: |
| The following Debian packages should be installed from the git repositories: * [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagenb.git;a=summary|sagenb]] * [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/singular.git;a=summary|singular]] (branch master-3-1-5, build with DEB_BUILD_OPTIONS=nodoc) |
||{{{git clone --recursive https://salsa.debian.org/science-team/sagemath.git && cd sagemath}}}|| Recursive git clone to also get sage which is included as a git submodule.|| |
| Line 16: | Line 13: |
| For the remaining packages where the Debian package is missing or is missing features required by Sage, it is currently most convenient to use the versions from Sage. This is done by configuring with | To work with this repository, you must start with '''Debian unstable''' (sid) with [[DebianExperimental|experimental]] sources enabled. It is nice - optional, but highly recommended - to do this in an schroot, which you can set up using the instructions in [[sbuild]], after which you'll probably need to apply the following workarounds: <<FootNote(If your build fails with "Permission denied", try adding the following line to `/etc/schroot/sbuild/fstab`: __`tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0`__ - more details [[https://falstaff.agner.ch/2013/10/29/setting-up-schroot-for-python-multiprocessing/|here]])>>. <<FootNote(If your build fails with "out of pty devices", log into the chroot using __`sudo sbuild-shell source:${chroot_name}`__ then run __`rm /dev/ptmx && mknod -m 666 /dev/ptmx c 5 2`__ - more details in DebianBug:817236.)>>. Next, see `debian/README.source` section "Local system patches", for any workarounds you might have to apply. Then: |
| Line 18: | Line 15: |
| {{{ ../configure \ --disable-all \ --enable-{boehm_gc,cython,ecl,flintqs,genus2reduction,maxima,pari,pexpect,pynac,rubiks,sagelib,sagetex,tachyon} \ --with-gmp }}} |
||{{{debian/rules build-dep-maint}}}|| Install packages that are required before installing the build dependencies. || |
| Line 25: | Line 17: |
| == New build system for Sage (the library) == | To build the package, run: |
| Line 27: | Line 19: |
| There is some work in progress trying to make sagelib work on top of any software distribution. we use autotools to sort out dependencies, run time paths and provide standard build/installation routines. unlike the first approach, this is heading towards a fork and will likely not play well with the current upstream development process. Experimental tarballs are available at [[http://tool.em.cs.uni-frankfurt.de/~felix/sagelib/]], there is no public git repo yet. | ||{{{debian/rules build-dep}}}|| Install build dependencies; you may need to run `apt-get -f install` afterwards. If this fails, install `aptitude` then try it again. || ||{{{debian/rules reset}}}|| Apply patches and force-reset the upstream sage submodule. || ||{{{debian/rules prune}}} || Run the pruner script. Check that the output matches the one below. || ||{{{debian/rules build}}} || Build sage and run the test suite. || ||{{{fakeroot debian/rules binary}}} || Build the Debian packages. || |
| Line 29: | Line 25: |
| == Open tasks == | The output of the pruner tells you in the end which dependencies were not found on the machine ("__`XXX will not come from Debian`__"). Since all dependencies should come from Debian, such a line normally indicates that something went wrong. The pruner should only output __`XXX system packages will be used`__. Otherwise, if you are sure that you want to try this, you could go on nevertheless. Sage will then try to download and build the missing dependencies in the build step, before Sage itself is built. See `debian/README.source` for further details, such as how to debug the failing doctests. At present, we have a threshold of 40 failing doctests (out of ~300K total), below which we call it a "[[https://buildd.debian.org/status/package.php?p=sagemath|successful build]]". On amd64 we have ~12 failures, and on i386 we have (??? not tested recently). It would be good to lower these numbers: upstream aim for 0 failing tests and Gentoo typically has between 10-30. <<FootNote(It's perhaps "not worth it" for us to lower it to 0, since the final 20 or so are not "hard" errors but are uninteresting differences like different but equivalent symbolic mathematical expressions or slightly different random number algorithms; these are generally caused by different versions of dependencies in Debian. "Fixing" them would simply involve patching Debian's Sage to expect different values. This is good for documentation purposes but you may also decide that your time is better spent doing other, more important, tasks.)>> Once you are happy with the full build process, you can try a test release build: ||{{{debian/rules release-deb-sci-sage}}} || Build an UNRELEASED version using [[sbuild]], with our custom APT repo. || = Open tasks = The status pages ([[http://people.debian.org/~thansen/debian-sage-status.html|master]], [[http://people.debian.org/~thansen/debian-sage-dev-status.html|develop]]) give an overview about package versions in Sage and Debian. Current high-priority tasks are: 1. Deal with RC bugs in sagemath and its dependencies. (See the link "Dashboard" at the bottom of the status pages.) 1. Get the number of failed doctests down (see [[http://people.debian.org/~thansen/sage-test-status.html|unstable]], [[http://people.debian.org/~thansen/sage-test-status-experimental.html|experimental]]). 1. Upstream patches that are suitable for upstream (see [[https://salsa.debian.org/science-team/sagemath/blob/experimental/debian/patches/series|series]]). 1. Package new sagemath versions and dependencies in experimental, upload everything to unstable together. See `debian/TODO` for more tasks and more details. |
| Line 32: | Line 47: |
| ||<tablestyle="border:1pt solid"> '''Package''' ||<style="width: 70pt"> '''Bug / Link''' || '''Assigned''' ||'''Comments''' || ||<-4 style="border-top:1pt solid; border-bottom:1pt solid"> '''Packages that need a new maintainer and an upload to Debian:''' || || jmol || || || There is a request for help, see DebianBug:719330 || || pexpect 2.0 || || || (Debian pkg: python-pexpect): Debian ships version 2.4, but Sage requires version 2.0 (though one of the necessary patches is already in 2.4). Plotting in the notebook is likely to break with newer versions that are in Debian; and >2.0 is reported to be slow. There is a new upstream version so Sage might update, see [[http://trac.sagemath.org/sage_trac/ticket/10295|this sage bug]]. || || [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/singular.git;a=summary|singular]] || [[https://packages.qa.debian.org/s/singular.html|PTS]] || || singular 3.1.6 is packaged in debian-science's git and Sage >= 6.2 has also 3.1.6; an ITP has been filled to package singular 4.0.1, see DebianBug:764475. || ||<-4 style="border-top:1pt solid; border-bottom:1pt solid"> '''Sage components that need a new maintainer and an upload to Debian:''' || || rubiks || || || a toy spkg -- low priority || || [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagenb.git;a=summary|sagenb]] || || || the dependencies should be in Debian now (DebianBug:691509, DebianBug:703541, DebianBug:703531, DebianBug:703533, DebianBug:703534, DebianBug:703535 and DebianBug:703536) || || sagetex || || || || ||<-4 style="border-top:1pt solid; border-bottom:1pt solid"> '''Softwares that need to be packaged:''' || || 4ti2 || || || Singular 4 depends on [[http://www.4ti2.de/|4ti2]]; an ITP has been filled to package 4ti2 1.6.2, see DebianBug:769997; || || normaliz || || || Singular 4 depends on [[http://www.home.uni-osnabrueck.de/wbruns/normaliz/|normaliz|]]; an ITP has been filled to package normaliz 2.12.2, see DebianBug:776019; || ||<-4 style="border-top:1pt solid; border-bottom:1pt solid"> '''Packages that are (hopefully) not needed:''' || || cephes || || || The Cephes Mathematical Library is only needed for systems where the libc doesn't provide appropriate mathematical functions (cygwin and FreeBSD) and so is not required for Debian.|| || iconv || || || is part of the libc || || libgap || || || Will most likely be replaced by a wrapper for GAP. || || [[http://anonscm.debian.org/gitweb/?p=debian-science/packages/mpir.git;a=summary|mpir]] || DebianBug:708391 || || mpir is supposed to be ABI-compatible with [[http://packages.qa.debian.org/g/gmp.html|gmp]]. A lot of math packages use gmp and we hope we can have a purely gmp based Sage in Debian. Combining the two will most likely lead to conflicts, using mpir we would need an extra mpir version of many packages. || |
= Updating Sage in Debian = Every version of Sage is tested upstream with a specific version of each dependency (see our status pages). It is not always possible to provide the same versions of all dependencies in Debian. It is however advisable to follow the versions as closely as possible, especially for specialized math packages, which tend to have less stable interfaces than general purpose libraries. This means that each update of the package '''sagemath''' comes together with a bunch of updates of other packages. As some of these updates could break the previous sagemath version in unstable, many of these updates should first be uploaded to Debian experimental. The new version of Sage will then be uploaded to experimental as well. Once the new Sage version has been tested in experimental and is considered ready for unstable, the batch of package updates for this version is migrated from Debian experimental to unstable. This should be coordinated and not take too long, to avoid having a broken '''sagemath''' package in unstable for a long time. == Testing migration == In order to have a working '''sagemath''' package in Debian testing at all times, the whole batch of package updates for a new Sage version needs to migrate to testing together. To this end, the dependencies should declare {{{ Breaks: sagemath (<< x.y~) }}} (where x.y is the new Sage version) for at least one binary package on which the binary package '''sagemath''' depends. An exception to this are libraries with a soname jump (if sagemath depends only on the library package). In this case, the new version of the library may migrate to testing before sagemath, because the old library package will be kept around until it is not used anymore. |
| Line 51: | Line 66: |
| = Links = | |
| Line 52: | Line 68: |
== Other known or suspected issues == * ecl in sage has several modifications to upstream, which the debian package doesn't have. [[http://trac.sagemath.org/sage_trac/ticket/14636|This sage bug]] has some details, but here is a quick summary: (1) some sage patches make sense for upstream => Snark asked the debian maintainer to include (2) sage disables SIGCHLD in ecl by patching it => Snark provided a fix (3) sage has unicode issues => sage has a potential fix, Snark is pushing for its inclusion (4) sage compiles ecl without threading support => Snark will have a look when the rest will go forward * sage expects to use both ecl and maxima, together, but maxima in debian is using gcl and has no support for ecl ; Snark has fixes for that, but can't be sure they're complete until the ecl situation has cleared * cddlib in sage is a fork so the current debian package won't replace it even if the versions match (see [[http://trac.sagemath.org/sage_trac/ticket/13026|this sage bug]]) * sagenb (not a dependency, however recommendable) imports OldSecureCookieSessionInterface from flask_oldsessions, which has been deprecated for ages and is not provided by the flask package. affected: sagenb<=0.11.1 == Information bits that are no issues == * Sage upstream would like to get rid of flintqs since some recent versions of flint ship with a quadratic sieve (see [[http://trac.sagemath.org/sage_trac/ticket/11792|this ticket]], but it hasn't been done yet). This [[https://groups.google.com/d/msg/flint-devel/lmRdEPV6el8/JNbZ4eTy_n0J|mail of William Hart]] tells more about the state of the quadratic sieve code in both flint and flint2. flintqs was uploaded to Debian for the meantime. == Old sagemath package == The old, buggy sagemath Debian package source code is available via git ([[http://anonscm.debian.org/gitweb/?p=debian-science/packages/sagemath.git;a=summary|sagemath]]) and at http://snapshot.debian.org/package/sagemath/3.0.5dfsg-5.1/ == Links == |
|
| Line 84: | Line 70: |
| * [[http://wiki.debian.org/JulienPuydt|Graph of dependencies between Sage packages]] * [[http://fedoraproject.org/wiki/SIGs/SciTech/SAGE|Sage in Fedora]] |
* [[http://people.debian.org/~thansen/debian-sage-7.4-status.html|Sage 7.4 in Debian status page]] * [[http://people.debian.org/~thansen/sage-test-status.html|Overview over failed doctests in Debian unstable]] * [[http://people.debian.org/~thansen/sage-test-status-experimental.html|Overview over failed doctests in Debian experimental]] * [[https://salsa.debian.org/science-team/sagemath|Our sagemath git repo]] * [[https://lists.alioth.debian.org/mailman/listinfo/debian-science-sagemath|Our mailing list]] * [[https://tracker.debian.org/teams/debian-sagemath/|Our team on tracker.debian.org]] Sage out of Debian: * [[http://fedoraproject.org/wiki/SIGs/SciTech/SAGE|Sage in Fedora]] ([[https://apps.fedoraproject.org/packages/sagemath/sources|patches]]) * [[https://wiki.archlinux.org/index.php/SageMath|Sage in Arch Linux]] ([[https://git.archlinux.org/svntogit/community.git/tree/trunk?h=packages/sagemath|patches]]) * [[https://github.com/cschwan/sage-on-gentoo/tree/master/sci-mathematics/sage/files|Sage-on-Gentoo patches]] * [[https://trac.sagemath.org/query?summary=~upgrade&desc=1&order=status|List of planned package upgrades in the SageMath Trac]] Historical information: * [[https://lists.debian.org/debian-science/2015/02/msg00024.html|E-mail]] announcing the pruner script. * [[http://wiki.debian.org/JulienPuydt|Graph of dependencies between Sage 5.2 packages]] * Sage [[http://snapshot.debian.org/package/sagemath/|3.0.5]] got packaged long time ago by Tim Abbott, but, because of lack of maintainance, it became totally unusable and was [[https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=573538|removed]] from Debian. |
Some people within DebianScience have again packaged Sage for Debian. This is one of the most complex packages in Debian and will need continual maintenance - we are always looking for new contributors! If you want to help out, read below to learn how to build the package from source, then pick one of the open tasks and get going. You can contact us via the mailing list or in #debian-science on irc.oftc.net.
Build from source
Our packages in the Debian official archives should be buildable using the normal Debian build processes. This is for users that wish to build from source, as well as build bots.
For maintainers, our development workflow is to build from our git repository. Important elements are the build dependencies of Sage, patches for Sage and a script called "pruner" that uses autoconf to determine if the necessary packages are installed.
git clone --recursive https://salsa.debian.org/science-team/sagemath.git && cd sagemath |
Recursive git clone to also get sage which is included as a git submodule. |
To work with this repository, you must start with Debian unstable (sid) with experimental sources enabled. It is nice - optional, but highly recommended - to do this in an schroot, which you can set up using the instructions in sbuild, after which you'll probably need to apply the following workarounds: 1. 2. Next, see debian/README.source section "Local system patches", for any workarounds you might have to apply. Then:
debian/rules build-dep-maint |
Install packages that are required before installing the build dependencies. |
To build the package, run:
debian/rules build-dep |
Install build dependencies; you may need to run apt-get -f install afterwards. If this fails, install aptitude then try it again. |
debian/rules reset |
Apply patches and force-reset the upstream sage submodule. |
debian/rules prune |
Run the pruner script. Check that the output matches the one below. |
debian/rules build |
Build sage and run the test suite. |
fakeroot debian/rules binary |
Build the Debian packages. |
The output of the pruner tells you in the end which dependencies were not found on the machine ("XXX will not come from Debian"). Since all dependencies should come from Debian, such a line normally indicates that something went wrong. The pruner should only output XXX system packages will be used. Otherwise, if you are sure that you want to try this, you could go on nevertheless. Sage will then try to download and build the missing dependencies in the build step, before Sage itself is built.
See debian/README.source for further details, such as how to debug the failing doctests. At present, we have a threshold of 40 failing doctests (out of ~300K total), below which we call it a "successful build". On amd64 we have ~12 failures, and on i386 we have (??? not tested recently). It would be good to lower these numbers: upstream aim for 0 failing tests and Gentoo typically has between 10-30. 3
Once you are happy with the full build process, you can try a test release build:
debian/rules release-deb-sci-sage |
Build an UNRELEASED version using sbuild, with our custom APT repo. |
Open tasks
The status pages (master, develop) give an overview about package versions in Sage and Debian. Current high-priority tasks are:
- Deal with RC bugs in sagemath and its dependencies. (See the link "Dashboard" at the bottom of the status pages.)
Get the number of failed doctests down (see unstable, experimental).
Upstream patches that are suitable for upstream (see series).
- Package new sagemath versions and dependencies in experimental, upload everything to unstable together.
See debian/TODO for more tasks and more details.
Updating Sage in Debian
Every version of Sage is tested upstream with a specific version of each dependency (see our status pages). It is not always possible to provide the same versions of all dependencies in Debian. It is however advisable to follow the versions as closely as possible, especially for specialized math packages, which tend to have less stable interfaces than general purpose libraries.
This means that each update of the package sagemath comes together with a bunch of updates of other packages. As some of these updates could break the previous sagemath version in unstable, many of these updates should first be uploaded to Debian experimental. The new version of Sage will then be uploaded to experimental as well.
Once the new Sage version has been tested in experimental and is considered ready for unstable, the batch of package updates for this version is migrated from Debian experimental to unstable. This should be coordinated and not take too long, to avoid having a broken sagemath package in unstable for a long time.
Testing migration
In order to have a working sagemath package in Debian testing at all times, the whole batch of package updates for a new Sage version needs to migrate to testing together. To this end, the dependencies should declare
Breaks: sagemath (<< x.y~)
(where x.y is the new Sage version) for at least one binary package on which the binary package sagemath depends. An exception to this are libraries with a soname jump (if sagemath depends only on the library package). In this case, the new version of the library may migrate to testing before sagemath, because the old library package will be kept around until it is not used anymore.
Links
Sage out of Debian:
Historical information:
E-mail announcing the pruner script.
Sage 3.0.5 got packaged long time ago by Tim Abbott, but, because of lack of maintainance, it became totally unusable and was removed from Debian.
If your build fails with "Permission denied", try adding the following line to /etc/schroot/sbuild/fstab: tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 - more details here (1)
If your build fails with "out of pty devices", log into the chroot using sudo sbuild-shell source:${chroot_name} then run rm /dev/ptmx && mknod -m 666 /dev/ptmx c 5 2 - more details in 817236. (2)
It's perhaps "not worth it" for us to lower it to 0, since the final 20 or so are not "hard" errors but are uninteresting differences like different but equivalent symbolic mathematical expressions or slightly different random number algorithms; these are generally caused by different versions of dependencies in Debian. "Fixing" them would simply involve patching Debian's Sage to expect different values. This is good for documentation purposes but you may also decide that your time is better spent doing other, more important, tasks. (3)