Differences between revisions 32 and 34 (spanning 2 versions)
Revision 32 as of 2007-03-03 14:46:07
Size: 6639
Editor: ?PsychedelysGuest
Comment: updated the procedure for building BIOC packages
Revision 34 as of 2007-03-03 18:11:40
Size: 6643
Editor: ?PsychedelysGuest
Comment: added cvs as need package
Deletions are marked like this. Additions are marked like this.
Line 18: Line 18:
aptitude install libapt-pkg-perl libdbd-sqlite3-perl r-base-core r-base-dev debhelper cdbs devscripts aptitude install libapt-pkg-perl libdbd-sqlite3-perl r-base-core r-base-dev debhelper cdbs devscripts cvs
Line 40: Line 40:
cd tools && sh ./r_pkg_prepare.sh --update-bioc --de cd tools && sh ./r_pkg_prepare.sh --update-bioc --eu

Debian BioConductor archive

The [http://alioth.debian.org/projects/pkg-bioc/ Alioth pkg-bioc] is preparing a repository for Debian packages for the ?BioConductor collection of R libraries.

The [http://alioth.debian.org/projects/pkg-bioc/ Alioth pkg-bioc] is working on a repository for Debian packages of the R libraries at ?BioConductor (http://www.bioconductor.org). More information is found on the Wiki page to ?AliothPkgBioc.

The general idea is that upstream's (the ?BioConductor developers's) install scripts are not aware of the respective Linux distribution and do now know how to install those libraries external to CRAN that are required for the compilation. Some other bits require manual intervention, i.e. for the integration with web servers.

You will need about 10Go just for building the BIOC packages.

Be carefull : it's better to do everything in a Chroot/Vserver/Xen

The following steps are required to prepare all ?BioConductor packages:

1. Prepare local Debian installation:

aptitude install libapt-pkg-perl libdbd-sqlite3-perl r-base-core r-base-dev debhelper cdbs devscripts cvs

2. Create an empty directory and cd in it.

3. Checkout the pkg-bioc's tools module via CVS, either

  • anonymously without account on alioth.debian.org

    cvs -z3 -d:pserver:anonymous@cvs.alioth.debian.org:/cvsroot/pkg-bioc co tools
  • or (preferred) with one's on alioth.debian.org

    export CVS_RSH=ssh
    cvs -d :ext:developername@cvs.alioth.debian.org:/cvsroot/pkg-bioc checkout tools
    Only this second route allows the contribution to the project.

4. Preparations for the packaging script: execute the r_pkg_prepare.sh with the right option. It will prepare

  • the directories : create a source,build directories and a directory for the changelogs to be passed across builds
  • download the packages from CRAN. It retrieves the top-level files as well as the Descriptions dir and nothing else.
  • on US side

    cd tools && sh ./r_pkg_prepare.sh --update-bioc --us
  • on Europe side (currently Germany)

    cd tools && sh ./r_pkg_prepare.sh --update-bioc --eu
    For later runs only just change to that directory.

5. go to the freshly created directory bioc

cd ../bioc

6. Compilation

   ./cran2deb

cran2deb Options

perl cran2deb.pl --help

cran2deb -- Convert R packages from CRAN/BioConductor into Debian packages

Usage:
    cran2deb [options]

Options:
    --help                 show this help
    --verbose              more verbose operation 
    --debug                even more verbose for debugging purposes
    --dryrun               only simulate but don't build
    --dbname               specify filename of database to read/store permanent info.
                           The data should be shared between repositories (set to 'database')
    --binarch arch         binary architecture
    --subset regexp        regular expression to match package name (set to '')
    --repository path      repository
    --changelobdbdir path  changelogdb directory
    --overwrite            build if debian package is found in build directory
    --spotChangesToScript  rebuild package if change to this script was detected
                           (set to '0')
    --ignore               ignore build errors (for running over night)
    --keepPatchedSource    do not removed the source directory (set to '')

    --displayknown         display the entries already in the DB
    --skipTestOnInstalledBuildDependencies (set to '0')
                           If set, the build dependencies are not checked for their installation.
    --installRequiredPackages (set to 'echo', can be 'echo' or 'sudo')
                           If test on installed packages is not skipped, then this routine
                           will find ways to install the missing packages. If set to echo, the
                           routine will only declare to execute a command for package
                           installation but not perform such.

Separate Scripts

findMissingDebs.sh

Run within or above the builds folder. It will give you an output like the following:

Attempted to package:
:-( package affycoretools_1.7.7 not successfully packaged.
:-( package applera_1.5.0 not successfully packaged.
:-( package beadarraysnp_1.1.10 not successfully packaged.
:-( package beadexplorer_1.1.3 not successfully packaged.
:-( package biostrings_2.3.7 not successfully packaged.
:-( package bufferedmatrix_0.1.27 not successfully packaged.
:-( package bufferedmatrixmethods_0.1.1 not successfully packaged.
:-( package cellhts_1.5.8 not successfully packaged.
:-( package isnetwork_1.4.0 not successfully packaged.
:-( package pathrender_1.2.0 not successfully packaged.
:-( package prada_1.11.4 not successfully packaged.
:-( package stam_1.7.0 not successfully packaged.
:-( package tilingarray_1.13.3 not successfully packaged.
:-( package weaver_1.0.1 not successfully packaged.
:-( package widgetinvoke_1.7.0 not successfully packaged.

Implemented only for bioc: The first argument "sources" will inspect the sources folder for those .tar.gz files that where not even attempted to be built.

orig2debSource.sh

Given a set of .log files as an argument, this script practically reverses the deletion of the source directory after a failed build.

Bits not removed from the Wiki (yet)

  • Software

       wget -r --reject css,txt,zip,pdf,html,htm -l 2 -N -nH \
       http://www.bioconductor.org/packages/2.0/bioc/
  • Experiment Data
  • Annotation

       rooturl="http://www.bioconductor.org/packages/2.0/data/annotation/html/"
    
       for p in \
         ${rooturl}["humanCHRLOC"].html \
         ${rooturl}["humanLLMappings"].html \
         ${rooturl}["ratCHRLOC"].html \
         ${rooturl}["ratLLMappings"].html \
         ${rooturl}["mouseCHRLOC"].html \
         ${rooturl}["mouseLLMappings"].html \
         ${rooturl}KEGG.html \
         ${rooturl}KEGGdb.html \
         ${rooturl}["ratCHRLOC"].html \
         ${rooturl}["ratLLMappings"].html \
         ${rooturl}GO.html \
         ${rooturl}GOdb.html \
         ${rooturl}PFAM.html \
     ; \
     do
             wget -r --reject css,txt,pdf,html,htm,zip -l 1 -N -nH $p
     done

Any suggestion on how to improve the download of a selected number of packages from the annotation packages are welcome.

  • For later runs only:

        (cd packages/bioc/devel/src/contrib/Source/ && deldoubles.pl)

The tool indicates packages that are outdated and should be removed.