Differences between revisions 31 and 32
Revision 31 as of 2007-02-23 09:04:35
Size: 5232
Comment: added libapt-pkg-perl as required package.
Revision 32 as of 2007-03-03 14:46:07
Size: 6639
Editor: ?PsychedelysGuest
Comment: updated the procedure for building BIOC packages
Deletions are marked like this. Additions are marked like this.
Line 11: Line 11:
You will need about 10Go just for building the BIOC packages.

'''Be carefull : it's better to do everything in a Chroot/Vserver/Xen'''
Line 13: Line 17:
 1. Prepare local Debian installation: {{{
apt-get install libapt-pkg-perl libdbd-sqlite3-perl r-base-core r-base-dev
export PERL5LIB=$PERL5LIB:/usr/share/R/share/perl:.
mkdir
changelogdb
1. Prepare local Debian installation: {{{
aptitude install libapt-pkg-perl libdbd-sqlite3-perl r-base-core r-base-dev debhelper cdbs devscripts
Line 18: Line 20:
 1. Checkout the pkg-bioc's ''tools'' module via CVS, either
2.
Create an empty directory and cd in it.

3
. Checkout the pkg-bioc's ''tools'' module via CVS, either
Line 26: Line 31:
 1. Preparations for the packaging script:
 ** for first run{{{
mkdir bioc && cd bioc && ln -s ../tools/{cran2deb.pl,CRAN2DEB.pm,patches} .

4. Preparations for the packaging script: execute the r_pkg_prepare.sh with the right option. It will prepare
 * the directories : create a source,build directories and a directory for the changelogs to be passed across builds
 * download the packages from CRAN. It retrieves the top-level files as well as the Descriptions dir and nothing else.

 * on US side{{{
cd tools && sh ./r_pkg_prepare.sh --update-bioc --us
}}}
 * on Europe side (currently Germany){{{
cd tools && sh ./r_pkg_prepare.sh --update-bioc --de
Line 30: Line 42:
 1. Download {{{
../bioc_update.sh --help
# Take some time reading it.
sleep 30 && ../bioc_update.sh Yes!

5. go to the freshly created directory bioc
{{{
cd ../bioc
Line 35: Line 47:
 1. Compilation
6
. Compilation
Line 40: Line 53:
== cran2deb Options == ==== cran2deb Options ====
Line 43: Line 56:
Line 50: Line 64:
    --verbose more verbose operation     --verbose more verbose operation 
Line 53: Line 67:
    --dbname specify filename of database to read/store permanent info.
                           The data should be shared between repositories (set to 'database')
Line 55: Line 71:
    --archive path archive mirror
Line 59: Line 74:
    --spotChangesToScript rebuild package if change to this script was detected
                           (set to '0')
Line 60: Line 77:
    --keepPatchedSource do not removed the source directory (set to '')
Line 61: Line 80:
    --skipTestOnInstalledBuildDependencies (set to '0')
                           If set, the build dependencies are not checked for their installation.
    --installRequiredPackages (set to 'echo', can be 'echo' or 'sudo')
                           If test on installed packages is not skipped, then this routine
                           will find ways to install the missing packages. If set to echo, the
                           routine will only declare to execute a command for package
                           installation but not perform such.
Line 62: Line 88:





Debian BioConductor archive

The [http://alioth.debian.org/projects/pkg-bioc/ Alioth pkg-bioc] is preparing a repository for Debian packages for the ?BioConductor collection of R libraries.

The [http://alioth.debian.org/projects/pkg-bioc/ Alioth pkg-bioc] is working on a repository for Debian packages of the R libraries at ?BioConductor (http://www.bioconductor.org). More information is found on the Wiki page to ?AliothPkgBioc.

The general idea is that upstream's (the ?BioConductor developers's) install scripts are not aware of the respective Linux distribution and do now know how to install those libraries external to CRAN that are required for the compilation. Some other bits require manual intervention, i.e. for the integration with web servers.

You will need about 10Go just for building the BIOC packages.

Be carefull : it's better to do everything in a Chroot/Vserver/Xen

The following steps are required to prepare all ?BioConductor packages:

1. Prepare local Debian installation:

aptitude install libapt-pkg-perl libdbd-sqlite3-perl r-base-core r-base-dev debhelper cdbs devscripts

2. Create an empty directory and cd in it.

3. Checkout the pkg-bioc's tools module via CVS, either

  • anonymously without account on alioth.debian.org

    cvs -z3 -d:pserver:anonymous@cvs.alioth.debian.org:/cvsroot/pkg-bioc co tools
  • or (preferred) with one's on alioth.debian.org

    export CVS_RSH=ssh
    cvs -d :ext:developername@cvs.alioth.debian.org:/cvsroot/pkg-bioc checkout tools
    Only this second route allows the contribution to the project.

4. Preparations for the packaging script: execute the r_pkg_prepare.sh with the right option. It will prepare

  • the directories : create a source,build directories and a directory for the changelogs to be passed across builds
  • download the packages from CRAN. It retrieves the top-level files as well as the Descriptions dir and nothing else.
  • on US side

    cd tools && sh ./r_pkg_prepare.sh --update-bioc --us
  • on Europe side (currently Germany)

    cd tools && sh ./r_pkg_prepare.sh --update-bioc --de
    For later runs only just change to that directory.

5. go to the freshly created directory bioc

cd ../bioc

6. Compilation

   ./cran2deb

cran2deb Options

perl cran2deb.pl --help

cran2deb -- Convert R packages from CRAN/BioConductor into Debian packages

Usage:
    cran2deb [options]

Options:
    --help                 show this help
    --verbose              more verbose operation 
    --debug                even more verbose for debugging purposes
    --dryrun               only simulate but don't build
    --dbname               specify filename of database to read/store permanent info.
                           The data should be shared between repositories (set to 'database')
    --binarch arch         binary architecture
    --subset regexp        regular expression to match package name (set to '')
    --repository path      repository
    --changelobdbdir path  changelogdb directory
    --overwrite            build if debian package is found in build directory
    --spotChangesToScript  rebuild package if change to this script was detected
                           (set to '0')
    --ignore               ignore build errors (for running over night)
    --keepPatchedSource    do not removed the source directory (set to '')

    --displayknown         display the entries already in the DB
    --skipTestOnInstalledBuildDependencies (set to '0')
                           If set, the build dependencies are not checked for their installation.
    --installRequiredPackages (set to 'echo', can be 'echo' or 'sudo')
                           If test on installed packages is not skipped, then this routine
                           will find ways to install the missing packages. If set to echo, the
                           routine will only declare to execute a command for package
                           installation but not perform such.

Separate Scripts

findMissingDebs.sh

Run within or above the builds folder. It will give you an output like the following:

Attempted to package:
:-( package affycoretools_1.7.7 not successfully packaged.
:-( package applera_1.5.0 not successfully packaged.
:-( package beadarraysnp_1.1.10 not successfully packaged.
:-( package beadexplorer_1.1.3 not successfully packaged.
:-( package biostrings_2.3.7 not successfully packaged.
:-( package bufferedmatrix_0.1.27 not successfully packaged.
:-( package bufferedmatrixmethods_0.1.1 not successfully packaged.
:-( package cellhts_1.5.8 not successfully packaged.
:-( package isnetwork_1.4.0 not successfully packaged.
:-( package pathrender_1.2.0 not successfully packaged.
:-( package prada_1.11.4 not successfully packaged.
:-( package stam_1.7.0 not successfully packaged.
:-( package tilingarray_1.13.3 not successfully packaged.
:-( package weaver_1.0.1 not successfully packaged.
:-( package widgetinvoke_1.7.0 not successfully packaged.

Implemented only for bioc: The first argument "sources" will inspect the sources folder for those .tar.gz files that where not even attempted to be built.

orig2debSource.sh

Given a set of .log files as an argument, this script practically reverses the deletion of the source directory after a failed build.

Bits not removed from the Wiki (yet)

  • Software

       wget -r --reject css,txt,zip,pdf,html,htm -l 2 -N -nH \
       http://www.bioconductor.org/packages/2.0/bioc/
  • Experiment Data
  • Annotation

       rooturl="http://www.bioconductor.org/packages/2.0/data/annotation/html/"
    
       for p in \
         ${rooturl}["humanCHRLOC"].html \
         ${rooturl}["humanLLMappings"].html \
         ${rooturl}["ratCHRLOC"].html \
         ${rooturl}["ratLLMappings"].html \
         ${rooturl}["mouseCHRLOC"].html \
         ${rooturl}["mouseLLMappings"].html \
         ${rooturl}KEGG.html \
         ${rooturl}KEGGdb.html \
         ${rooturl}["ratCHRLOC"].html \
         ${rooturl}["ratLLMappings"].html \
         ${rooturl}GO.html \
         ${rooturl}GOdb.html \
         ${rooturl}PFAM.html \
     ; \
     do
             wget -r --reject css,txt,pdf,html,htm,zip -l 1 -N -nH $p
     done

Any suggestion on how to improve the download of a selected number of packages from the annotation packages are welcome.

  • For later runs only:

        (cd packages/bioc/devel/src/contrib/Source/ && deldoubles.pl)

The tool indicates packages that are outdated and should be removed.