Differences between revisions 1 and 26 (spanning 25 versions)
Revision 1 as of 2005-05-18 07:48:34
Size: 188
Editor: anonymous
Comment:
Revision 26 as of 2007-02-11 18:22:54
Size: 5274
Comment: Describing scripts.
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## Auto-converted by kwiki2moinmoin v2005-10-07
Line 3: Line 2:
The {alioth:pkg-bioc} is working on Debian packages
repository for the libraries at BioConductor (link?).
The [http://alioth.debian.org/projects/pkg-bioc/ Alioth pkg-bioc] is preparing a repository for Debian packages for the BioConductor collection of R libraries.

The [http://alioth.debian.org/projects/pkg-bioc/ Alioth pkg-bioc] is working on a
repository for Debian packages
of the R libraries at BioConductor (http://www.bioconductor.org).
More information is found on the Wiki page to AliothPkgBioc.

The general idea is that upstream's (the BioConductor developers's) install scripts are not aware of the respective Linux distribution and do now know how to install those libraries external to CRAN that are required for the compilation. Some other bits require manual intervention, i.e. for the integration with web servers.

The following steps are required to prepare all BioConductor packages:

 1. Prepare local Debian installation: {{{
apt-get install libdbd-sqlite3-perl r-base-core r-base-dev
export PERL5LIB=$PERL5LIB:/usr/share/R/share/perl:.
mkdir changelogdb builds
}}}
 1. Checkout the pkg-bioc's ''tools'' module via CVS, either
 * anonymously without account on alioth.debian.org{{{
cvs -z3 -d:pserver:anonymous@cvs.alioth.debian.org:/cvsroot/pkg-bioc co tools
}}}
 * or (preferred) with one's on alioth.debian.org{{{
export CVS_RSH=ssh
cvs -d :ext:developername@cvs.alioth.debian.org:/cvsroot/pkg-bioc checkout tools
}}} Only this second route allows the contribution to the project.
 1. Preparations for the packaging script:
 ** for first run{{{
mkdir bioc && cd bioc && ln -s ../tools/{cran2deb.pl,CRANDEB.pl,patches) .
mkdir html
}}} For later runs only just change to that directory.
 1. Download {{{
}}} Well. This was long hampering progress since upstream somehow
does not open the directory for accession and hence inspection of the version number of the packages. This would be required for wget to perform the download, much in contrast to CRAN. Also, there are different repositories for data, experiments and the computational libraries. Upstream provides a script to perform the automated installation. The following approach will deny access to hackers' heaven, but, I kind of liked it: {{{
Rversion=`R --version | head -1 |cut -f3 -d\ | cut -f1,2 -d.`
if [ -z "\$Rversion" ]; then
   Rversion=2.4
fi
if [ ! -d sources ]; then mkdir sources; fi
wget http://www.bioconductor.org/installScripts/$Rversion/biocinstall.R
sed -e 's/install.packages/download.packages/' -e 's/biocinstall/biocdownload/g' -e 's/dependencies=dependencies/destdir="sources"/' < biocinstall.R > biocdownload.R
echo 'biocdownload(group="lite")' >> biocdownload.R
R --no-save < biocdownload.R
}}}

 1. Compilation
{{{
   ./cran2deb
}}}

== cran2deb Options ==



== Separate Scripts ==

=== findMissingDebs.sh ===

Run within or above the ''builds'' folder. It will give you an output like the following: {{{
Attempted to package:
:-( package affycoretools_1.7.7 not successfully packaged.
:-( package applera_1.5.0 not successfully packaged.
:-( package beadarraysnp_1.1.10 not successfully packaged.
:-( package beadexplorer_1.1.3 not successfully packaged.
:-( package biostrings_2.3.7 not successfully packaged.
:-( package bufferedmatrix_0.1.27 not successfully packaged.
:-( package bufferedmatrixmethods_0.1.1 not successfully packaged.
:-( package cellhts_1.5.8 not successfully packaged.
:-( package isnetwork_1.4.0 not successfully packaged.
:-( package pathrender_1.2.0 not successfully packaged.
:-( package prada_1.11.4 not successfully packaged.
:-( package stam_1.7.0 not successfully packaged.
:-( package tilingarray_1.13.3 not successfully packaged.
:-( package weaver_1.0.1 not successfully packaged.
:-( package widgetinvoke_1.7.0 not successfully packaged.
}}}

Implemented only for ''bioc'': The first argument "sources" will inspect the sources folder for those .tar.gz files that where not even attempted to be built.

=== orig2debSource.sh ===

Given a set of .log files as an argument, this script practically reverses the deletion of the source directory after a failed build.

= Bits not removed from the Wiki (yet) =
 * Software{{{
   wget -r --reject css,txt,zip,pdf,html,htm -l 2 -N -nH \
   http://www.bioconductor.org/packages/2.0/bioc/
}}}
 * Experiment Data
 * Annotation{{{
   rooturl="http://www.bioconductor.org/packages/2.0/data/annotation/html/"

   for p in \
     ${rooturl}["humanCHRLOC"].html \
     ${rooturl}["humanLLMappings"].html \
     ${rooturl}["ratCHRLOC"].html \
     ${rooturl}["ratLLMappings"].html \
     ${rooturl}["mouseCHRLOC"].html \
     ${rooturl}["mouseLLMappings"].html \
     ${rooturl}KEGG.html \
     ${rooturl}KEGGdb.html \
     ${rooturl}["ratCHRLOC"].html \
     ${rooturl}["ratLLMappings"].html \
     ${rooturl}GO.html \
     ${rooturl}GOdb.html \
     ${rooturl}PFAM.html \
 ; \
 do
         wget -r --reject css,txt,pdf,html,htm,zip -l 1 -N -nH $p
 done
}}}
Any suggestion on how to improve the download of a selected number of packages from the annotation packages are welcome.
 * For later runs only:{{{
    (cd packages/bioc/devel/src/contrib/Source/ && deldoubles.pl)
}}}
The tool indicates packages that are outdated and should be removed.

Debian BioConductor archive

The [http://alioth.debian.org/projects/pkg-bioc/ Alioth pkg-bioc] is preparing a repository for Debian packages for the ?BioConductor collection of R libraries.

The [http://alioth.debian.org/projects/pkg-bioc/ Alioth pkg-bioc] is working on a repository for Debian packages of the R libraries at ?BioConductor (http://www.bioconductor.org). More information is found on the Wiki page to ?AliothPkgBioc.

The general idea is that upstream's (the ?BioConductor developers's) install scripts are not aware of the respective Linux distribution and do now know how to install those libraries external to CRAN that are required for the compilation. Some other bits require manual intervention, i.e. for the integration with web servers.

The following steps are required to prepare all ?BioConductor packages:

  1. Prepare local Debian installation:

    apt-get install libdbd-sqlite3-perl r-base-core r-base-dev
    export PERL5LIB=$PERL5LIB:/usr/share/R/share/perl:.
    mkdir changelogdb builds
  2. Checkout the pkg-bioc's tools module via CVS, either

  3. anonymously without account on alioth.debian.org

    cvs -z3 -d:pserver:anonymous@cvs.alioth.debian.org:/cvsroot/pkg-bioc co tools
  4. or (preferred) with one's on alioth.debian.org

    export CVS_RSH=ssh
    cvs -d :ext:developername@cvs.alioth.debian.org:/cvsroot/pkg-bioc checkout tools
    Only this second route allows the contribution to the project.
  5. Preparations for the packaging script:
  6. * for first run

    mkdir bioc && cd bioc && ln -s ../tools/{cran2deb.pl,CRANDEB.pl,patches) .
    mkdir html
    For later runs only just change to that directory.
  7. Download

     Well. This was long hampering progress since upstream somehow

does not open the directory for accession and hence inspection of the version number of the packages. This would be required for wget to perform the download, much in contrast to CRAN. Also, there are different repositories for data, experiments and the computational libraries. Upstream provides a script to perform the automated installation. The following approach will deny access to hackers' heaven, but, I kind of liked it:

Rversion=`R --version | head -1 |cut -f3 -d\ | cut -f1,2 -d.`
if [ -z "\$Rversion" ]; then
   Rversion=2.4
fi
if [ ! -d sources ]; then mkdir sources; fi
wget http://www.bioconductor.org/installScripts/$Rversion/biocinstall.R
sed -e 's/install.packages/download.packages/' -e 's/biocinstall/biocdownload/g' -e 's/dependencies=dependencies/destdir="sources"/' < biocinstall.R > biocdownload.R
echo 'biocdownload(group="lite")' >> biocdownload.R
R --no-save < biocdownload.R
  1. Compilation

   ./cran2deb

cran2deb Options

Separate Scripts

findMissingDebs.sh

Run within or above the builds folder. It will give you an output like the following:

Attempted to package:
:-( package affycoretools_1.7.7 not successfully packaged.
:-( package applera_1.5.0 not successfully packaged.
:-( package beadarraysnp_1.1.10 not successfully packaged.
:-( package beadexplorer_1.1.3 not successfully packaged.
:-( package biostrings_2.3.7 not successfully packaged.
:-( package bufferedmatrix_0.1.27 not successfully packaged.
:-( package bufferedmatrixmethods_0.1.1 not successfully packaged.
:-( package cellhts_1.5.8 not successfully packaged.
:-( package isnetwork_1.4.0 not successfully packaged.
:-( package pathrender_1.2.0 not successfully packaged.
:-( package prada_1.11.4 not successfully packaged.
:-( package stam_1.7.0 not successfully packaged.
:-( package tilingarray_1.13.3 not successfully packaged.
:-( package weaver_1.0.1 not successfully packaged.
:-( package widgetinvoke_1.7.0 not successfully packaged.

Implemented only for bioc: The first argument "sources" will inspect the sources folder for those .tar.gz files that where not even attempted to be built.

orig2debSource.sh

Given a set of .log files as an argument, this script practically reverses the deletion of the source directory after a failed build.

Bits not removed from the Wiki (yet)

  • Software

       wget -r --reject css,txt,zip,pdf,html,htm -l 2 -N -nH \
       http://www.bioconductor.org/packages/2.0/bioc/
  • Experiment Data
  • Annotation

       rooturl="http://www.bioconductor.org/packages/2.0/data/annotation/html/"
    
       for p in \
         ${rooturl}["humanCHRLOC"].html \
         ${rooturl}["humanLLMappings"].html \
         ${rooturl}["ratCHRLOC"].html \
         ${rooturl}["ratLLMappings"].html \
         ${rooturl}["mouseCHRLOC"].html \
         ${rooturl}["mouseLLMappings"].html \
         ${rooturl}KEGG.html \
         ${rooturl}KEGGdb.html \
         ${rooturl}["ratCHRLOC"].html \
         ${rooturl}["ratLLMappings"].html \
         ${rooturl}GO.html \
         ${rooturl}GOdb.html \
         ${rooturl}PFAM.html \
     ; \
     do
             wget -r --reject css,txt,pdf,html,htm,zip -l 1 -N -nH $p
     done

Any suggestion on how to improve the download of a selected number of packages from the annotation packages are welcome.

  • For later runs only:

        (cd packages/bioc/devel/src/contrib/Source/ && deldoubles.pl)

The tool indicates packages that are outdated and should be removed.