Differences between revisions 125 and 126
Revision 125 as of 2012-12-30 21:51:16
Size: 34184
Comment: added image for initial status page
Revision 126 as of 2012-12-31 02:19:12
Size: 36033
Comment: Rounding up the presentation of the web server
Deletions are marked like this. Additions are marked like this.
Line 264: Line 264:
The web pages show whenever there should be the project name the string "REPLACE WITH PROJECT NAME". To fix this and other things like your public email address, edit html/project/project.inc .

The project start page is not appreciated by everyone. Amend it in './html/user/index.php' but wary of subsequent updates. How exactly this will all be dealt with for updating is not yet clear. For the moment it is just suggested to have a backup when installing the next version of BOINC and then compare the HTML references made from that page. Many are providing a home page to their project that is kept separate from the BOINC-intrinsic workload distribution, i.e. that BOINC-provided index.php page.
The web pages show whenever there should be the project name the string "REPLACE WITH PROJECT NAME". To fix this and other things like your public email address, edit the fiel {{{html/project/project.inc}}} with any text editor, as in {{{
sudo vim html/project/project.inc
}}}
That include file is read by various other PHP files and amonst the above mentioned user-visibible project name and email addresses also allows for extra decorations for headers or footers
.

While a project develops, one is likely to perform many additional changes well beyond initial cosmetics. The project's home/start page is a common initial target. Amend it in './html/user/index.php' but wary of subsequent updates. How exactly this will all be dealt with for updating is not yet clear. For the moment it is just suggested to have a backup when installing the next version of BOINC and then compare the HTML references made from that page. There are several tools known for software development which may come to a rescue, like bazaar, mercurial or git, that may help with the orchestrations of updates that come from new upstream developments and local additions. But that is independent from the Debian packaging side. 
Line 270: Line 273:
The "news" section suggests "No news forum. Run html/ops/create_forums.php". To follow that advice, on the command line Now you have the user-interface of your BOINC server up: the PHP web pages. The subfolder 'html/ops' contains the website management tools that are not called by the user but demand access through that admin password set above. The .PHP files are executed by a little engine within the Apache web server to display the results, in reaction to the respective PATH being selected via the browser. An alternative, and easier to many, is to call the same .PHP files through the command line.

As an example may serve the initialisation of the news. The screen shot shows it, where there should be the latest project news, there is a mere suggestion, reading "No news forum. Run html/ops/create_forums.php". To follow that advice, on the command line
Line 289: Line 294:
to get the default for a start.
to get the default for a start. The 'sed' command (so nice and helpful, but initially awkward) edits the .php script by removing an early exit. I means: search for a string "remove the die" and then deletes that line. If avoiding the sed (stream editor) tool, do so manually with your editor.

The execution of 'create_forums.php' can then be performed on the command line by
Line 294: Line 300:

or the 'ops/create_forums.php' file be found in the browser.

The user interface is - from a technical point of view - now set. It remains the application side to be configured. With Debian we can help again. The tasks are:
 * set up binaries for applications to be run by the volunteers
 * produce work for those applications
 * validate the and archive the results

1. Intro

This page is a tutorial for the setup of a BOINC project on the basis of the boinc-server-maker package. It is currently (12/2012) being updated and that recent version of the boinc-server-maker package is not yet publicly availalble. Also, the tutorial is not consistent in the amount of insights in Linux that is required from the reader. At some point, it should understood by anyone with sincere interest in computing and sufficient time to follow things up.

http://boinc.berkeley.edu/logo/www_logo.gif

2. Preparations

2.1. Environment

2.1.1. Hardware: disk space and internet connection

The hardware you need depends on the problem you address. When distributing many jobs, you need strong upload bandwidth, which means that with the typical asymmetric DSL connects at home this is difficult. Universities obviously will not have such a problem. In clinical or other strongly firewalled setups find an external hoster or a collaborating non-clinical institute. You may also decide to use the Cloud. A regular 32 bit machine is likely to be sufficient, if that machine can concentrate on distributing the job and does not need too much of work for post-processing of results.

2.1.2. Software: BOINC Server Template files

This section first explains the installation of the BOINC server template sources from the source tree and then the same for those using the Debian boinc-server-maker package.

Previously

Now

1.

Install BOINC server dependencies:
sudo apt-get install subversion build-essential apache2 php5 \
     mysql-server php5-gd php5-cli php5-mysql python-mysqldb \
     libtool automake autoconf pkg-config libmysql++-dev libssl-dev

Install boinc-server-maker package from unstable:
sudo apt-get install boinc-server-maker

2.

Create a user and a group for BOINC and add www-data to that group:
sudo addgroup  --system boincadm
sudo adduser www-data boincadm

3.

Download BOINC source:
svn co http://boinc.berkeley.edu\nsvn/branches/server_stable boinc

4.

Compile BOINC:
cd boinc; ./_autosetup; ./configure --disable-client; make

2.2. Project-specific configuration

The preparation of the final project from the BOINC server template is a manual process. That should be applied only once for every project and then for updates of the BOINC server sources. One could (in principle) prepare Debian packages for such project specific configurations. The boinc-server-maker package described on this page would then become a build dependency for the project. See BOINC/Server/Projects for current (early) attempts.

2.2.1. Create and config BOINC project

The BOINC server as compiled from the BOINC sources does not perform directly.

The following needs to be performed for every project, with or without the Debian BOINC server package. In the following we will work on a single BASH shell. This allows us to use mostly self-explanatory variables to help your local adaptation.

To help working consistently on multiple shells, or to get the settings when loggin in again, we need a place to redirect the parameters of the project configuration. We do that only to help ourselves; the BOINC code will not use it. This approach is well accepted, e.g. for .bash_aliases.

2.2.1.1. Initialisation of the configuration

For a first start of a project create the file that the project parameters shall be stored in.

touch ~/.boinc_test.conf
chmod 600 ~/.boinc_test.conf

and whenever starting a shell, these settings should be set, so you may decide to have an account dedicated for your project and have the configration sourced immediately after the shell was started, e.g. for BASH like:

cat >> ~/.bashrc <<EOBASHRC
if [ -r ~/.boinc_test.conf ]; then
   echo "Sourcing ~/.boinc_test.conf"
   . ~/.boinc_test.conf
fi
EOBASHRC

2.2.1.2. Parameters for the MySQL database

cat << EODBCONFIG >> ~/.boinc_test.conf
# password for write access to your project database
pw=MYSQLPASSWORDFORBOINCUSER
# name of the MySQL database
dbprojectname=boinctest
EODBCONFIG

For your convenience you may copy above code and then use an editor to change it.

2.2.1.3. Create MySQL database for BOINC project

With the variables defined before, you can just copy and pasted the code below. The password asked is the password for the root user to access the database, which is not unlikely to differ from the root password to your UNIX system.

# read config if available
[ -r ~/.boinc_test.conf ] && . ~/.boinc_test.conf
if ! echo "DROP USER 'boincadm'@'localhost'" | mysql -u root -p; then
   echo "If the removal of the previous user fails because the user is not existing, then this does not matter. Other errors would be required a manual removal."
fi
if   [ -z "$dbprojectname" ]; then echo "Variable 'dbprojectname' not set";
elif [ -z "$pw" ]; then echo "Variable 'pw' not set";
else
   # piping commands to mysql shell
   cat <<EOMYSQL | mysql -u root -p mysql
DROP DATABASE IF EXISTS $dbprojectname;
CREATE USER 'boincadm'@'localhost' IDENTIFIED BY '$pw';
GRANT ALL PRIVILEGES ON $dbprojectname.* TO 'boincadm'@'localhost';
EOMYSQL
fi

If MySQL is new to you, then this may not be dramatic for a first start. Over time though, and if it is alone for backups, a basic familiarity is required.

2.2.2. Create project

Please replace a.b.c.d with your full hostname or your server's IP address. If you only have dynamic IP addresses, then this is unfortunate, but investigate the service of dyndns.org to help out.

cat <<EOCONF >> ~/.boinc_test.conf
# address of host (via DNS or IP number) at which project server shall be reached
hosturl=http://a.b.c.d
# name of folder in which data shall be stored, also becomes part of project URL
fileprojectname=$dbprojectname
# more human-compatible way to read the project name
niceprojectname="BoincTestProject@Home"
# location to host the user-presented web site
installroot=/var/tmp/boinc-server-test
EOCONF

Again, please edit '~/.boinc_test.conf' with a text editor if required. This example installroot is where you cannot harm anything. Some linux distributions may however have that directory cleaned at boot time. Please adjust for permanent location.

So, to make sure you have indeed sourced the configuration paramters:

if [ ! -r  ~/.boinc_test.conf ]; then
  echo "Configuration file '~/.boinc_test.conf' not existing."
else
  . ~/.boinc_test.conf
  echo -n "Basic configuration test: "
  if [ -z "$installroot" -o -z "$hosturl" -o -z "$dbprojectname" -o -z "$pw" \
    -o -z "$niceprojectname" -o -z "$fileprojectname" ] ; then
     echo "Missing configuration parameter."
  else
     echo "[ok]"
  fi
fi

Previously

Now

Installation according to the books - ''without'' the Debian server package

The installation is expected to happen from within the BOINC source tree.

[ -d "$installroot" ] || sudo mkdir -p "$installroot"
sudo ./tools/make_project \
  --url_base "$hosturl" \
  --db_name "$dbname" \
  --db_user "$dbuser" \
  --db_passwd "$dbpasswd" \
  --drop_db_first  \
  --project_root /var/www/boinc/$projectname \
    "$projectname" "$projectnicename"

Installation ''with'' the Debian server package

If you use boinc-server-maker package you need to use this command instead:

sudo PYTHONPATH=$PYTHONPATH:/usr/share/python-support/boinc-server/Boinc/ \
 /usr/share/boinc-server/tools/make_project \
  --url_base "$hosturl" \
  --db_name "$dbname" \
  --db_user "$dbuser" \
  --db_passwd "$dbpasswd" \
  --drop_db_first  \
  --project_root "$installroot"/"$projectname" \
  --srcdir /usr/share/boinc-server-maker/ \
   "$projectname" "$projectnicename"

source

source

With the original source tree or the Debian package, the project configuration is basically identical. The specification of the 'PYTHONPATH' should not be required. But it is for the moment, leaving some room for optimisation.

The '--test_app' flag is omitted to help the understanding of the setup. Also, it requires the compilation of an example_app which are all distributed separately from the boinc-server-maker package in the boinc-app-examples package, but not in the location that the install scripts expects them.

The screen output should not look too different from

Creating project 'BoincTestProject@Home' (short name 'boinctest'):
   PROJECT_ROOT = /var/tmp/boinc/boinctest/
       URL_BASE = http://boincserver.dyndns.org/
  HTML_USER_URL = http://boincserver.dyndns.org/boinctest/
   HTML_OPS_URL = http://boincserver.dyndns.org/boinctest_ops/
        CGI_URL = http://boincserver.dyndns.org/boinctest_cgi/
        KEY_DIR = /var/tmp/boinc/boinctest/keys/
        DB_NAME = boinctest
        DB_HOST =

Delete /var/tmp/boinc/boinctest/? [y/N]  y
Deleting /var/tmp/boinc/boinctest/

Continue? [Y/n]
Setting up server: creating directories

Keys don't exist in /var/tmp/boinc/boinctest/keys/; generate them? [Y/n]
Setting up server files: generating keys
Setting up server files: copying files
Setting up database
Setting up server files: writing config files
Setting up server files: linking cgi programs
update_translations finished
Done installing default daemons.

Done creating project.  Please view
/var/tmp/boinc/boinctest/boinctest.readme
for important additional instructions.

That 'boinctest.readme' basically says what the next section is presenting. If running into issues of various sorts, you may be suggested to repeat and overwrite ("clobber") the previous install. Caveat: when adding "--delete_prev_inst --drop_db_first", make sure to add them somewhere in the middle, not just comfortably at the end. The last two arguments are interpreted in a special way.

2.2.3. Continue - with or without the Debian BOINC server package

2.2.3.1. Adjusting permissions for project directory

  1. Make sure the configuration variables are set

    [ -r ~/.boinc_test.conf ] && . ~/.boinc_test.conf
  2. Change files and directories permission:

    if [ -z "$installroot" -o -z "$fileprojectname" ]; then
      echo "Not all variables are set for the configuration"
      echo "Error, do not continue."
    elif [ ! -d "$installroot"/"$fileprojectname" ]; then
      echo "The directory '$installroot/'$fileprojectname' is not existing"
      echo "Error, do not continue."
    else
      cd "$installroot"/"$fileprojectname"
      sudo chown root:boincadm  -R .
      sudo chmod g+w -R .
      sudo chmod 02770 -R upload html/cache html/inc html/languages html/languages/compiled html/user_profile
    fi

A successful execution will once ask for the user's sudo password but otherwise remain quiet.

If those permissions are sufficient depends on the user the the web service runs with. For Debian, running apache as user www-data, the include files need to be accessible. Those they should be with www-data added to the group boincadm with those files group accessible and group writable. However, there seem to be some remaining issues. To circumvent them, run

if [ -d html/inc -a -d cgi-bin ]; then
  echo -n "html/inc: "; sudo chmod o+x html/inc && sudo chmod -R o+r html/inc && echo "[ok]" || echo "[failed]"
  echo -n "html/languages: "; sudo chmod o+x html/languages/ html/languages/compiled && echo "[ok]" || echo "[failed]"
else
  echo "You are not in your project directory"
fi

The "-r" is important to reach all the files since the directory cannot be read before the execution of "sudo".

2.2.3.2. Automated restart upon failure

  1. Add project cronjob:

    sudo EDITOR=vim crontab -e

Please check the exact path of the 'start' script. Then add a line like the following in the editor that spawned by the 'crontab -e'

  • command above. You can set the editor to be started by the EDITOR environment variable. Adjust to your liking.

    */5 * * * * /var/tmp/boinc-server-test/boinctest/bin/start --cron

The above indicates a restart (in case of failure) every five minutes. Verify that the cron entry was indeed accepted

sudo crontab -l
  1. Config password for BOINC server administrative page:

    sudo htpasswd -c html/ops/.htpasswd USERNAME
  2. Config Apache to call BOINC server

    sudo cp ${fileprojectname}.httpd.conf  /etc/apache2/sites-available/ \
    && sudo a2ensite  ${fileprojectname}.httpd.conf \
    && sudo /etc/init.d/apache2 reload

2.3. Test web interface

[ATTACH]

The web site should now be accessible. It should be available as an alias "$fileprojectname", please look it up in "/etc/apache2/sites-enabled/${filesprojectname}.httpd.conf". You will find the automated home page to need some further manual adjustments - which does not really encourage to update the server version too frequently, we presume.

If the home page does not show up then there may be some missing file or some bogus file permission. The inspection of "/var/log/apache/error.log" should bring sufficient insights to fix the issue. The image shows a URL with a ":8080", which indicates the browser to ask for the page at a different port than the default ":80". For doing so, add a virtual host configuration in above httpd.conf of your project. The Apache Documentation explains vividly how to do so.

2.3.1. Showing index.php by default

[ATTACH]

With a typical Debian apache setting, the expected BOINC web interface only shows up after clicking on index.php. It may be different for your setup, just if your project homepage looks more like source code then edit /etc/apache2/sites-enabled/${filesprojectname}.httpd.conf as follows to show index.php files directly:

<Directory "...fileprojectname.html">
   Options Indexes FollowSymlinks MultiViews +ExecCGI
   ...not shown...
   # Adding these three lines
   <IfModule mod_dir.c>
       DirectoryIndex index.php
   </IfModule>
</Directory>

Perform /etc/init.d/apache reload after the change. Once PHP is properly configurated, all dynamic web pages, including for accessing the database and message boards will run. You can also get your own account, just the pages all look ugly and unpersonal, still.

2.3.2. Substituting ProjectName in *.html and *.php

The web pages show whenever there should be the project name the string "REPLACE WITH PROJECT NAME". To fix this and other things like your public email address, edit the fiel html/project/project.inc with any text editor, as in

sudo vim html/project/project.inc

That include file is read by various other PHP files and amonst the above mentioned user-visibible project name and email addresses also allows for extra decorations for headers or footers.

While a project develops, one is likely to perform many additional changes well beyond initial cosmetics. The project's home/start page is a common initial target. Amend it in './html/user/index.php' but wary of subsequent updates. How exactly this will all be dealt with for updating is not yet clear. For the moment it is just suggested to have a backup when installing the next version of BOINC and then compare the HTML references made from that page. There are several tools known for software development which may come to a rescue, like bazaar, mercurial or git, that may help with the orchestrations of updates that come from new upstream developments and local additions. But that is independent from the Debian packaging side.

2.3.3. Get up forums up (optional)

Now you have the user-interface of your BOINC server up: the PHP web pages. The subfolder 'html/ops' contains the website management tools that are not called by the user but demand access through that admin password set above. The .PHP files are executed by a little engine within the Apache web server to display the results, in reaction to the respective PATH being selected via the browser. An alternative, and easier to many, is to call the same .PHP files through the command line.

As an example may serve the initialisation of the news. The screen shot shows it, where there should be the latest project news, there is a mere suggestion, reading "No news forum. Run html/ops/create_forums.php". To follow that advice, on the command line

[ -r ~/.boinc_test.php ] && . ~/.boinc_test.php

if [ ! -r create_forums.php ]; then
    if [ -r html/ops/create_forums.php ]; then 
        cd html/ops
    elif [ -r  $installroot/$fileprojectname/html/ops/create_forums.php ]; then
        cd $installroot/$fileprojectname/html/ops
    else
        echo "Could not find create_forums.php"
    fi
fi

Then edit the file to adjust the forums you want to be created or just

sudo sed -i '/remove the die/d' create_forums.php

to get the default for a start. The 'sed' command (so nice and helpful, but initially awkward) edits the .php script by removing an early exit. I means: search for a string "remove the die" and then deletes that line. If avoiding the sed (stream editor) tool, do so manually with your editor.

The execution of 'create_forums.php' can then be performed on the command line by

php5 create_forums.php

or the 'ops/create_forums.php' file be found in the browser.

The user interface is - from a technical point of view - now set. It remains the application side to be configured. With Debian we can help again. The tasks are:

  • set up binaries for applications to be run by the volunteers
  • produce work for those applications
  • validate the and archive the results

3. Add functionality

This section brings your BOINC project to life.

3.1. Setting up BOINC's Example Application ''upper_case''

This section now has its own page at BOINC/ServerGuide/AppDeployment.

3.2. Start BOINC demons

Now its time to start the project, assuming we are still inside the project root,

$./bin/start
Staying in ENABLED mode
Starting daemons
  Starting daemon: feeder -d 3
  Starting daemon: transitioner -d 3
  Starting daemon: file_deleter -d 3

If you receive the message "No daemons for this host found - check host name in config.xml", then do just that and set the X in <hostname>X</hostname> to what the UNIX command hostname returns on your machine.

And ./bin/status can be used to check status of the project.

$ ./bin/status
BOINC is ENABLED

DAEMON  pid  status      lockfile disabled  commandline
   1  16574  running     locked   no        feeder -d 3
   2  16576  running     locked   no        transitioner -d 3
   3  16579  running     locked   no        file_deleter -d 3

TASK last run period    next run lock file disabled commandline
   1    ?     24 hours     NOW   unlocked  yes       db_dump -d 2 -dump_spec ../db_dump_spec.xml
   2    ?     1 days       NOW   unlocked  yes       run_in_ops ./update_uotd.php
   3    ?     1 hour       NOW   unlocked  yes       run_in_ops ./update_forum_activities.php
   4    ?     7 days       NOW   unlocked  yes       update_stats -update_users -update_teams -update_hosts
   5    ?     24 hours     NOW   unlocked  yes       run_in_ops ./update_profile_pages.php
   6    ?     24 hours     NOW   unlocked  yes       run_in_ops ./team_import.php
   7    ?     24 hours     NOW   unlocked  yes       run_in_ops ./notify.php

3.3. Config clients

The project should now be ready to run a couple of work units. Please use any Linux BOINC client. The Debian boinc-client package will do. It needs to be Linux since our application was yet only prepared for this OS. The project URL is $hosturl/$fileprojectname.

3.4. Treatment of results

3.4.1. Inspection of internal representation

Once the user was created, the respective entry in the database will look like

mysql> select id,create_time,email_addr,name,authenticator,country,postal_code,url,donated from user;
 id | create_time | email_addr    | name | authenticator | country | postal_code | url  | donated
----+-------------+---------------+------+---------------+---------+-------------+------+-----
  1 |  1310218977 | your@addr.com |  you | ce5e2e50akle6 | Country | nnn         | NULL |    0 

and when the first result was computed, it will show as

mysql> select id,create_time,workunitid,outcome,hostid,userid,name,cpu_time,app_version_num,appid
         from result;
id | create_time | workunitid |outcome|hostid|userid| name   | cpu_time |app_version_num|appid
---+-------------+------------+-------+------+------+--------+----------+---------------+-----
 1 |  1310217514 |          1 |      1|    1 |    1 | test_0 | 19.94525 |           612 |   11

the workunitid points to the compute challenge/job. When the job is executed multiple times, there will be multiple results for the same work unit

mysql> select id,create_time,appid,name,need_validate from workunit;
 id | create_time | appid | name | need_validate 
----+-------------+-------+------+---------------
  1 |  1310213354 |    11 | test |             1 

The "need_validate" on 1 means that this result is not yet trusted to be correct.

3.4.2. Validation

The validation is nicely described at http://boinc.berkeley.edu/trac/wiki/ValidationIntro . It is refering to a tool that decides if a result can be trusted or not. Typically this is by replication: two independent runs by different individuals should give the same results. The difficulty is not the false positive result, you just recompute. It is the false negatives - a good result falsely uploaded, which is of concern.

Here we aim not at wild internet project but at projects for local groups. We shall do without verification. If this is acceptable will depend on your project. You take the following security measures:

  • a single false negative result will not be considered too dramatic because of some redundancy in the analysis, say by an identification of a cluster of results
  • you invite colleagues, students and their families to contribute, so the scientific value becomes evident to everyone and one feels connected to the research
  • to cheat shall be not too difficult, hence not technically interesting
  • you don't give credit for the results
  • you compare individual results with your own calculations

The boinc-server-maker package provides the example validatators in /usr/lib/boinc-server-maker/bin/sample_binary_validator . We suggest to gain some experience with this one, basically asking it not to validate at all except for completeness of the upload. This reduces the complexity for now and your very first project is likely not to profit too much from your investment at this corner.

For the integration of the validator with the project, you do not call the thing directly. Well you could, actually, just make sure you are running as the right user, otherwise you cannot read the files:

/usr/lib/boinc-server-maker/bin/sample_bitwise_validator -app upper_case --one_pass -d 4 

The workunit needed to be validated then is assigned the credits and so are the users. Since one is not interested in executing such manually too often, one could either create a UNIX-typical cron file, or the daemon facility that BOINC provides. It may look like

    <daemon>
      <cmd>
         sample_bitwise_validator -app upper_case --sleep_interval 600 -d 3
      </cmd>
    </daemon>

Every 10 minutes should be sufficient.

3.4.3. Access on result files

The previously described output template describes what files are expected to be created. Those are shipped back and can then be prepared for subsequent scrutiny. This is explained neatly on http://boinc.berkeley.edu/trac/wiki/AssimilateIntro . For projects that have only results that are successfully validated and that have only one file per workunit, the action to be performed is basically "none". Well, One moves the right file of the many (for complicated projects) into some result area that is then archived. For our purpose, the assimilator could just be "mv", the file mover. But there is also some special communication with the server, so one takes the sample_assimilator, which gets the files from their original location

$ find $installroot/$fileprojectname/upload/
upload/
upload/ba
upload/ba/test_0_0

And look like expected

$ cat upload/ba/test_0_0 
TEST STRING

into a new directory. From the upload directory they will otherwise be removed by the file_deleter daemon. With some deeper insights one can decide to do additional things. To mind come to ring a bell for exceptionally nice results, or to immediately remove a result rather than copying it when it is too bad to be true. The sample_assimilator names the files after their workunit name, basically to bring relief to the programmer to think about a unique namespace. Again, one specifies a daemon section. In complete analogy to the verification:

$ /usr/lib/boinc-server-maker/bin/sample_assimilator --app upper_case --one_pass -d 4
2011-08-10 02:43:07.3138  Starting
query: select * from app where name='upper_case'
query: select * from workunit where appid=11 and assimilate_state=1  limit 1000
2011-08-10 02:43:07.3234 [debug] [test] assimilating WU 1; state=1
query: select * from result where workunitid=1
query: update workunit set assimilate_state=2, transition_time=1312936987 where id=1
query: COMMIT
2011-08-10 02:43:07.3502  Assimilated 1 workunits.

The entry for the config.xml is

    <daemon>
      <cmd>
         sample_assimilator --app upper_case --sleep_interval 610 -d 3     
      </cmd>
    </daemon>

and the completed workunits are then found in

$ find sample_results -type f
sample_results/test

and indeed

$ cat sample_results/test 
TEST STRING

where "test" is indeed the name of the workunit as can be seen in the table above. Please make better workunit names than shown here. Those should have a meaning of some sort. Files in here will not be touched by the system since the workunit will be completed. Anyway - make backups.

3.5. Add your own application

The only thing that counts to have is the application that helps your very personal task at hand. There are two ways to achieve that:

  1. a newly designed or modified app that knows how to speak the BOINC API
  2. a wrapper to execute the working application through the regular UNIX command line

The creation of native BOINC-speaking applications is nothing a distribution can help much with, except for providing the boinc-dev package. Please raise your hand if you are inclined to prepare a similar tutorial for that. We continue with using the wrapper to bring your (or someone else's) legacy applications to BOINC:

3.5.1. Preparations for using the wrapper for legacy (read: regular) applications

Previously

Now

Compiling sources oneself

Have the BOINC source code untarred to ~/boinc, the libraries compiled, and then

cd ~/boinc/samples/wrapper
make

This process needs to be repeated for every Linux architecture you plan to support. This is why typically only the most mainstreamish Linux flavours are supported, i.e. amd64 and i386.

Starting from the Debian package

all done

source

source

3.5.2. Getting the wrapper and binaries into your server's setup

Please follow BOINC/ServerGuide/WrapperApp for detailed instructions on how to get your application wrapped. You then have a complete environment for getting arbitrary compute jobs performed by volunteers.

4. Management of your own project

This section describes how to keep a project running.

4.1. Server side setup

BOINC as a server only runs on Linux at the time speaking. If you are running in a Windows-only environment. The cloud, a hosted machine or local virtualisation are options.

4.1.1. Configuration - summary

Above, a series of environment variables was specified to separate the invariant instructions from those bits that are project specific. Truly, each BOINC project needs its very own set of configuration variables. It is needed as long the project lives to allow for updates and routine maintenance. To avoid expensive typos, project-specific configurations are suggested, like ~/.boinc_projectname.conf.

For our test project, the configuration file is ~/.boinc_test.conf and this will look like,

# password for write access to your project database
pw=MYSQLPASSWORDFORBOINCUSER
# name of the MySQL database
dbprojectname=boinctest
# address of host (via DNS or IP number) at which project server shall be reached
hosturl=http://a.b.c.d
# name of folder in which data shall be stored, also becomes part of project URL
fileprojectname=$dbprojectname
# more human-compatible way to read the project name
niceprojectname="BoincTestProject@Home"
# location at which sources shall be kept
installroot=/var/tmp/boinc

So while maintaining the project all we have to do is

source ~/.boinc_test.conf

and execute the commands using the standard variables.

A one go script for all steps mentioned here is availabe here

4.2. Client side setup: none for Linux, VirtualBox for others

To attract as many as possible clients, those those should not need to perform too much of a configuration. And the Linux users will indeed just install the BOINC client and then run the jobs. But one does not want to prepare binaries for all the Windows and Apple platforms, especially not when those platforms are not available. The idea to circumvent that is a virtual environment. With VirtualBox there is a freely available highly performant technology available to execute code of foreign operating systems under one's own.

4.2.1. How to create the virtual image (optional)

  1. Download VirtualBox for your OS from http://www.virtualbox.org/wiki/Downloads

  2. Download an installer CD .iso (another image file) for Debian testing from http://www.debian.org/releases/stable/debian-installer/. It is your pick, really, but you may prefer the business card CD.

  3. Install VirtualBox and start it up

  4. Press "New" button and follow questions, leave defaults where unsure
    • give it a fancy name, it is a Linux machine of type Debian, not oracle (important for the default Logo later)
    • the amount of memory depends on the application you run. Make it 750 MB.
    • create a 1.5GB empty disk image (space is not taken until used)
  5. Give mass storage a second drive (click on it and then find the green "plus" symbol), which is the Debian installer .iso from above
  6. Start the machine, which will find the .iso as the only bootable decide, which will start the Debian installer, just follow the instructions
  7. Reboot and start from it, only install the base system, do the graphics after the reboot since otherwise the disk won't be large enough to cache the packages
  8. Log in as root
    1. sudo apt-get install lxde as a reasonably sized reasonably functional desktop
    2. sudo apt-get install boinc-client boinc-manager
  9. done. Exit VirtualBox.

It is tedious if it does not go through smoothly ... take a readily prepared one.

4.2.2. The larger Debian community kindly offers a such prepared image for us

  1. Download the image from http://bio4.dnsalias.net/download/BioNode/Boinc-20110808.box and untar it

  2. Download VirtualBox for your OS from http://www.virtualbox.org/wiki/Downloads

  3. Start the image
  4. If starting the first time
    • change root password
    • start BOINC manager (boincmgr) to add project of preference

Later starts of the image will see the application starting directly.

4.2.3. Disclaimer

This section is not yet complete. While the instructions on how to get your project up is basically fine, if extendable, there are a few issues that frankly we don't know how to solve exactly, yet, i.e.:

  • updates of the project pages
    • the web pages are most likely straight forward to bring to a new version
    • the database needs the schema to be adjusted ... how?
  • integration with Debian infrastructure
    • what clients are built through the Debian build daemons and which are not?
    • shall there be static applications only or do we also allow dynamically linking ones?
    • shall there be additional tests be performed by the client to report on issues prior to the distribution of work? Say checking the output of ldd?
  • cross-platform compatibility with MacOS and Windows
    • virtualisation is obvious and
      • may be preferred by many, needs to be prepared, though
      • GPU computing may not be supported that route
    • support for cross-platform wrapper
  • ... what is there that we are not even aware of, yet ...?

5. Example project: Docking with AutoDock

Numerous projects with BOINC can be imagined. We have chosen to address AutoDock as an examples since

  • one of the BOINC maintainers at Debian also maintains the packages for AutoDock and the MGLTools

  • there is prior art, so quite a number of individuals can immediately understand the problem
  • many groups around the globe are likely to profit from these instructions while working on many different biological questions

If your background is not biochemistry you can still understand it all from the technical side. But you will barely come up with your own docking project. If you are interested to help out some drug researchers local to you, just find your nearest university and ask around. Even if you are still going to high school, you may run into someone to mentally feed you or even to offer some real collaboration.

The steps for creating the example project, is described at BOINC/ServerGuide/AutoDockApp.

6. See also


BOINC. BOINC/Server, BOINC/Development, BOINC/Development/GitUsage,