THESE PAGES ARE STILL UNDER DEVELOPMENT

This tutorial will help you create BOINC project with Debian server without using virtual images provide from BOINC. See BOINC/Server for an overview on BOINC-Server with Debian related issues. For an overview and guide on how to compile the BOINC package from the pkg-boinc git repository, see BOINC/Development/GitUsage.

To see more info about how to install and config BOINC server, please look at BOINC project wiki and the "See also" section on this page.

1. Preparations

The BOINC server as compiled from the BOINC sources does not perform directly. That BOINC server is "only" the skeleton for the real server. With one BOINC server installation as coined by the BOINC developers, one can create many BOINC project servers as seen by regular users of the boinc client who then subscribe to one or multiple of such projects. For that reason, the Debian package is named no longer "boinc-server" but now presents itself as "boinc-server-maker".

In the following, please distinguish

  1. the folder to which the files of the BOINC server templates install ('/usr/share/boinc-server')
  2. the name of the Debian package providing those files ('boinc-server-maker')
  3. the location of your very own project ('you/name/it')

The preparation from the Debian sources is not too different from a preparation from the original source tree. Debian should just shorten the process to a first success.

1.1. Install BOINC Server Template files

This section first explains the installation of the BOINC server template sources from the source tree and then the same for those using the Debian boinc-server-maker package.

Previously

Now

1.

Install BOINC server dependencies:
sudo apt-get install subversion build-essential apache2 php5 \
     mysql-server php5-gd php5-cli php5-mysql python-mysqldb \
     libtool automake autoconf pkg-config libmysql++-dev libssl-dev

Install boinc-server-maker package from unstable:
sudo apt-get install boinc-server-maker

2.

Create a user and a group for BOINC and add www-data to that group:
sudo addgroup  --system boincadm
sudo adduser www-data boincadm

3.

Download BOINC source:
svn co http://boinc.berkeley.edu\nsvn/branches/server_stable boinc

4.

Compile BOINC:
cd boinc; ./_autosetup; ./configure --disable-client; make

1.2. Project-specific configuration

The preparation of the final project from the BOINC server template is a manual process. That should be applied only once for every project and then for updates of the BOINC server sources. One could (in principle) prepare Debian packages for such project specific configurations. The boinc-server-maker package described on this page would then become a build dependency for the project. This is likely to happen but has not yet been addressed.

1.2.1. Create and config BOINC project

The BOINC server as compiled from the BOINC sources does not perform directly.

The following needs to be performed for every project, with or without the Debian BOINC server package. In the following we will work on a single BASH shell. This allows us to use mostly self-explanatory variables to help your local adaptation.

To help working consistently on multiple shells, or to get the settings when loggin in again, we need a place to redirect the parameters of the project configuration. We do that only to help ourselves; the BOINC code will not use it. This approach is well accepted, e.g. for .bash_aliases.

1.2.1.1. Initialisation of the configuration

For a first start of a project create the file that the project parameters shall be stored in.

touch ~/.boinc_test.conf
chmod 600 ~/.boinc_test.conf

and whenever starting a shell, these settings should be set:

cat <<EOBASHRC
if [ -r .boinc_test.conf ]; then
   . .boinc_test.conf
fi
EOBASHRC

1.2.1.2. Parameters for the MySQL database

cat << EODBCONFIG >> ~/.boinc_test.conf
# password for write access to your project database
pw=MYSQLPASSWORDFORBOINCUSER
# name of the MySQL database
dbprojectname=boinctest
EODBCONFIG

For your convenience you may copy above code and then use an editor to change it.

1.2.1.3. Create MySQL database for BOINC project

With the variables defined before, you can just copy and pasted the code below. The password asked is the password for the root user to access the database.

# read config if available
[ -r ~/.boinc_test.conf ] && . ~/.boinc_test.conf
if ! echo "DROP USER 'boincadm'@'localhost'" | mysql -u root -p; then
   echo "If the removal of the previous user fails because the user is not existing, then this does not matter. Other errors would be required a manual removal."
fi
if   [ -z "$dbprojectname" ]; then echo "Variable 'dbprojectname' not set";
elif [ -z "$pw" ]; then echo "Variable 'pw' not set";
else
   # piping commands to mysql shell
   cat <<EOMYSQL | mysql -u root -p mysql
DROP DATABASE IF EXISTS $dbprojectname;
CREATE USER 'boincadm'@'localhost' IDENTIFIED BY '$pw';
GRANT ALL PRIVILEGES ON $dbprojectname.* TO 'boincadm'@'localhost';
EOMYSQL
fi

1.2.2. Create project

Please replace a.b.c.d with your full hostname or your server's IP address. If you only have dynamic IP addresses, then this is unfortunate, but investigate the service of dyndns.org to help out.

cat <<EOCONF >> ~/.boinc_test.conf
# address of host (via DNS or IP number) at which project server shall be reached
hosturl=http://a.b.c.d
# name of folder in which data shall be stored, also becomes part of project URL
fileprojectname=$dbprojectname
# more human-compatible way to read the project name
niceprojectname="BoincTestProject@Home"
# location at which sources shall be kept
installroot=/var/tmp/boinc
EOCONF

Again, please edit '~/.boinc_test.conf' with a text editor if required. This example installroot is where you cannot harm anything. Some linux distributions may however have that directory cleaned at boot time. Please adjust.

Previously

Now

1.2.2.1. Installation according to the books - ''without'' the Debian server package

The installation is expected to happen from within the BOINC source tree.

[ -d "$installroot" ] || sudo mkdir -p "$installroot"
sudo ./tools/make_project \
  --url_base "$hosturl" \
  --db_name "$dbname" \
  --db_user "$dbuser" \
  --db_passwd "$dbpasswd" \
  --drop_db_first  \
  --project_root /var/www/boinc/$projectname \
    "$projectname" "$projectnicename"

1.2.2.2. Installation ''with'' the Debian server package

If you use boinc-server-maker package you need to use this command instead:

sudo PYTHONPATH=$PYTHONPATH:/usr/share/python-support/boinc-server/Boinc/ \
 /usr/share/boinc-server/tools/make_project \
  --url_base "$hosturl" \
  --db_name "$dbname" \
  --db_user "$dbuser" \
  --db_passwd "$dbpasswd" \
  --drop_db_first  \
  --project_root "$installroot"/"$projectname" \
  --srcdir /usr/share/boinc-server-maker/ \
   "$projectname" "$projectnicename"

source

source

The screen output should not look too different from

Creating project 'BoincTestProject@Home' (short name 'boinctest'):
   PROJECT_ROOT = /var/tmp/boinc/boinctest/
       URL_BASE = http://boincserver.dyndns.org/
  HTML_USER_URL = http://boincserver.dyndns.org/boinctest/
   HTML_OPS_URL = http://boincserver.dyndns.org/boinctest_ops/
        CGI_URL = http://boincserver.dyndns.org/boinctest_cgi/
        KEY_DIR = /var/tmp/boinc/boinctest/keys/
        DB_NAME = boinctest
        DB_HOST =

Delete /var/tmp/boinc/boinctest/? [y/N]  y
Deleting /var/tmp/boinc/boinctest/

Continue? [Y/n]
Setting up server: creating directories

Keys don't exist in /var/tmp/boinc/boinctest/keys/; generate them? [Y/n]
Setting up server files: generating keys
Setting up server files: copying files
Setting up database
Setting up server files: writing config files
Setting up server files: linking cgi programs
update_translations finished
Done installing default daemons.

Done creating project.  Please view
/var/tmp/boinc/boinctest/boinctest.readme
for important additional instructions.

That 'boinctest.readme' basically says what the next section is presenting.

1.2.3. Continue - with or without the Debian BOINC server package

1.2.3.1. Adjusting permissions for project directory

  1. Make sure the configuration variables are set

    [ -r ~/.boinc_test.conf ] && . ~/.boinc_test.conf
  2. Change files and directories permission:

    if [ -z "$installroot" -o -z "$fileprojectname" ]; then
      echo "Not all variables are set for the configuration"
      echo "Error, do not continue."
    elif [ ! -d "$installroot"/"$fileprojectname" ]; then
      echo "The directory '$installroot/'$fileprojectname' is not existing"
      echo "Error, do not continue."
    else
      cd "$installroot"/"$fileprojectname"
      sudo chown root:boincadm  -R .
      sudo chmod g+w -R .
      sudo chmod 02770 -R upload html/cache html/inc html/languages html/languages/compiled html/user_profile
    fi

A successful execution will once ask for the user's sudo password but otherwise remain quiet.

If those permissions are sufficient depends on the user the the web service runs with. For Debian, running apache as user www-data, the include files need to be accessible. Those they should be with www-data added to the group boincadm with those files group accessible and group writable. However, there seem to be some remaining issues. To circumvent them, run

if [ -d html/inc -a -d cgi-bin ]; then
  sudo chmod    o+x html/inc
  sudo chmod -R o+r html/inc
  sudo chmod    o+x html/languages/
  sudo chmod    o+x html/languages/compiled
else
  echo "You are not in your project directory"
fi

The "-r" is important to reach all the files since the directory cannot be read before the execution of "sudo".

1.2.3.2. Automated restart upon failure

  1. Add project cronjob:

    sudo crontab -e

    Then add this line to editor appear after run command above

    0-60/5 * * * * /var/boinc/testproj/bin/start --cron

    Verify that the cron entry was indeed accepted

    sudo crontab -l
  2. Config password for BOINC server administrative page:

    sudo htpasswd -c html/ops/.htpasswd USERNAME
  3. Config Apache to call BOINC server

    sudo cp ${fileprojectname}.httpd.conf  /etc/apache2/sites-available/
    sudo a2ensite  ${fileprojectname}.httpd.conf
    sudo /etc/init.d/apache2 reload

1.3. Test web interface

The web site should now be accessible. It should be available as an alias "$fileprojectname", please look it up in "/etc/apache2/sites-enabled/${filesprojectname}.httpd.conf". You will find the automated home page to need some further manual adjustments - which does not really encourage to update the server version too frequently, we presume.

If the home page does not show up then there may be some missing file or some bogus file permission. The inspection of "/var/log/apache/error.log" should bring sufficient insights to fix the issue.

1.3.1. Showing index.php by default

With a typical Debian apache setting, the expected BOINC web interface only shows up after clicking on index.php. Edit /etc/apache2/sites-enabled/${filesprojectname}.httpd.conf as follows to show index.php files directly:

<Directory "...fileprojectname.html">
        Options Indexes FollowSymlinks MultiViews +ExecCGI
...
        # Adding these three lines
        <IfModule mod_dir.c>
          DirectoryIndex index.php
        </IfModule>
        #
</Directory>

1.3.2. Substituting ProjectName in *.html and *.php

The web pages show whenever there should be the project name the string "REPLACE WITH PROJECT NAME". To fix this and other things like your public email address, edit html/project/project.inc .

The project start page is not appreciated by everyone. Amend it in './html/user/index.php' but wary of subsequent updates. How exactly this will all be dealt with for updating is not yet clear. For the moment it is just suggested to have a backup when installing the next versionof BOINC and then compare the HTML references made from that page. Many are providing a home page to their project that is kept separate from the BOINC-intrinsic workload distribution, i.e. that BOINC-provided index.php page.

2. Add functionality

This section brings your BOINC project to life.

2.1. Setting up BOINC's Example Application ''upper_case''

This section is currently been brought to some better shape at BOINC/ServerGuide/AppDeployment.

2.2. Add your own application

The only thing that counts to have is the application that helps your very personal task at hand. This will be addressed in the upcoming section. The section at hand will take one of the example projects.

Previously

Now

2.2.1. Compiling sources oneself

Have the BOINC source code untarred to ~/boinc, the libraries compiled, and then

cd ~/boinc/samples/wrapper
make

This process needs to be repeated for every Linux architecture you plan to support. This is why typically only the most mainstreamish Linux flavours are supported, i.e. amd64 and i386.

2.2.2. Starting from the Debian package

all done

source

source

2.3. Start BOINC demons

Now its time to start the project, assuming we are still inside the project root,

$./bin/start
Staying in ENABLED mode
Starting daemons
  Starting daemon: feeder -d 3
  Starting daemon: transitioner -d 3
  Starting daemon: file_deleter -d 3

If you receive the message "No daemons for this host found - check host name in config.xml", then do just that and set the X in <hostname>X</hostname> to what the UNIX command hostname returns on your machine.

And ./bin/status can be used to check status of the project.

$ ./bin/status
BOINC is ENABLED

DAEMON  pid  status      lockfile disabled  commandline
   1  16574  running     locked   no        feeder -d 3
   2  16576  running     locked   no        transitioner -d 3
   3  16579  running     locked   no        file_deleter -d 3

TASK last run period    next run lock file disabled commandline
   1    ?     24 hours     NOW   unlocked  yes       db_dump -d 2 -dump_spec ../db_dump_spec.xml
   2    ?     1 days       NOW   unlocked  yes       run_in_ops ./update_uotd.php
   3    ?     1 hour       NOW   unlocked  yes       run_in_ops ./update_forum_activities.php
   4    ?     7 days       NOW   unlocked  yes       update_stats -update_users -update_teams -update_hosts
   5    ?     24 hours     NOW   unlocked  yes       run_in_ops ./update_profile_pages.php
   6    ?     24 hours     NOW   unlocked  yes       run_in_ops ./team_import.php
   7    ?     24 hours     NOW   unlocked  yes       run_in_ops ./notify.php

2.4. Config clients

The project should now be ready to run a couple of work units. Please use any Linux BOINC client. The Debian boinc-client package will do. It needs to be Linux since our application was yet only prepared for this OS. The project URL is $hosturl/$fileprojectname.

2.5. Inspection

Once the user was created, the respective entry in the database will look like

mysql> select id,create_time,email_addr,name,authenticator,country,postal_code,venue,url,email_validated,donated from user;
 id | create_time | email_addr       | name | authenticator | country | postal_code | venue | url  |  email_validated | donated
----+-------------+------------------+------+----------------------------------+---------+---------+------------------+---------+
  1 |  1310218977 | your@address.com |  you | ce45e2fe50ae6 | Country | nnn         |       | NULL |                0 |       0 |

and when the first result was computed, it will show as

mysql> select id,create_time,workunitid,server_state,outcome,hostid,userid,name,cpu_time,app_version_num,appid from result; +


+


+


+


+


+


+


+


+


+


+


+ | id | create_time | workunitid | server_state | outcome | hostid | userid | name | cpu_time | app_version_num | appid | +


+


+


+


+


+


+


+


+


+


+


+ | 1 | 1310217514 | 1 | 5 | 1 | 1 | 1 | test_0 | 19.94525 | 612 | 11 | +


+


+


+


+


+


+


+


+


+


+


+

Result files are at

$ find upload/
upload/
upload/ba
upload/ba/test_0_0

And look like expected

$ cat upload/ba/test_0_0 
TEST STRING

3. Management of your own project

This section describes how to keep a project running.

3.0.1. Configuration - summary

Above, a series of environment variables was specified to separate the invariant instructions from those bits that are project specific. Truly, each BOINC project needs its very own set of configuration variables. It is needed as long the project lives to allow for updates and routine maintenance. To avoid expensive typos, project-specific configurations are suggested, like ~/.boinc_projectname.conf.

For our test project, the configuration file is ~/.boinc_test.conf and this will look like,

# password for write access to your project database
pw=MYSQLPASSWORDFORBOINCUSER
# name of the MySQL database
dbprojectname=boinctest
# address of host (via DNS or IP number) at which project server shall be reached
hosturl=http://a.b.c.d
# name of folder in which data shall be stored, also becomes part of project URL
fileprojectname=$dbprojectname
# more human-compatible way to read the project name
niceprojectname="BoincTestProject@Home"
# location at which sources shall be kept
installroot=/var/tmp/boinc

So while maintaining the project all we have to do is

source ~/.boinc_test.conf

and execute the commands using the standard variables.

This section still needs to be written. Frankly, we don't know, yet. To clarify

4. Why to use (or not to use) the Debian BOINC server maker package

The concept of the BOINC server package is slightly irritating in that it itself is not a working server but something from which a server infrastructure is generated. It provides all the binaries and header files for local developments and thus saves some considerable time to prepare such and ensures to have all the libraries in place that the community has determined to be of help.

The typical advantages of Debian do apply. It should for instance be possible, to change the server architecture at any time from one platform to another. If it is running on your laptop, it should e.g. also run on an ARM netplug.

The package is still too experimental to use in a production environment. But, hey, we will be getting there over time.

5. See also

5.1. Back to