1. BOINC Server Setup: Project Initialisation and the Default Website
1.1. Preparation
There is nothing to prepare if you do not go for a real project. Just try with the machine at hand. We maintain the boinc-server-maker package in the "experimental" section of Debian. The reason for that is that we do not want our users to use code that is likely to be two years old and invite the public to that server. It would miss any security leak that has since been identified and made public. As for any package distributed in experimental, it only installs to Debian "testing" (Stretch at the time of writing) or Debian "unstable" because of its most modern dependencies.
For projects inviting the public prepare for
- disk space
- network security
- extra IT-savvy eyeballs for all the unforeseen things that may happen and need quick reactions to get fixed
For the moment, the Debian package does not create its own boincadm (speak: admin) user. We suggest the following, your local admins may have different ideas - talk back to us:
sudo adduser --system --group boincadm --no-create-home sudo usermod -aG boincadm www-data
1.2. Software: BOINC Server Template files
This section first explains the installation of the BOINC server template sources from the source tree and then the same for those using the Debian boinc-server-maker package.
Previously |
Now |
||
1. |
Install BOINC server dependencies: |
1. |
Make sure that the experimental repository is in /etc/apt/sources.list: |
2. |
Make sure that your source list is up-to-date: |
||
2. |
Download BOINC source: |
3. |
Install boinc-server-maker package just like any other: |
3. |
Compile BOINC: |
The compilation of the upstream source tree is nothing like magic. Go for it if you can. If our packaging we have done the right way, then this tutorial is of value for you because of the complete identity of all downstream steps. We ship the package in the experimental section for two reasons. A first is that we do not have any particular routine in setting those servers up. The second is that developments of BOINC Server and Client sides are not always stable at the same time, thus having the server in experimental grants us some extra flexibility.
1.3. Project-specific configuration
The preparation of the final project from the BOINC server template is a manual process. That should be applied only once for every project and then for updates of the BOINC server sources. One could (in principle) prepare Debian packages for such project specific configurations. The boinc-server-maker package described on this page would then become a build dependency for the project. See BOINC/Server/Projects for current (early) attempts.
1.3.1. Create and config BOINC project
The BOINC server as compiled from the BOINC sources does not perform directly.
The following needs to be performed for every project, with or without the Debian BOINC server package. In the following we will work on a single BASH shell. This allows us to use mostly self-explanatory variables to help your local adaptation.
To help working consistently on multiple shells, or to get the settings when logging in again, we need a place to redirect the parameters of the project configuration. We do that only to help ourselves; the BOINC code will not use it. This approach is well accepted, e.g. for .bash_aliases.
1.3.1.1. Initialisation of the configuration
For a first start of a project create the file that the project parameters shall be stored in.
echo "boincserverconfig=~/.boinc_test.conf" > ~/.boinc_test.conf chmod 600 ~/.boinc_test.conf
Please run through this file now to set the boincserverconfig variable we also need below
. ~/.boinc_test.conf
and whenever starting a shell, these settings should be set, so you may decide to have an account dedicated for your project and have the configration sourced immediately after the shell was started, e.g. for BASH like:
cat >> ~/.bashrc <<EOBASHRC if [ -r "$boincserverconfig" ]; then . $boincserverconfig fi EOBASHRC
1.3.1.2. Parameters for the MySQL database
cat << EODBCONFIG >> $boincserverconfig # name of the MySQL database dbname=boinctest # name of the MySQL user with access to above database dbuser=boincadm # password for write access to your project database dbpasswd=MYSQLPASSWORDFORBOINCUSER EODBCONFIG
For your convenience you may copy above code and then use an editor to change it.
1.3.1.3. Create MySQL database for BOINC project
With the variables defined before, you can just copy and pasted the code below. The password asked is the password for the root user to access the database, which is not unlikely to differ from the root password to your UNIX system.
# read config if available [ -r "$boincserverconfig" ] && . $boincserverconfig if [ -z "$dbuser" ]; then echo "Variable 'dbuser' not set"; elif ! echo "DROP USER '$dbuser'@'localhost'" | mysql -u root -p; then echo "If the removal of the previous user fails because the user is not existing, then this does not matter. Other errors please investigate." fi
You are asked in the block above for a password and you will be again in the block below.
if [ -z "$dbname" ]; then echo "Variable 'dbname' not set"; elif [ -z "$dbpasswd" ]; then echo "Variable 'dbpasswd' not set"; else # piping commands to mysql shell cat <<EOMYSQL | mysql -u root -p; DROP DATABASE IF EXISTS $dbname; CREATE USER '$dbuser'@'localhost' IDENTIFIED BY '$dbpasswd'; GRANT ALL PRIVILEGES ON $dbname.* TO '$dbuser'@'localhost'; EOMYSQL fi
Over time, and if it is alone for backups, a basic familiarity with MySQL is (re|a)quired for the maintenance of BOINC projects.
1.3.2. Create project
Please replace a.b.c.d with your full hostname or your server's IP address. If you only have dynamic IP addresses, then this is unfortunate, but investigate the service of dyndns.org to help out.
cat <<EOCONF >> $boincserverconfig # address of host (via DNS or IP number) at which project server shall be reached hosturl=http://a.b.c.d # name of folder in which data shall be stored, also becomes part of project URL projectname=boinctest # more human-compatible way to read the project name projectnicename="BoincTestProject@Home" # location to store the project files, a projectname subdirectory will be created installroot=/var/lib/boinc-server-test EOCONF
Again, please edit '~/.boinc_test.conf' with a text editor if required. This example installroot is where you cannot harm anything. Some linux distributions may however have that directory cleaned at boot time. Please adjust for permanent location.
So, to make sure you have indeed sourced the configuration paramters:
if [ ! -r "$boincserverconfig" ]; then echo "Configuration file '$boincserverconfig' not existing." else . $boincserverconfig echo -n "Basic configuration test: " if [ -z "$installroot" -o -z "$hosturl" -o -z "$dbname" -o -z "$dbpasswd" \ -o -z "$projectnicename" -o -z "$projectname" ] ; then echo "Missing configuration parameter." else echo "[ok]" fi fi
Previously |
Now |
1.3.2.1. Installation according to the books - ''without'' the Debian server packageThe installation is expected to happen from within the BOINC source tree. [ -d "$installroot" ] || sudo mkdir -p "$installroot" sudo ./tools/make_project \ --url_base "$hosturl" \ --db_name "$dbname" \ --db_user "$dbuser" \ --db_passwd "$dbpasswd" \ --drop_db_first \ --project_root /var/www/boinc/$projectname \ "$projectname" "$projectnicename" |
1.3.2.2. Installation ''with'' the Debian server packageIf you use boinc-server-maker package you need to use this command instead: sudo PYTHONPATH=$PYTHONPATH:/usr/share/python-support/boinc-server/Boinc/ \ /usr/share/boinc-server/tools/make_project \ --url_base "$hosturl" \ --db_name "$dbname" \ --db_user "$dbuser" \ --db_passwd "$dbpasswd" \ --drop_db_first \ --project_root "$installroot"/"$projectname" \ --srcdir /usr/share/boinc-server-maker/ \ "$projectname" "$projectnicename" |
With the original source tree or the Debian package, the project configuration is basically identical. The specification of the 'PYTHONPATH' should not be required. But it is for the moment, leaving some room for optimisation.
The '--test_app' flag is omitted to help the understanding of the setup. Also, it requires the compilation of an example_app which are all distributed separately from the boinc-server-maker package in the boinc-app-examples package, but not in the location that the install scripts expects them.
The screen output should not look too different from
Creating project 'BoincTestProject@Home' (short name 'boinctest'): PROJECT_ROOT = /var/tmp/boinc/boinctest/ URL_BASE = http://boincserver.dyndns.org/ HTML_USER_URL = http://boincserver.dyndns.org/boinctest/ HTML_OPS_URL = http://boincserver.dyndns.org/boinctest_ops/ CGI_URL = http://boincserver.dyndns.org/boinctest_cgi/ KEY_DIR = /var/tmp/boinc/boinctest/keys/ DB_NAME = boinctest DB_HOST = Delete /var/tmp/boinc/boinctest/? [y/N] y Deleting /var/tmp/boinc/boinctest/ Continue? [Y/n] Setting up server: creating directories Keys don't exist in /var/tmp/boinc/boinctest/keys/; generate them? [Y/n] Setting up server files: generating keys Setting up server files: copying files Setting up database Setting up server files: writing config files Setting up server files: linking cgi programs update_translations finished Done installing default daemons. Done creating project. Please view /var/tmp/boinc/boinctest/boinctest.readme for important additional instructions.
That 'boinctest.readme' basically says what the next section is presenting. If running into issues of various sorts, you may be suggested to repeat and overwrite ("clobber") the previous install. Caveat: when adding "--delete_prev_inst --drop_db_first", make sure to add them somewhere in the middle, not just comfortably at the end. The last two arguments are interpreted in a special way.
1.3.3. Continue
The instructions below are the same for installs that originated with the original BOINC distibution and those that used boinc-server-maker.
1.3.3.1. Adjusting permissions for project directory
Make sure the configuration variables are set
[ -r $boincserverconfig ] && . $boincserverconfig
Change files and directories permission:
if [ -z "$installroot" -o -z "$projectname" ]; then echo "Not all variables are set for the configuration" echo "Error, do not continue." elif [ ! -d "$installroot"/"$projectname" ]; then echo "The directory '$installroot/'$projectname' is not existing" echo "Error, do not continue." else cd "$installroot"/"$projectname" sudo chown boincadm:boincadm -R . sudo chmod g+w -R . sudo chmod 02770 -R upload html/cache html/inc html/languages html/languages/compiled html/user_profile hostname=`hostname` if [ -z "$hostname" ]; then echo "Please specify the hostname" else sudo chgrp -R www-data log_"$hostname" upload fi fi
A successful execution will once ask for the user's sudo password but otherwise remain quiet.
If those permissions are sufficient depends on the user the the web service runs with. For Debian, running apache as user www-data, the include files need to be accessible. FIXME: How exactly this is best achieved, still needs some extra thinking. To circumvent all issues for the moment, run
if [ -d html/inc -a -d cgi-bin ]; then echo -n "html/inc: "; sudo chmod o+x html/inc && sudo chmod -R o+r html/inc && echo "[ok]" || echo "[failed]" echo -n "html/languages: "; sudo chmod o+x html/languages/ html/languages/compiled && echo "[ok]" || echo "[failed]" else echo "You are not in your project directory" fi
The "-r" is important to reach all the files since the directory cannot be read before the execution of "sudo".
1.3.3.2. Start of BOINC background programs ("daemons")
Now its time to start the project, assuming we are still inside the project root,
$./bin/start Staying in ENABLED mode Starting daemons Starting daemon: feeder -d 3 Starting daemon: transitioner -d 3 Starting daemon: file_deleter -d 3
If you receive the message "No daemons for this host found - check host name in config.xml", then do just that and set the X in <hostname>X</hostname> to what the UNIX command hostname returns on your machine.
And ./bin/status can be used to check status of the project.
$ ./bin/status BOINC is ENABLED DAEMON pid status lockfile disabled commandline 1 16574 running locked no feeder -d 3 2 16576 running locked no transitioner -d 3 3 16579 running locked no file_deleter -d 3 TASK last run period next run lock file disabled commandline 1 ? 24 hours NOW unlocked yes db_dump -d 2 -dump_spec ../db_dump_spec.xml 2 ? 1 days NOW unlocked yes run_in_ops ./update_uotd.php 3 ? 1 hour NOW unlocked yes run_in_ops ./update_forum_activities.php 4 ? 7 days NOW unlocked yes update_stats -update_users -update_teams -update_hosts 5 ? 24 hours NOW unlocked yes run_in_ops ./update_profile_pages.php 6 ? 24 hours NOW unlocked yes run_in_ops ./team_import.php 7 ? 24 hours NOW unlocked yes run_in_ops ./notify.php
1.3.3.3. Automated restart upon failure
One can restart and restart the BOINC project without any harm. If it is already running, no change is performed. If something had stopped functioning, for whatever reason (it is a bug, most likely of because of some rare and hence barely tested condition) then the failing tool should be restarted. Here is how to do it.
Add project cronjob:
sudo EDITOR=vim crontab -e
Please check the exact path of the 'start' script. Then add a line like the following in the editor that spawned by the 'crontab -e'
command above. You can set the editor to be started by the EDITOR environment variable. Adjust to your liking.
*/5 * * * * /var/tmp/boinc-server-test/boinctest/bin/start --cron
The above indicates a restart (in case of failure) every five minutes. Verify that the cron entry was indeed accepted
sudo crontab -l
Config password for BOINC server administrative page:
sudo htpasswd -c html/ops/.htpasswd USERNAME
Config Apache to call BOINC server
sudo cp ${projectname}.httpd.conf /etc/apache2/sites-available/ \ && sudo a2ensite ${projectname}.httpd.conf \ && sudo /etc/init.d/apache2 reload
1.4. Access web interface
1.4.1. Apache preparation: Ensuring PHP and cgi apache modules are enabled, not only installed
If you see a page but that looks weird, then PHP may not be executed on that page but the source of the file is shown directly. To enable it, perform
cd /etc/apache2/mods-enabled sudo ln -s ../mods-available/php5.* . sudo ln -s ../mods-available/cgi.* . sudo /etc/init.d/apache2 restart
alternative to the above you may prefer to write from anywhere
sudo a2enmod cgi sudo a2enmod php5 sudo /etc/init.d/apache2 restart
1.4.2. Have a look
The web site should now be accessible. It should be available as an alias "$projectname", please look it up in "/etc/apache2/sites-enabled/${projectname}.httpd.conf". You will find the automated home page to need some further manual adjustments - which does not really encourage to update the server version too frequently, we presume.
If the home page does not show up then there may be some missing file or some bogus file permission. The inspection of "/var/log/apache/error.log" should bring sufficient insights to fix the issue. The image shows a URL with a ":8080", which indicates the browser to ask for the page at a different port than the default ":80". For doing so, add a virtual host configuration in above httpd.conf of your project. The Apache Documentation explains vividly how to do so.
1.4.3. Trouble shooting
1.4.3.1. Showing index.php by default
With a typical Debian apache setting, the expected BOINC web interface only shows up after clicking on index.php. It may be different for your setup, just if your project homepage looks more like source code then edit /etc/apache2/sites-enabled/${filesprojectname}.httpd.conf as follows to show index.php files directly:
<Directory "...fileprojectname.html"> Options Indexes FollowSymlinks MultiViews +ExecCGI ...not shown... # Adding these three lines <IfModule mod_dir.c> DirectoryIndex index.php </IfModule> </Directory>
Perform sudo /etc/init.d/apache2 reload after the change. Once PHP is properly configured, all dynamic web pages, including for accessing the database and message boards will run. You can also get your own account, just the pages all look ugly and unpersonal, still.
1.4.3.2. Ensure compatibility with extra security of Apache 2.4
Apache gained some extra security by not allowing access to files outside /var/www in its default configuration. To overcome this, add the line
Require all granted
to both directory instructions at /etc/apache2/sites-enabled/sleep.httpd.conf. This should already be addressed by later versions of the boinc-server-maker package.
1.4.4. Substituting ProjectName in *.html and *.php
The web pages show whenever there should be the project name the string "REPLACE WITH PROJECT NAME". To fix this and other things like your public email address, edit the file html/project/project.inc with any text editor, as in
sudo vim html/project/project.inc
That include file is read by various other PHP files and amongst the above mentioned user-visible project name and email addresses also allows for extra decorations for headers or footers.
While a project develops, one is likely to perform many additional changes well beyond initial cosmetics. The project's home/start page is a common initial target. Amend it in './html/user/index.php' but wary of subsequent updates. How exactly this will all be dealt with for updating is not yet clear. For the moment it is just suggested to have a backup when installing the next version of BOINC and then compare the HTML references made from that page. There are several tools known for software development which may come to a rescue, like bazaar, mercurial or git, that may help with the orchestrations of updates that come from new upstream developments and local additions. But that is independent from the Debian packaging side.
1.4.5. Get up forums up (optional)
Now you have the user-interface of your BOINC server up: the PHP web pages. The subfolder 'html/ops' contains the website management tools that are not called by the user but demand access through that admin password set above. The .PHP files are executed by a little engine within the Apache web server to display the results, in reaction to the respective PATH being selected via the browser. An alternative, and easier to many, is to call the same .PHP files through the command line.
As an example may serve the initialisation of the news. The screen shot shows it, where there should be the latest project news, there is a mere suggestion, reading "No news forum. Run html/ops/create_forums.php". To follow that advice, on the command line
[ -r "$boincserverconfig" ] && . $boincserverconfig if [ ! -r create_forums.php ]; then if [ -r html/ops/create_forums.php ]; then cd html/ops elif [ -r $installroot/$fileprojectname/html/ops/create_forums.php ]; then cd $installroot/$fileprojectname/html/ops else echo "Could not find create_forums.php" fi fi
Then edit the file to adjust the forums you want to be created or just
sudo sed -i '/remove the die/d' create_forums.php
to get the default for a start. The 'sed' command (so nice and helpful, but initially awkward) edits the .php script by removing an early exit. I means: search for a string "remove the die" and then deletes that line. If avoiding the sed (stream editor) tool, do so manually with your editor.
The execution of 'create_forums.php' can then be performed on the command line by
php5 create_forums.php
The user interface is - from a technical point of view - now set. It remains the application side to be configured.
Proceed with AppDeployment to learn about collecting binaries from multiple platforms of Debian and configure BOINC to serve them.
1.4.6. Comments
I tested this guide on Debian stretch with experimental repos. I faced an error mysql_exceptions.?OperationalError: (1071, 'Specified key was too long; max key length is 767 bytes') The reason this happens was that the default config of mysqld is
character-set-server = utf8mb4 collation-server = utf8mb4_general_ci
while the recommended settings for BOINC are
character-set-server=utf8 collation-server=utf8_general_ci
So the mysqld config should be changed.
Another issue was a PHP exception Call to undefined function simplexml_load_file() boinc This is simply solved by
apt-get install php7.0-mbstring php7.0-zip php7.0-xml