Contents
This page explains how to create units of work for one's self-run BOINC project.
1. Add a Work Unit
A work unit is a portion of the project's data that the project should be analysed with a particular application. Applications and data can be shared between work units. The following parts need to be known for the creation of a workunit in the database.
- Input file(s)
- Work Unit template
- Result Template
1.1. Input files
Input files are what the application should be working on. Typically those vary for every work unit, like the range of numbers in which to search for primes. We will learn at some later stage that such inputs can be shared between work units, like a particular protein structure against which a large set of ligands shall be tested for their molecular interaction.
The example application this tutorial addresses converts files in mixed case to a file with all letters in upper case. For this, we manually create an invariant test input file. It is a one-lined arbitrary string, stored in $installroot/$projectname/download/input_file in all lower case. The application run will convert it into all upper case.
. ~/.boinc_test.conf if test -n "$installroot" -a -n "$projectname" && cd $installroot/$projectname; then sudo strace -o download/input_file -- strace -h > /dev/null && echo "[ok]" || echo "[failed]" else echo "E: please check parameters in ~/.boinc_test.conf" fi
1.2. Template files
Copy the default work unit and result template for upper case application to the templates folder.
# The paths are to be changed in the next version of package, so watch out. sudo cp /usr/share/doc/boinc-server-maker/examples/upper_case_{result,wu} $installroot/$projectname/templates/
That is just two files
$ ls templates/ upper_case_result upper_case_wu
Those are XML formatted descriptions of how workunits and results shall be looking like. In deep theory, every result could well look different, say lets one expect larger or smaller result files, depending on what the input is. But typically one would just specify an upper limit of file sizes, e.g. to reduce the negative effect that an evil doer or an error in the application might otherwise have.
The files are not overly complicated. In their basic form, this means. There is a job sumission documentation on the BOINC Wiki indicating much more thoughts to possibly be put into it all:
$ cat /var/tmp/boinc-server-test/boinctest/templates/upper_case_result <file_info> <name><OUTFILE_0/></name> <generated_locally/> <upload_when_present/> <max_nbytes>10000</max_nbytes> <url><UPLOAD_URL/></url> </file_info> <result> <file_ref> <file_name><OUTFILE_0/></file_name> <open_name>out</open_name> </file_ref> </result>
and similarly
$ cat /var/tmp/boinc-server-test/boinctest/templates/upper_case_wu <file_info> <number>0</number> </file_info> <workunit> <file_ref> <file_number>0</file_number> <open_name>in</open_name> </file_ref> <min_quorum>1</min_quorum> <target_nresults>1</target_nresults> </workunit>
which with some confidence is working for quite a number of different projects.
1.3. Fill the database
We will now introduce the system to a single work unit - only a single. The input remains described in the file input_file, templates are specified individually for result and work unit (wu). The application name is read from the command line, not from the wu description. This way, one could have different applications work on the same work unit more easily. Every unit is also passed a name, which the user sees in the BOINC manager and shall identify the work unit.
We will call the workunit test and to commit it to the database, execute
[ -n "$installroot" -a -n "$projectname" ] && \ cd $installroot/$projectname && \ sudo ./bin/create_work -appname upper_case -wu_name test -wu_template templates/upper_case_wu -result_template templates/upper_case_result input_file || echo "E: something failed"
1.4. Uploading multiple work units
All input files need to be created at some stage. So they will be available, most likely as some file or they can be made available as a file. For batch invocations of create_work, one can use the shell's loop functionality. For the BASH, this could for instance be
for i in $(seq 1 3); do echo ./bin/create_work -appname upper_case -wu_name test$i input_${i}_file done
yielding
./bin/create_work -appname upper_case -wu_name test1 input_1_file ./bin/create_work -appname upper_case -wu_name test2 input_2_file ./bin/create_work -appname upper_case -wu_name test3 input_3_file
1.5. Inspections (optional)
1.5.1. Files created
The download folder will now have a directory created especially for this workunit. To find this (or other modifications after the execution of some BOINC tool) seek the latest modified folder:
$ ls -ltr $installroot/$projectname ... drwxrwxr-x 2 root boincadm 4096 Jul 9 13:40 templates drwxrwxr-x 3 root boincadm 4096 Jul 9 14:09 download
and in download this is
$ ls -ltr $installroot/$projectname/download | tail -1 drwxrwx--x 2 root root 4096 Jul 9 14:09 76
which again shows
ls -ltr $installroot/$projectname/download/76 total 4 -rw-r--r-- 1 root root 12 Jul 9 14:09 input_file
which is the very same file.
1.5.2. Database inspection
mysql> select id,create_time,appid,name,fileset_id from workunit; | id | create_time | appid | name | fileset_id | +----+-------------+-------+------+------------+ | 1 | 1310213354 | 11 | test | 0 |
2. Multifile Applications
2.1. Multiple Binaries
With the renewed application-version-platform hierarchy outlined earlier, applications that consists of more than one file are already covered. We will get back to it in the context of legacy applications being wrapped for BOINC compatibility, which needs two apps - the wrapper and the legacy one. Please skim through ?BOINC/WrapperApp to mentally prepare for it.
2.2. Multiple Input/Output Files
The input/output files are specified in the workunit and result templates respectively. The <file_info>, <file_ref> tag pairs each describes a file. For example, if we need to create a workunit for test application which takes in1, in2 as input files, the workunit template would be
$ cat templates/test_wu <file_info> <number>0</number> </file_info> <file_info> <number>1</number> </file_info> <workunit> <file_ref> <file_number>0</file_number> <open_name>in1</open_name> </file_ref> <file_ref> <file_number>1</file_number> <open_name>in2</open_name> </file_ref> </workunit>
Please see each file has a <file_info> and <file_ref> tag. The same tags are used in case of result template,i.e if our application generates two output files, out1, out2 which should be transported back, The result template would be
$ cat template/test_result <file_info> <name><OUTFILE_0/></name> <generated_locally/> <upload_when_present/> <max_nbytes>10000</max_nbytes> <url><UPLOAD_URL/></url> </file_info> <file_info> <name><OUTFILE_1/></name> <generated_locally/> <upload_when_present/> <max_nbytes>10000</max_nbytes> <url><UPLOAD_URL/></url> </file_info> <result> <file_ref> <file_name><OUTFILE_0/></file_name> <open_name>out1</open_name> </file_ref> <file_ref> <file_name><OUTFILE_1/></file_name> <open_name>out2</open_name> </file_ref> </result>
3. References
JobSubmission entry on BOINC Wiki
AppVersionNew entry on BOINC Wiki
Back to BOINC/ServerGuide