11106
Comment: more text, more quotes
|
11078
Added sample outputs to start and status programs, fixed xadd output.
|
Deletions are marked like this. | Additions are marked like this. |
Line 37: | Line 37: |
sudo mkdir "$appdir" | sudo mkdir -p "$appdir" |
Line 43: | Line 43: |
appver=6.12.33 # adjust to the right version boincplat=amd64-pc-linux-gnu # adjust to your architecture, maybe i686-pc-linux-gnu sudo cp $(dpkg -L boinc-app-examples | grep upper_case) $appdir/uppercase_${appver}_${boincplat} |
appver=6.12 # adjust to the right version, only have single "." boincplat=$(arch)-pc-linux-gnu # adjust to your architecture, maybe i686-pc-linux-gnu sudo cp $(dpkg -L boinc-app-examples | grep upper_case) $appdir/upper_case_${appver}_${boincplat} |
Line 49: | Line 49: |
Please keep the version formatted that simply - or change the BOINC source code. |
|
Line 61: | Line 63: |
(you have used "upper_case" below and "uppercase" above ?!?!?) |
|
Line 67: | Line 67: |
echo 'Variables $installroot ('$installroot ') and $fileprojectname (' $fileprojectname ') are both required.' | echo 'Variables $installroot (' $installroot ') and $fileprojectname (' $fileprojectname ') are both required.' |
Line 70: | Line 70: |
<app> | <boinc> <app> |
Line 73: | Line 74: |
</app> | </app> <platform> <name>i686-pc-linux-gnu</name> <user_friendly_name>Linux/x86</user_friendly_name> </platform> <platform> <name>x86_64-pc-linux-gnu</name> <user_friendly_name>Linux/amd64</user_friendly_name> </platform> </boinc> |
Line 79: | Line 89: |
The platforms need not be defined here, since the default ''project.xml'' contains entries for all platforms, They are shown here for the user to understand finer details, and having duplicate entries on database will generate errors. In most cases, you can just append the application entries to the default ''project.xml'' file and run xadd. |
|
Line 83: | Line 96: |
./bin/xadd }}} The screen output will look like this, {{{ Processing <Platform#None windows_intelx86> ... Committed <Platform#1 windows_intelx86> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 1L, 'name': 'windows_intelx86', 'user_friendly_name': 'Microsoft Windows (98 or later) running on an Intel x86-compatible CPU'} Processing <Platform#None windows_x86_64> ... Committed <Platform#2 windows_x86_64> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 2L, 'name': 'windows_x86_64', 'user_friendly_name': 'Microsoft Windows running on an AMD x86_64 or Intel EM64T CPU'} |
sudo bin/xadd }}} The local screen output will be similar to {{{ |
Line 110: | Line 104: |
'create_time': 1308990279L, | 'create_time': 1308988632L, |
Line 115: | Line 109: |
Processing <Platform#None x86_64-pc-linux-gnu> ... Committed <Platform#4 x86_64-pc-linux-gnu> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 4L, 'name': 'x86_64-pc-linux-gnu', 'user_friendly_name': 'Linux running on an AMD x86_64 or Intel EM64T CPU'} Processing <Platform#None powerpc-apple-darwin> ... Committed <Platform#5 powerpc-apple-darwin> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 5L, 'name': 'powerpc-apple-darwin', 'user_friendly_name': 'Mac OS X 10.3 or later running on Motorola PowerPC'} Processing <Platform#None i686-apple-darwin> ... Committed <Platform#6 i686-apple-darwin> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 6L, 'name': 'i686-apple-darwin', 'user_friendly_name': 'Mac OS 10.4 or later running on Intel'} Processing <Platform#None x86_64-apple-darwin> ... Committed <Platform#7 x86_64-apple-darwin> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 7L, 'name': 'x86_64-apple-darwin', 'user_friendly_name': 'Intel 64-bit Mac OS 10.5 or later'} Processing <Platform#None sparc-sun-solaris2.7> ... Committed <Platform#8 sparc-sun-solaris2.7> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 8L, 'name': 'sparc-sun-solaris2.7', 'user_friendly_name': 'Solaris 2.7 running on a SPARC-compatible CPU'} Processing <Platform#None sparc-sun-solaris> ... Committed <Platform#9 sparc-sun-solaris> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 9L, 'name': 'sparc-sun-solaris', 'user_friendly_name': 'Solaris 2.8 or later running on a SPARC-compatible CPU'} Processing <Platform#None sparc64-sun-solaris> ... Committed <Platform#10 sparc64-sun-solaris> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 10L, 'name': 'sparc64-sun-solaris', 'user_friendly_name': 'Solaris 2.8 or later running on a SPARC 64-bit CPU'} Processing <Platform#None powerpc64-ps3-linux-gnu> ... Committed <Platform#11 powerpc64-ps3-linux-gnu> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 11L, 'name': 'powerpc64-ps3-linux-gnu', 'user_friendly_name': 'Sony Playstation 3 running Linux'} Processing <Platform#None anonymous> ... Committed <Platform#12 anonymous> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308990279L, 'deprecated': 0, 'id': 12L, 'name': 'anonymous', 'user_friendly_name': 'anonymous'} |
|
Line 197: | Line 111: |
/var/tmp/boinc/test2/py/Boinc/db_base.py:63: Warning: Field 'host_scale_check' doesn't have a default value | /var/tmp/boinc/boinctest/py/Boinc/db_base.py:63: Warning: Field 'host_scale_check' doesn't have a default value |
Line 199: | Line 113: |
Committed <App#1 upper_case> ; values: | Committed <App#11 upper_case> ; values: |
Line 203: | Line 117: |
'create_time': 1308990279L, | 'create_time': 1309737828L, |
Line 207: | Line 121: |
'id': 1L, | 'id': 11L, |
Line 215: | Line 129: |
The xadd tool will parse the project.xml and will put the configurations to the database. (can you explain what this means? why did the i686/amd64 entries not show anything?) Please also show the respective entries in the database, e.g. with {{{ echo 'select * from ... | mysql -u $host -p }}} sm) |
This is the output of xadd parsing a single platform specification and a single application, the actual output is much longer due the increased number of platforms. Also it should be noted that currently xadd has no provision to delete from databases, it always appends the entries to databse, if you want to remove/change existing entries, you should do it manually. And when executing that line again, nothing happens since everything here is already inside database, : {{{ # bin/xadd Processing <App#None upper_case> ... Skipped existing <App#None upper_case> }}} The file ''project.xml'' is not touched. The database has now seen the application and platforms: {{{ $ echo "select * from app" | mysql -u root -p $dbprojectname Enter password: id create_time name min_version deprecated user_friendly_name homogeneous_redundancy weight beta target_nresults min_avg_pfc host_scale_check 1 1308465648 upper_case 0 0 upperCASE 0 1 0 0 1 0 $ echo "select * from platform;"| mysql -u root -p test Enter password: id create_time name user_friendly_name deprecated 1 1308465648 i686-pc-linux-gnu Linux running on an Intel x86-compatible CPU 0 2 1308465648 x86_64-pc-linux-gnu Linux running on an AMD x86_64 or Intel EM64T CPU 0 }}} |
Line 228: | Line 162: |
(Is this a different upper_case folder? If not, where is the version of the upper_case/uppercase filenames? sm) {{{ ./bin/sign_executable apps/upper_case/upper_case_$(boincplat) code_sign_private_key_file te >> apps/upper_case/upper_case_$(boincplat).sig |
{{{ privateKeyfile="./keys/code_sign_private" if [ -z "$appver" -o -z "$boincplat" ]; then echo "Please set appver and boincplat variables from above." elif [ ! -r "$privateKeyfile" ]; then echo 'Have your private key ready as created during setup, expected at $privateKeyfile .' else sudo ./bin/sign_executable apps/upper_case/upper_case_${appver}_${boincplat} "$privateKeyfile" | sudo tee apps/upper_case/upper_case_${appver}_${boincplat}.sig fi |
Line 244: | Line 182: |
Found <App#1 upper_case> version 100 for <Platform#3 i686-pc-linux-gnu>: upper_case_1.00_i686-pc-linux-gnu Using signature file /var/tmp/boinc/test2/apps/upper_case/upper_case_1.00_i686-pc-linux-gnu.sig Copying upper_case_1.00_i686-pc-linux-gnu to /var/tmp/boinc/test2/download/upper_case_1.00_i686-pc-linux-gnu |
Toshiba:/var/tmp/boinc/boinctest# ./bin/update_versions Found <App#11 upper_case> version 612 for <Platform#2 x86_64-pc-linux-gnu>: upper_case_6.12_x86_64-pc-linux-gnu Using signature file /var/tmp/boinc/boinctest/apps/upper_case/upper_case_6.12_x86_64-pc-linux-gnu.sig Copying upper_case_6.12_x86_64-pc-linux-gnu to /var/tmp/boinc/boinctest/download/upper_case_6.12_x86_64-pc-linux-gnu |
Line 248: | Line 187: |
<AppVersion#None upper_case 100 i686-pc-linux-gnu> Continue [Y/n] |
<AppVersion#None upper_case 612 x86_64-pc-linux-gnu> Continue [Y/n] y |
Line 251: | Line 190: |
<AppVersion#1 upper_case 100 i686-pc-linux-gnu> | <AppVersion#1 upper_case 612 x86_64-pc-linux-gnu> |
Line 258: | Line 197: |
* Input file(s) * Work Unit template * Result Template Create a test input file, with some text in $installroot/$fileprojectname/download/in {{{ echo test string >> download/input_file |
* Input file(s) * Work Unit template * Result Template Create a test input file, with some text in $installroot/$fileprojectname/download/input_file {{{ $ echo test string >> download/input_file |
Line 271: | Line 210: |
% cp /usr/share/doc/boinc-server-maker/examples/upper_case_* $installroot/$fileprojectname/templates/ }}} and finally create the work, {{{ cd $installroot/$fileprojectname ./bin/create_work -appname upper_case -wu_name test -wu_template templates/upper_case_wu -result_temp |
$ cp /usr/share/doc/boinc-server-maker/examples/upper_case_* $installroot/$fileprojectname/templates/ }}} We will call the workunit test and to commit it to the databse, execute {{{ $ cd $installroot/$fileprojectname $ ./bin/create_work -appname upper_case -wu_name test -wu_template templates/upper_case_wu -result_temp |
Line 281: | Line 220: |
This will create one work unit named test in the database. | |
Line 288: | Line 226: |
./bin/start | $./bin/start Staying in ENABLED mode Starting daemons Starting daemon: feeder -d 3 Starting daemon: transitioner -d 3 Starting daemon: file_deleter -d 3 |
Line 292: | Line 235: |
{{{ $ ./bin/status BOINC is ENABLED DAEMON pid status lockfile disabled commandline 1 16574 running locked no feeder -d 3 2 16576 running locked no transitioner -d 3 3 16579 running locked no file_deleter -d 3 TASK last run period next run lock file disabled commandline 1 ? 24 hours NOW unlocked yes db_dump -d 2 -dump_spec ../db_dump_spec.xml 2 ? 1 days NOW unlocked yes run_in_ops ./update_uotd.php 3 ? 1 hour NOW unlocked yes run_in_ops ./update_forum_activities.php 4 ? 7 days NOW unlocked yes update_stats -update_users -update_teams -update_hosts 5 ? 24 hours NOW unlocked yes run_in_ops ./update_profile_pages.php 6 ? 24 hours NOW unlocked yes run_in_ops ./team_import.php 7 ? 24 hours NOW unlocked yes run_in_ops ./notify.php }}} |
THESE PAGES ARE STILL UNDER DEVELOPMENT
Contents
This page is about deploying a test application with BOINC/Debian, and this will replace add functionality section of the BOINC/ServerGuide page.
1. Add scientific applications to be distributed
The BOINC project managers need to provide all the binaries for all the supported platforms. This is of some difficulty especially for those platforms that one does not own oneself. This page first demonstrates the workflow using xadd for a single platform. The second half of this page is dedicated to employing the binaries Debian provides for the purpose.
Another page describes how to perform the equivalent installation for the Wrapper application.
1.1. Add a single example app for a single architecture to the BOINC project
1.1.1. Get binary of local platform
Install the application package, boinc-app-examples:
apt-get install boinc-app-examples
The example application used in the document is upper_case, which converts text inside a file to all capitals. Have a quick look that it is truly contained, since this walk-through may already be outdated (we are all volunteers):
$ dpkg -L boinc-app-examples | grep upper_case /usr/lib/boinc-server/apps/upper_case
1.1.2. Create a directory and add the app to project configuration.
This directory becomes an intrinsic part of your project.
[ -z "$installroot" -o -z "$fileprojectname" ] || . ~/.boinc_test.conf appdir="$installroot"/"$fileprojectname"/apps/upper_case sudo mkdir -p "$appdir"
Copy the file from the installed "boinc-app-examples" Debian package into that directory and rename it to distinguish versions and architectures. In our case, the app_ver variable is that of the BOINC server, the second part of the filename is that of the BOINC architecture.
appver=6.12 # adjust to the right version, only have single "." boincplat=$(arch)-pc-linux-gnu # adjust to your architecture, maybe i686-pc-linux-gnu sudo cp $(dpkg -L boinc-app-examples | grep upper_case) $appdir/upper_case_${appver}_${boincplat}
Upstream lists official BOINC architectures here.
Please keep the version formatted that simply - or change the BOINC source code.
1.2. Use the Debian-provided script to install binaries for multiple platforms
(D.: ... I am refering to /usr/share/doc/boinc-server-maker/examples/fetch_example_applications.sh , which is coming with the BOINC-server-maker package. Please amend that to fit the structure above described. sm)
When applications do not have dependencies non-standard dynamically loaded libraries (test with the tool 'ldd'), then one can use the regular binary from Debian. This should then be functional also for non-Debian/Ubuntu platforms. The boinc-server-maker package provides a shell script that downloads the Debian packages of a given name (the default is the boinc-app-examples package) and unpacks it to retrieve the binary from there.
To be implemented: this will create a structure as described above for the single application.
2. Inform local database of available binaries
Now add following lines to project.xml in the project root
[ -z "$installroot" -o -z "$fileprojectname" ] || . ~/.boinc_test.conf if [ -z "$installroot" -o -z "$fileprojectname" ]; then echo 'Variables $installroot (' $installroot ') and $fileprojectname (' $fileprojectname ') are both required.' elif [ -d "$installroot/$fileprojectname" ]; then (cat << EOPROJECTXML <boinc> <app> <name>upper_case</name> <user_friendly_name>upperCASE</user_friendly_name> </app> <platform> <name>i686-pc-linux-gnu</name> <user_friendly_name>Linux/x86</user_friendly_name> </platform> <platform> <name>x86_64-pc-linux-gnu</name> <user_friendly_name>Linux/amd64</user_friendly_name> </platform> </boinc> EOPROJECTXML ) | sudo tee "$installroot"/"$fileprojectname"/project.xml fi
The platforms need not be defined here, since the default project.xml contains entries for all platforms, They are shown here for the user to understand finer details, and having duplicate entries on database will generate errors. In most cases, you can just append the application entries to the default project.xml file and run xadd.
Change to the projectroot
cd "$installroot"/"$fileprojectname"
and run initiate the addition of the binary found in the directory structure to the local database
sudo bin/xadd
The local screen output will be similar to
Processing <Platform#None i686-pc-linux-gnu> ... Committed <Platform#3 i686-pc-linux-gnu> ; values: {'_dirty': False, '_lazy_lookups': {}, 'create_time': 1308988632L, 'deprecated': 0, 'id': 3L, 'name': 'i686-pc-linux-gnu', 'user_friendly_name': 'Linux running on an Intel x86-compatible CPU'} Processing <App#None upper_case> ... /var/tmp/boinc/boinctest/py/Boinc/db_base.py:63: Warning: Field 'host_scale_check' doesn't have a default value cursor.execute(command) Committed <App#11 upper_case> ; values: {'_dirty': False, '_lazy_lookups': {}, 'beta': 0, 'create_time': 1309737828L, 'deprecated': 0, 'homogeneous_redundancy': 0, 'host_scale_check': 0, 'id': 11L, 'min_avg_pfc': 1.0, 'min_version': 0L, 'name': 'upper_case', 'target_nresults': 0, 'user_friendly_name': 'upperCASE', 'weight': 1.0}
This is the output of xadd parsing a single platform specification and a single application, the actual output is much longer due the increased number of platforms. Also it should be noted that currently xadd has no provision to delete from databases, it always appends the entries to databse, if you want to remove/change existing entries, you should do it manually.
And when executing that line again, nothing happens since everything here is already inside database, :
# bin/xadd Processing <App#None upper_case> ... Skipped existing <App#None upper_case>
The file project.xml is not touched.
The database has now seen the application and platforms:
$ echo "select * from app" | mysql -u root -p $dbprojectname Enter password: id create_time name min_version deprecated user_friendly_name homogeneous_redundancy weight beta target_nresults min_avg_pfc host_scale_check 1 1308465648 upper_case 0 0 upperCASE 0 1 0 0 1 0 $ echo "select * from platform;"| mysql -u root -p test Enter password: id create_time name user_friendly_name deprecated 1 1308465648 i686-pc-linux-gnu Linux running on an Intel x86-compatible CPU 0 2 1308465648 x86_64-pc-linux-gnu Linux running on an AMD x86_64 or Intel EM64T CPU 0
3. Sign the application binary
BOINC need to sign the application binaries before dispatch for security reasons.
privateKeyfile="./keys/code_sign_private" if [ -z "$appver" -o -z "$boincplat" ]; then echo "Please set appver and boincplat variables from above." elif [ ! -r "$privateKeyfile" ]; then echo 'Have your private key ready as created during setup, expected at $privateKeyfile .' else sudo ./bin/sign_executable apps/upper_case/upper_case_${appver}_${boincplat} "$privateKeyfile" | sudo tee apps/upper_case/upper_case_${appver}_${boincplat}.sig fi
Update the boinc database,
./bin/update_versions
and prompt yes when asked for confirmation.
Sample output:
Toshiba:/var/tmp/boinc/boinctest# ./bin/update_versions Found <App#11 upper_case> version 612 for <Platform#2 x86_64-pc-linux-gnu>: upper_case_6.12_x86_64-pc-linux-gnu Using signature file /var/tmp/boinc/boinctest/apps/upper_case/upper_case_6.12_x86_64-pc-linux-gnu.sig Copying upper_case_6.12_x86_64-pc-linux-gnu to /var/tmp/boinc/boinctest/download/upper_case_6.12_x86_64-pc-linux-gnu Ready to commit 1 items: <AppVersion#None upper_case 612 x86_64-pc-linux-gnu> Continue [Y/n] y Committed: <AppVersion#1 upper_case 612 x86_64-pc-linux-gnu> Touched trigger file to make feeder re-read app_version table from database Done
4. Add a Work Unit
A work unit is the portion of data that the project should be analyzed, a work unit has following parts,
- Input file(s)
- Work Unit template
- Result Template
Create a test input file, with some text in $installroot/$fileprojectname/download/input_file
$ echo test string >> download/input_file
Copy the default work unit and result template for upper case application to the templates folder.
# The paths are to be changed in the next version of package, so watch out. $ cp /usr/share/doc/boinc-server-maker/examples/upper_case_* $installroot/$fileprojectname/templates/
We will call the workunit test and to commit it to the databse, execute
$ cd $installroot/$fileprojectname $ ./bin/create_work -appname upper_case -wu_name test -wu_template templates/upper_case_wu -result_temp late templates/upper_case_result input_file
5. Start the project.
Now its time to start the project, assuming we are still inside the project root,
$./bin/start Staying in ENABLED mode Starting daemons Starting daemon: feeder -d 3 Starting daemon: transitioner -d 3 Starting daemon: file_deleter -d 3
And ./bin/status can be used to check status of the project.
$ ./bin/status BOINC is ENABLED DAEMON pid status lockfile disabled commandline 1 16574 running locked no feeder -d 3 2 16576 running locked no transitioner -d 3 3 16579 running locked no file_deleter -d 3 TASK last run period next run lock file disabled commandline 1 ? 24 hours NOW unlocked yes db_dump -d 2 -dump_spec ../db_dump_spec.xml 2 ? 1 days NOW unlocked yes run_in_ops ./update_uotd.php 3 ? 1 hour NOW unlocked yes run_in_ops ./update_forum_activities.php 4 ? 7 days NOW unlocked yes update_stats -update_users -update_teams -update_hosts 5 ? 24 hours NOW unlocked yes run_in_ops ./update_profile_pages.php 6 ? 24 hours NOW unlocked yes run_in_ops ./team_import.php 7 ? 24 hours NOW unlocked yes run_in_ops ./notify.php