Out of Date : The svnbuildstat service described here is currently unavailable. The following is for reference/historical interest only.

Prerequisites: To get the most from this article, understand these concepts and terms before reading: basic Core Utilities, configuration files, INSERT YOUR PREREQUISITES HERE


Table of Contents


It would be really helpful if there would be a tool that does automatic building of Debian packages that are maintained either in Subversion, git, or just uploaded as a source package. The service should provide a Debian repository of these binary packages, all automatically built using a pbuilder, together with lintian and piuparts logs. Anyone registered could upload his source package (maybe only if he provides a build bot first). Just like Ubuntu PPA but for Debian and more integrated with Debian (alioth, BTS, svn.debian.org, git.debian.org, etc.).

This would be helpful for both team and individual maintainance and simply make it easier for new Debian maintainers to create their first packages and make sure they are high quality and also to provide a means to anyone to just fix a package, upload it to svnbuildstat and allow the problem to be fixed not only to him, but to anyone adding the svnbuildstat package repository to his sources.list. The svnbuildstat service should also create a simple patch against the official package in unstable, so that the one, who fixes the problem is encouraged to send the patch to the respective maintainer.

This project has arised from the need to have some automated checks done on the Debian Games Team's packages which are team maintained in a Subversion repository.




The source is available in the Debian QA Subversion repository. It can be checked out

svn co svn://svn.debian.org/svn/collab-qa/svnbuildstat

Source structure

Svnbp uses these scripts to feed/refresh the database:

 */30 *    * * * root /home/sites/svnbuildstat.debian.net/svnbuildstat/script/svnbuildstat_update-db.pl > /dev/null 2>&1
 5 *    * * * root /home/sites/svnbuildstat.debian.net/svnbuildstat/script/svnbuildstat_update-bugs.pl > /dev/null 2>&1
 * *    * * * root /home/sites/svnbuildstat.debian.net/svnbuildstat/script/svnbuildstat_update-repository.pl > /dev/null 2>&1
 * *    * * * root /home/sites/svnbuildstat.debian.net/svnbuildstat/script/svnbuildstat_import-new-report.pl > /dev/null 2>&1
 0 0    * * * root /home/sites/svnbuildstat.debian.net/svnbuildstat/script/svnbuildstat_purge-buildinprogress.pl > /dev/null 2>&1

TODO list from the next release



How to get it working

the server

Install prerequisites (Etch):

aptitude install libcatalyst-perl libcatalyst-modules-perl libdbd-pg-perl postgresql-8.1 subversion libfile-touch-perl dpkg-dev

Install svnbuildstat: Today some paths are still hardcoded and we strongly advise you to use the /home/sites/svnbuildstat.debian.net directory as root directory:

sudo mkdir -p /home/sites/svnbuildstat.debian.net
sudo chown username:usergroup /home/sites/svnbuildstat.debian.net
cd /home/sites/svnbuildstat.debian.net
svn co svn://svn.debian.org/svn/collab-qa/svnbuildstat/branches/stable svnbuildstat
cd svnbuildstat
vim lib/SvnBuildStat/Config.pm

If you don't want to use the standard path for the config file (/etc/svnbuildstat.conf), edit the lines 7, 8:

$file = "./svnbuildstat.conf";

create the database (enter the password you put in ./svnbuildstat.conf):

$ sudo su postgres
$ createuser -P svnbuildstat
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n
$ createdb -O svnbuildstat svnbuildstat -E UTF8
$ psql svnbuildstat
svnbuildstat=# \i schema-pg.sql

If you prefere to have an initialised database, complet dump can be downloaded here: http://nana.rulezlan.org/~goneri/svnbuildstat/

You also need to setup the work directory:

svnbuildstat_update-repository.pl and svnbuildstat_import-new-report.pl need both to have work directory
svnbuildstat_import-new-report.pl use the setting from the [report] section
whereas svnbuildstat_import-new-report.pl work directory is hardcoded at the begining of the script (be careful, it plays with rm -Rf)
svnbuildstat_update-repository.pl is the last script I did and I still don't have clean up it
It will use the svnbuildstat.conf too
you'll probably find some new missing dependency with these scripts

run the svnbuildstat:

$ script/svnbuildstat_www_server.pl
DBIx::Class::Schema::Loader::connection(): viewchangelog has no primary key at /usr/share/perl5/Catalyst/Model/DBIC/Schema.pm line 292
DBIx::Class::Schema::Loader::connection(): viewpackage has no primary key at /usr/share/perl5/Catalyst/Model/DBIC/Schema.pm line 292
DBIx::Class::Schema::Loader::connection(): viewtobuild has no primary key at /usr/share/perl5/Catalyst/Model/DBIC/Schema.pm line 292
DBIx::Class::Schema::Loader::connection(): viewtobuild2 has no primary key at /usr/share/perl5/Catalyst/Model/DBIC/Schema.pm line 292
You can connect to your server at http://fuji:3000

Point your browser to http://localhost:3000 and enjoy. :)

the agent

Install prerequisites:

aptitude install libwww-perl

development version

To download development version, use:

svn co svn://svn.debian.org/svn/collab-qa/svnbuildstat/trunk svnbuildstat

Development debian repository:

deb     http://debppa.certik.cz/debian/ unstable main contrib non-free
deb-src http://debppa.certik.cz/debian/ unstable main contrib non-free

New dependencies:

aptitude install libcatalyst-perl libcatalyst-modules-perl libdbd-pg-perl postgresql-8.3 subversion libfile-touch-perl dpkg-dev  libconfig-inifiles-perl devscripts fakeroot
liblogger-syslog-perl (not in Etch but the one from Lenny works fine after a rebuild)

The current devel schema http://nana.rulezlan.org/~goneri/svnbuildstat_devel_db_schema.png

To refresh it:

postgresql_autodoc -d svnbuildstat -t dot
dot -Tpng svnbuildstat.dot > svnbuildstat_devel_db_schema.png

server/builbot communication

The build bot send a ini file to the server through a HTTP POST. The contains the arch of the buildbot. The server reply by sending the same ini file but with a build id and the URL to the dsc file to buid. Once the build is finish the buildbot send back the ini file and the build result. Take a look on getBuildJob() and postFile() in svnbuildstat_agent.pl.