Differences between revisions 71 and 91 (spanning 20 versions)
Revision 71 as of 2013-01-26 20:01:08
Size: 6442
Editor: ?IvoDeDecker
Comment:
Revision 91 as of 2014-05-20 11:15:03
Size: 1438
Editor: HolgerLevsen
Comment: move to piuparts/piuparts.debian.org
Deletions are marked like this. Additions are marked like this.
Line 8: Line 8:
Description: .deb package installation, upgrading, and removal testing tool Description: .deb package installation, upgrading, and removal testing tool
 piuparts tests that .deb packages (as used by Debian) handle
 installation, upgrading, and removal correctly. It does this by
 creating a minimal Debian installation in a chroot, and installing,
 upgrading, and removing packages in that environment, and comparing the
 state of the directory tree before and after. piuparts reports any
 files that have been added, removed, or modified during this process.
 .
 piuparts is meant as a quality assurance tool for people who create .deb
 packages to test them before they upload them to the Debian package archive.
Homepage: http://piuparts.debian.org
Line 11: Line 21:
piuparts is being run constantly on the main archive, the results are public on [[http://piuparts.debian.org|piuparts.debian.org]]. piuparts itself has a [[http://piuparts.debian.org/doc/README.html|README]] and a [[http://piuparts.debian.org/doc/piuparts.1.html|piuparts manpage]], for the piuparts.debian.org setup there is also an [[http://wiki.debian.org/piuparts/FAQ|FAQ]] available. Below is an overview of information not covered elsewhere. piuparts is being run constantly on all suites of the main Debian archive, the results are made public on [[http://piuparts.debian.org|piuparts.debian.org]].
Line 13: Line 23:
= Developing piuparts = piuparts itself has a [[http://piuparts.debian.org/doc/README_1st.html|README_1st]] and a [[http://piuparts.debian.org/doc/piuparts.1.html|manpage]], for the piuparts.debian.org setup there is also an [[http://wiki.debian.org/piuparts/FAQ|FAQ]] available.
Line 15: Line 25:
 * alioth project: [[https://alioth.debian.org/projects/piuparts/]]
  * source code: Browse: [[https://alioth.debian.org/scm/browser.php?group_id=30965]] Get: `git clone git://anonscm.debian.org/piuparts/piuparts.git`
  * mailing list: [[http://lists.alioth.debian.org/mailman/listinfo/piuparts-devel|piuparts-devel@l.a.d.o]]
  * commit diff list: [[http://lists.alioth.debian.org/mailman/listinfo/piuparts-commits|piuparts-commits@l.a.d.o]]
 * todo
  * codebase related: [[http://anonscm.debian.org/gitweb/?p=piuparts/piuparts.git;a=blob;f=TODO;hb=HEAD]]
  * piuparts.debian.org related: [[http://anonscm.debian.org/gitweb/?p=piuparts/piatti.git;a=blob;f=README.txt;hb=HEAD]]
 * there was a [[SummerOfCode2007/PiupartsImprovements]]
 * [[http://www.ohloh.net/p/9038|code analysis]] and more on ohloh

= Developing piuparts, the rewrite of version 2 =

 * stalled at planning stage
 * in bzr: bzr get http://code.liw.fi/piuparts2/bzr/trunk/ (may move to bzr.debian.org later)

== From the BoF at Debconf9 ==

 * new output format: http://lists.debian.org/debian-devel/2009/07/msg00662.html
 * piuparts should not fail just because dependencies fail

= How piuparts is setup on piuparts.debian.org =

Running piuparts is the boring, automated part and is supervised by Luk and Holger, who can su into the piupartss (slave) user.
piuparts-slave is run in a shared screen session and seldom needs interaction. The piupartss user needs to be able to run piuparts and thus can use sudo without restrictions.

The slave automatically starts the master, to reserve packages to process and to pass logfiles to the master.

Ideally all members of the Debian-QA group can su to the piupartsm user and process the logfiles - currently it's again only Luk and Holger. piupartsm has no special privileges. If you want to help, please join #debian-qa and say so there.

Once a day piuparts-reports is run as piupartsm by cron to generate the html output of the logs.

For more information see [[http://anonscm.debian.org/gitweb/?p=piuparts/piatti.git;a=blob;f=README.txt;hb=HEAD]]

== Machine info ==

 piuparts.cs.helsinki.fi aka [[http://db.debian.org/machines.cgi?host=piatti|piatti.debian.org]] aka piuparts.debian.org
 * hosted at [[http://cs.helsinki.fi/index.en.html|Department of Computer Science, University of Helsinki]]
 * donated by [[http://hp.com/go/debian/|HP]]
 * dual AMD Opteron(tm) Processor 275, 2.2 Ghz
 * 8GB RAM
 * 300 GB physical volume for data
 * currently capable of testing sid in 3 days and squeeze in 12. using schroot and lvm snaspshots this can be improved, but filing bugs and adding more tests (see the bts...) is more important and interesting today for me.

= Howto setup a piuparts test-instance for development =


*most of this is still to be written*

TODO these scripts should really be converted to a single script that can do all the different runs (at the same time, if needed).

== Setup a private repo ==

To be able to run piuparts on packages outside the archive, they need to be in a repository, which is available via http.

A private repo can be created with reprepro. TODO how

== Create pbuilder tarballs ==

This step cna be skipped, but if you create tarballs with pbuilder first, creating the initial chroot is faster, because there is no need for a debootstrap each time.

For each distro (eg wheezy) and each arch (eg amd64), create a configfile.

/etc/pbuilder/pbuilderrc.i386-wheezy:

{{{
ARCHITECTURE=i386
DISTRIBUTION=wheezy
BASETGZ=/var/cache/pbuilder/base-$DISTRIBUTION-$ARCHITECTURE.tgz
BUILDRESULT=/var/cache/pbuilder/result-$DISTRIBUTION-$ARCHITECTURE

MIRRORSITE=http://ftp.de.debian.org/debian

PKGNAME_LOGFILE_EXTENTION=.buidlog
}}}

Then just run

{{{
pbuilder --create --configfile /etc/pbuilder/pbuilderrc.i386-wheezy
}}}

== run install test ==


== run upgrade test squeeze2wheezy ==

In this example, /ram is a ramdisk which is large enough to do the piuparts run.


run_piuparts_squeeze2wheezy:

{{{#!

#! /bin/bash

PKG=$1
LOCALREPO=http://localhost/debian

piuparts \
 --tmpdir=/ram \
 --warn-on-others \
 --no-eatmydata \
 --warn-on-leftovers-after-purge \
 --skip-logrotatefiles-test \
 --scriptsdir /etc/piuparts/scripts \
 -d squeeze \
 -d wheezy \
 -m http://ftp.de.debian.org/debian/ \
 -m ${LOCALREPO} \
 --do-not-verify-signatures \
 -b /var/cache/pbuilder/base-squeeze-i386.tgz \
 --apt \
 $PKG | tee ${PKG}.`date +%s`.log

}}}


== run upgrade test lenny2squeeze2wheezy ==

As lenny isn't available on mirrors anymore, it needs to be downloaded from archive.debian.org. Currently, piuparts doesn't really have a nice way to deal with this (see bug #699028), but this patch works for now:

[[http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=699028#5]]

run run_piuparts_lenny2squeeze2wheezy:

{{{#!
#! /bin/bash

PKG=$1
LOCALREPO=http://localhost/debian
PIUPARTSDIR=/path/to/piuparts/dir

${PIUPARTSDIR}/piuparts.lenny \
 --tmpdir=/ram \
 --warn-on-others \
 --no-eatmydata \
 --warn-on-leftovers-after-purge \
 --skip-logrotatefiles-test \
 --scriptsdir /etc/piuparts/scripts \
 --dpkg-noforce-unsafe-io \
 --no-eatmydata \
 -d lenny \
 -d squeeze \
 -d wheezy \
 -m http://ftp.de.debian.org/debian/ \
 -m ${LOCALREPO} \
 --do-not-verify-signatures \
 -b /var/cache/pbuilder/base-lenny-i386.tgz \
 --apt \
 $PKG | tee ${PKG}.`date +%s`.log

}}}


== run custom scripts ==

TODO:

 * examples from scripts that are used in the piuparts runs on the archive
  * install some extra package before the piuparts run
  * others?
 * example scripts to display extra info in the log
= Advanced usage tips for piuparts =
 * This obviously implies you are familar with the manpage, README_1st and README_server :-D
 * [[piuparts/UseCases]]

Overview about piuparts

Description: .deb package installation, upgrading, and removal testing tool 
 piuparts tests that .deb packages (as used by Debian) handle
 installation, upgrading, and removal correctly. It does this by
 creating a minimal Debian installation in a chroot, and installing,
 upgrading, and removing packages in that environment, and comparing the
 state of the directory tree before and after. piuparts reports any
 files that have been added, removed, or modified during this process.
 .
 piuparts is meant as a quality assurance tool for people who create .deb
 packages to test them before they upload them to the Debian package archive.
Homepage: http://piuparts.debian.org

piuparts is being run constantly on all suites of the main Debian archive, the results are made public on piuparts.debian.org.

piuparts itself has a README_1st and a manpage, for the piuparts.debian.org setup there is also an FAQ available.

Advanced usage tips for piuparts

  • This obviously implies you are familar with the manpage, README_1st and README_server :-D
  • piuparts/UseCases


CategoryPermalink