Differences between revisions 74 and 75
Revision 74 as of 2009-05-04 04:38:34
Size: 16802
Editor: BenFinney
Comment: metrics dependent on the presence of a ‘debian/watch’ file
Revision 75 as of 2009-05-04 04:48:13
Size: 16952
Editor: BenFinney
Comment: restructure metrics
Deletions are marked like this. Additions are marked like this.
Line 196: Line 196:
 * package builds or FTBFS in sid on X architectures
  * including unofficial arches and with double builds, /bin/sh -> dash, -j2, DPKG_GENSYMBOLS_CHECK_LEVEL=4 etc
 * source package produces X lintian errors, Y lintian warnings, Z lintian infos, uses xx overrides
 * binary package produces X lintian errors, Y lintian warnings, Z lintian infos, uses xx overrides
 * orig.tar.gz matches the tarball inside upstream's orig.tar.bz2/lzma/xz (this will be obsoleted by dpkg-source v3)
 * orig.tar.gz contents matches the contents of upstreams zip/etc
 * source package contains a ‘debian/watch’ file
  * ‘uscan’ gives no warnings or errors
  * ‘uscan --report’ reports the package is up-to-date
  * ‘uscan --download’ generates an original upstream source that matches the one provided in the source package
 * if there is no ‘debian/watch’ file, is there a README.Source file or a get-orig-source target
 * total size of the source package
 * size of the orig.tar.gz/diff.gz
==== Package maintainer ====
Line 217: Line 206:
 * maintainer has >1, >5, >10, >20, >100 packages in Debian already
 * has the maintainer read SC/DFSG? policy? devref? maint-guide? mentors FAQ?

==== Package metadata ====
Line 221: Line 215:
 * maintainer has >1, >5, >10, >20, >100 packages in Debian already
Line 240: Line 233:
 * is it a native package
 * is the package in main? contrib? non-free?
 * what section is the package in? (mail/python/base/etc)
 * are the build-deps/deps/recommends/suggests transitional packages?
 * are the build-deps/deps/recommends/suggests satisfiable?
 * are all the build-deps/deps/recommends/suggests alternatives satisfiable?
 * are any versioned dependencies asking for versions older than oldstable?
 * are there any dependencies on transition or dummy packages?

==== Source package ====

 * package builds or FTBFS in sid on X architectures
   * including unofficial arches and with double builds, /bin/sh -> dash, -j2, DPKG_GENSYMBOLS_CHECK_LEVEL=4 etc
 * source package produces X lintian errors, Y lintian warnings, Z lintian infos, uses xx overrides
 * binary package produces X lintian errors, Y lintian warnings, Z lintian infos, uses xx overrides
 * orig.tar.gz matches the tarball inside upstream's orig.tar.bz2/lzma/xz (this will be obsoleted by dpkg-source v3)
 * orig.tar.gz contents matches the contents of upstreams zip/etc
 * source package contains a ‘debian/watch’ file
   * ‘uscan’ gives no warnings or errors
   * ‘uscan --report’ reports the package is up-to-date
   * ‘uscan --download’ generates an original upstream source that matches the one provided in the source package
 * if there is no ‘debian/watch’ file, is there a README.Source file or a get-orig-source target
 * total size of the source package
 * size of the orig.tar.gz/diff.gz
Line 241: Line 258:
 * is it a native package
Line 244: Line 260:
 * is the package in main? contrib? non-free?
 * what section is the package in? (mail/python/base/etc)
Line 255: Line 269:
 * are the build-deps/deps/recommends/suggests transitional packages?
 * are the build-deps/deps/recommends/suggests satisfiable?
 * are all the build-deps/deps/recommends/suggests alternatives satisfiable?
 * are any versioned dependencies asking for versions older than oldstable?
 * are there any dependencies on transition or dummy packages?
Line 262: Line 271:
 * what does the licensecheck output look like?

==== Resulting binary package ====
Line 271: Line 284:
 * has the maintainer read SC/DFSG? policy? devref? maint-guide? mentors FAQ?
 * what does the licensecheck output look like?
Line 277: Line 288:
  * SponsorChecklist
  * http://ftp-master.debian.org/REJECT-FAQ.html
  * [[qa.debian.org/bapase]]
  * http://www.debian.org/doc/debian-policy/
  * http://www.debian.org/doc/developers-reference/
  * SponsorChecklist
  * http://ftp-master.debian.org/REJECT-FAQ.html
   * [[qa.debian.org/bapase]]
   * http://www.debian.org/doc/debian-policy/
   * http://www.debian.org/doc/developers-reference/
Line 283: Line 294:
  * https://fedoraproject.org/wiki/Packaging/Guidelines
  * https://fedoraproject.org/wiki/Packaging/ReviewGuidelines
  * http://gauret.free.fr/fichiers/rpms/fedora/fedora-qa
  * https://fedoraproject.org/wiki/Packaging/Guidelines
  * https://fedoraproject.org/wiki/Packaging/ReviewGuidelines
   * http://gauret.free.fr/fichiers/rpms/fedora/fedora-qa
Line 287: Line 298:
  * http://gpnl.org/
  * http://www.gentoo.org/proj/en/qa/
  * http://gpnl.org/
  * http://www.gentoo.org/proj/en/qa/
Line 290: Line 301:
  * http://cpants.perl.org/kwalitee.html   * http://cpants.perl.org/kwalitee.html
Line 292: Line 303:
  * http://wiki.php.net/pear/qa/ci   * http://wiki.php.net/pear/qa/ci

The software that will be used is debexpo:

http://debexpo.workaround.org/

mentors.debian.net Todo/Feature list

The mentors.debian.net service is being partly refactored. This page is supposed to collect ideas and features.

Since there is a certain need for software like PPA (personal package archive) the new software will probably be generic and rather serve as a basis for the new site.

Feature list

(this list may not cover all existing features)

General ideas

  • no binary packages (it is a developer service and we don't want complaints from end-users) by default
    • might want to relax this to allow personal package archives and also lintian checks on the resulting binaries? Alternatively, allow binaries to be uploaded, run lintian, store the lintian results and then remove the binaries.
  • Emphasis on social interaction/networking
  • More interactivity - AJAX/AHAH - Web 3.0 ;)

  • Better looks of the site (steal from http://launchpad.net, http://xing.com, http://debian.wgdd.de/debian/, www.haveamint.com/)

  • User accounts for Debian developers/sponsors, too. For subscription of packages (e.g. sponsors would get automatic notifications of new uploads).
  • Proper URLs (e.g. http://mentors.debian.net/packages/cream instead of http://mentors.debian.net/cgi-bin/view_packages?package=cream) - RESTful

  • Making information available to others through proper interfaces (JSON, XML, SOAP, whatever) - might help sponsors.debian.net - and document that interface properly ("mentors API")
  • Getting BTS information through its LDAP (or SOAP) interface
  • More icons for better overview (http://www.famfamfam.com/lab/icons/silk/)

  • Clear start page with big buttons telling the purpose of the site (similar to http://flickr.com)

  • Syndication feed (preferably ?AtomPub http://atompub.org/) on new uploads

  • search functions (sponsors, sponsorees, packages) - search by Programming Language, build tool (CDBS), bugs closed (RC).
  • run the new site in parallel to the old site as a beta service (http://mentors.debian.net:5000)

  • make the Python code available (careful not to publish passwords in the repository)
  • proper documentation of modules, URLs, database schema in restructured-text format within the repository
  • make it generic so that other people can setup their own private repository servers
  • have a plugin architecture (to allow certain scripts or functions being executed at various times like when a package is uploaded, after it's checked, when it's sponsored, when someone leaves a comment etc.). That should allow people to extend the application in case they want a buildd-like function for automated binary package building.
  • for each package, produce a list of metrics. for each sponsor, have them rate the importance of the metrics to them. when the sponsor logs in, order the list of packages by score, which is based on the sum of importances of the metrics. allow sponsors to hide packages with low score. allow sponsees to see how different sponsors have scored their packages.
  • sponsoring queues for each of the large teams in Debian?
  • integrate svnbuildstat functionality?
  • merge REVU and mentors into one site?
  • expire old packages after X months and notify the maintainer that they should put more effort into finding a sponsor
  • ability to deal with VCS repositories in addition to uploading source packages
  • steal some features from http://fedoraproject.org/wiki/Features/ReviewOMatic

Proposed tools, modules and languages

  • Language: Python 2.4
  • Web framework: Pylons 0.9.6

  • Javascript library: jQuery

  • Database: PostgreSQL 8.1
  • Database layer/object-relational-mapper: SQLAlchemy 0.4

Assets (Database tables)

  • Users
    • package maintainers/sponsorees
    • Debian developers/sponsors
    • Database fields
      • Index
      • Real name
      • Email address (unique but can be updated)
      • PGP key (currently used for authenticating uploads)
      • IRC nickname on OFTC (so that users can be informed online if they wish)
      • Password (MD5 hashed)
  • User-Ratings (let other users voice their opinion on a certain maintainer)
    • Index
    • Timestamp
    • ID of the user this rating belongs to
    • ID of the submitter of this rating
    • Text
  • Files
    • Possible files are .dsc, .orig.tar.gz, .tar.gz, .diff.gz
    • These are the files in the actual repository that are stored in a diretory tree on disk
    • Database fields
      • Index
      • Timestamp (when the file was placed in the repository)
      • Filename
      • Size
      • MD5 checksum
      • Package ID the file belongs to
  • Packages
    • Index
    • User ID the package belongs to
    • Name (could also be used as the directory name in the repository tree)
    • Description
    • Programming Language
    • QA Warnings (lintian errors, piuparts errors, native package, etc.)
    • Closes bugs (list of bugs to be closed by the upload)
    • Logfile (textual output from the import process to debug problems)
    • Watch counter (how many times the package has been looked at)
    • Download counter (how many times the .dsc file has been downloaded)
    • Seek Sponsor Flag (set to "true" if the package needs a sponsor - otherwise a sponsor is found already)
    • Announced IRC Flag (will be set to "true" to mark that the upload has been announced on #debian-mentors)
    • Comment (can be set by user - e.g. "Security update" or "Replaces package foobar")
  • Package-History
    • Index
    • Timestamp
    • Type (set of 'newupload', 'sponsored', 'deleted', 'comment', 'seekingsponsor', etc.)
    • Text (the actual content of the history entry)
  • Package-Comments (let other users voice their opinion on a certain package)
    • Index
    • Timestamp
    • ID of the package this rating belongs to
    • ID of the submitter of this rating
    • Thumbs (thumbs up or thumbs down)
    • Text

Uploading/Importer

  • QA tests
    • syntax check of *.changes file
    • check if package is properly signed
    • check if maintainer is listed as registered user in the database
    • build test (perhaps through pbuilder - do we really need to spend so many resources? is it the scope of mentors?)
    • piuparts test (can we handle that with the resources?)
    • lintian checks (backport of lintian)
    • do the closed bugs belong to the package? (some packages accidentally closed other packages' bugs)
    • native package check
  • Uploads are done via HTTP with authentication (email address + password) so they can be received by the web framework (instead of an FTP server)
  • Removed files from previous package versions (only one version is kept)
  • Inform people who subscribed to the package
  • Pull orig.tar.gz from official Debian repositories if not included in the upload (is that really useful?)
  • maintainers should be forced to increase the package version/revision number to avoid confusion when different package contents had the same version/revision number (yes, this will be confusing at first)

Packages

  • Package list (http://mentors.debian.net/packages)

    • pagination (not displaying everything on the same page)
    • allow filtering (e.g. only show packages needing a sponsor)
  • Per-package page (http://mentors.debian.net/packages/foobar)

    • Closed bugs (clicking on a bug opens a popup getting information from the BTS over LDAP - ldap://bts2ldap.debian.net:10101 or SOAP)
    • Other packages from this maintainer
    • History
  • History (keep a history of uploads, comments, changes)
    • history is kept for all eternity - even for packages that were deleted already
  • Comments
    • if done comfortably this could replace the RFS-threads on the debian-mentors mailing list (duplication? some sponsors may prefer to retain RFS threads on the mailing list).
  • Check if the package is already in Debian
  • Creating RFS boilerplates (filling out as much information as possible with strong emphasis on maintainers adding meaningful content and not blindly filling in the fields)
  • Is there a new upstream version?
  • Popcon score
  • How long has the package been waiting for a sponsor?
  • display information on copyright, upstream homepage if possible

Maintainers/sponsorees

  • Information page
    • which packages does the sponsor have online
  • allow changing GPG key

Sponsors/DDs

  • Let DDs/sponsors register/have accounts, too
  • List of preferences (programming langue, CDBS/debhelper, ...) so that maintainers can try to find a matching sponsor

Repository access

  • don't store the packages in an Apache-index-style directory tree but serve the files through ?FileApp (for more flexibility)

    • (codehelp: I think that would be a mistake. Index style directories are more suitable for some sponsors.)
  • count downloads for statistics ("your package 'foobar' has been downloaded 23 times") (Nice to have but not essential.)
  • does anyone use mentors as a "deb-src" repository? or do we all use "dget -x"?

Background magic

  • monitor debian-devel-changes mailing list to see if packages were sponsored already
  • check if newer packages are availible in the Debian repository
  • announce new uploads on IRC (#debian-mentors on OFTC) - perhaps even /msg sponsors if they are online (codehelp: No thanks. email notifications are sufficient, sponsors are not at the beck and call of maintainers.)
  • warn maintainer about old packages (older than 3 months) / expire old packages (older than 6 months)

Metrics

Stuff that you would like to see checked automatically (or based on info provided by the maintainer) so you could use it to prioritise packages.

Package maintainer

  • maintainer is in NM
  • maintainer is willing to enter NM
  • approx date maintainer is planning to enter NM
  • maintainer is a DM
  • maintainer is willing to become a DM
  • approx date maintainer is planning to apply for DM
  • city maintainer lives in? (useful for determining if you can meet up)
  • timezone maintainer lives in?
  • maintainer has >1, >5, >10, >20, >100 packages in Debian already

  • has the maintainer read SC/DFSG? policy? devref? maint-guide? mentors FAQ?

Package metadata

  • this is a QA upload/NMU
  • is this an adopted package?
  • is this a hijacked package?
  • if this is an adopted/hijacked package, what is the rationale for that?
  • is this package available elsewhere (MergeDerivedDistributions/whohas)? what is the bugs/popcon/patches there?

  • this is a new package / this is an update to an existing package
  • if this is a new package, does it close an ITP or other wnpp bug
  • if this is a new package, how long has it been waiting to be uploaded
  • if this is a new package, does it have a history of or current security issues (CVE ids in security tracker)
  • if this is an updated package, what percentage/number of RC/important/normal/wishlist bugs does it close.
  • if this is an updated package, how long between the previous upload and this one
  • if this is an updated package, what proportion of existing lintian errors does it fix
  • if this is an updated package, who else have sponsored this package
  • if this is an updated package, does it have enough debtags?
  • if this is an updated package, and the dependencies or debtags indicate some kind of UI, does it have a up-to-date screenshot (latest version available in the archive)?
  • if this is an updated package, how popular is it according to Debian popcon? Ubuntu popcon?
  • if this is an updated package, does it fail piuparts tests (for stable2unstable, testing2unstable, unstable, unstable2experimental)
  • if this is an updated package, is the version the same in testing/unstable? if not, what are the excuses?
  • number of upload attempts for this version of this source package
  • is there a homepage field in the control file and is it in the Source: stanza?
  • does the homepage and all the URLs in the Description give a HTTP 200 OK?
  • are there Vcs-* fields in the control file
  • is it a native package
  • is the package in main? contrib? non-free?
  • what section is the package in? (mail/python/base/etc)
  • are the build-deps/deps/recommends/suggests transitional packages?
  • are the build-deps/deps/recommends/suggests satisfiable?
  • are all the build-deps/deps/recommends/suggests alternatives satisfiable?
  • are any versioned dependencies asking for versions older than oldstable?
  • are there any dependencies on transition or dummy packages?

Source package

  • package builds or FTBFS in sid on X architectures
    • including unofficial arches and with double builds, /bin/sh -> dash, -j2, DPKG_GENSYMBOLS_CHECK_LEVEL=4 etc

  • source package produces X lintian errors, Y lintian warnings, Z lintian infos, uses xx overrides
  • binary package produces X lintian errors, Y lintian warnings, Z lintian infos, uses xx overrides
  • orig.tar.gz matches the tarball inside upstream's orig.tar.bz2/lzma/xz (this will be obsoleted by dpkg-source v3)
  • orig.tar.gz contents matches the contents of upstreams zip/etc
  • source package contains a ‘debian/watch’ file
    • ‘uscan’ gives no warnings or errors
    • ‘uscan --report’ reports the package is up-to-date
    • ‘uscan --download’ generates an original upstream source that matches the one provided in the source package
  • if there is no ‘debian/watch’ file, is there a README.Source file or a get-orig-source target
  • total size of the source package
  • size of the orig.tar.gz/diff.gz
  • is the diff.gz clean (only changes files in debian/)
  • does it require the -v option to debuild/dpkg-buildpackage?
  • does it use cdbs? or debhelper? or manoj-cdbs? or xsf-cdbs? or nothing?
  • are any files outside debian/ modified?
    • if so: how files many are modified?
    • if not: is a common patch system used?
      • if so: how many patches are there?
      • if so: which patch system is used? quilt? dpatch? cdbs-simplepatchsys? dbs?
      • if not: is there a patches/ dir, or are there any .diff or .patch files in debian/?
  • build logs:
    • number of warnings from gcc/python/etc
    • number of warnings corresponding to security or portability issues.
  • do any static analysis tools find issues? how many? (for eg: rats, flawfinder, cppcheck, splint)
  • code size (for people who audit) using wc -l? using sloccount?
  • what does the licensecheck output look like?

Resulting binary package

  • if it is a GUI/console app, does it support i18n?
  • do input fuzzers affect it? (for eg: zzuf, fusil, fuzz, inguma, netsed, wapiti)
  • do any of the files conflict with an existing package? is there a Conflicts field?
  • all the lintian wontfix bugs (inc archived versions): http://bugs.debian.org/cgi-bin/pkgreport.cgi?archive=both;include=tags%3Awontfix;package=lintian

  • if this is a CPAN module, what is the Kwalitee level? >X? >Y? >Z?

  • if the package contains a ttf/otf font, what is the output of ftinfo, fontlint, ftvalid, ttfcoverage, otfinfo?
  • if the package contains a font, what does the automated font sample image look like?
  • if the package contains library, is there an ABI breakage or unnecessary ABI bump compared to the archive?
  • if the package contains a FreeDesktop menu file, what is the output of desktop-file-validate?

More ideas

Developers

This is a list of people involved in the 3.0 relaunch of the mentors site:

  • Christoph Haas
    Been there since the start. Founded the service years ago with Ivo Marino, Christoph Siess and D.K.Gebhart. Claims to always have done most of the work. Currently the only one working on the site, fixing bugs and doing user support

(add your name here)

The git repository is available. Public: read-only. Developers: full read-write with username/password.

Include this

Some documents from the repository need to be included in the above list.