Differences between revisions 120 and 121
Revision 120 as of 2012-09-23 10:35:03
Size: 20417
Editor: PaulWise
Comment:
Revision 121 as of 2017-05-23 08:43:47
Size: 20392
Editor: PaulWise
Comment: DMUA is gone
Deletions are marked like this. Additions are marked like this.
Line 276: Line 276:
 * if the package contains {{{DM-Upload-Allowed}}}, are there any [[DebianMaintainer|DMs]] amongst the {{{Maintainer}}}/{{{Uploader}}}s?  * are the [[DebianMaintainer|DMs]] in [[dm.txt]] for the package amongst the {{{Maintainer}}}/{{{Uploader}}}s?

The software that will be used is Debexpo.

expo.debian.net current status

  • Deployed on personal machine owned by Asheesh Laroia <paulproteus@debian.org>

  • To SSH in:
    • ssh debexpo@expo.debian.net

    • fingerprint: 2048 27:6b:1c:6e:b6:0e:64:ed:c8:4a:05:7e:b6:8a:ed:5c /etc/ssh/ssh_host_rsa_key.pub (RSA)
  • If you want permission to SSH in, send a PGP-signed email to paulproteus@debian.org with your key.

    • If you don't have a PGP key in the web of trust, email paulproteus@debian.org and I will reply with your key. Reply again to me, and I'll consider it trusted.

Things that seem to work:

  • Uploading packages.
  • ?

Notes:

  • The code lives in /home/debexpo/debexpo/
  • When you modify the code or configuration, do: touch deploy.wsgi. That causes Apache to reload the Python processes.

mentors.debian.net Todo/Feature list

The mentors.debian.net service is being partly refactored. This page is supposed to collect ideas and features.

Since there is a certain need for software like PPA (personal package archive) the new software will probably be generic and rather serve as a basis for the new site.

Ideas from lists

Threads with some wishlists or discussion about mentors.d.n:

http://lists.debian.org/1276121151.32039.6.camel@PC-Lorenzo

Feature list

(this list may not cover all existing features)

General ideas

  • no binary packages (it is a developer service and we don't want complaints from end-users) by default
    • might want to relax this to allow personal package archives and also lintian checks on the resulting binaries? Alternatively, allow binaries to be uploaded, run lintian, store the lintian results and then remove the binaries.
  • Emphasis on social interaction/networking
  • More interactivity - AJAX/AHAH - Web 3.0 ;)

  • Better looks of the site (steal from http://launchpad.net, http://xing.com, http://debian.wgdd.de/debian/, www.haveamint.com/)

  • User accounts for Debian developers/sponsors, too. For subscription of packages (e.g. sponsors would get automatic notifications of new uploads).
  • Proper URLs (e.g. http://mentors.debian.net/packages/cream instead of http://mentors.debian.net/cgi-bin/view_packages?package=cream) - RESTful

  • Making information available to others through proper interfaces (JSON, XML, SOAP, whatever) - might help sponsors.debian.net - and document that interface properly ("mentors API")
  • Getting BTS information through its LDAP (or SOAP) interface
  • More icons for better overview (http://www.famfamfam.com/lab/icons/silk/)

  • Clear start page with big buttons telling the purpose of the site (similar to http://flickr.com)

  • Syndication feed (preferably ?AtomPub http://atompub.org/) on new uploads

  • search functions (sponsors, sponsorees, packages) - search by Programming Language, build tool (CDBS), bugs closed (RC).
  • run the new site in parallel to the old site as a beta service (http://mentors.debian.net:5000)

  • make the Python code available (careful not to publish passwords in the repository)
  • proper documentation of modules, URLs, database schema in restructured-text format within the repository
  • make it generic so that other people can setup their own private repository servers
  • have a plugin architecture (to allow certain scripts or functions being executed at various times like when a package is uploaded, after it's checked, when it's sponsored, when someone leaves a comment etc.). That should allow people to extend the application in case they want a buildd-like function for automated binary package building.
  • for each package, produce a list of metrics. for each sponsor, have them rate the importance of the metrics to them. when the sponsor logs in, order the list of packages by score, which is based on the sum of importances of the metrics. allow sponsors to hide packages with low score. allow sponsees to see how different sponsors have scored their packages.
  • sponsoring queues for each of the large teams in Debian?
  • integrate svnbuildstat functionality?
  • merge REVU and mentors into one site?
  • expire old packages after X months and notify the maintainer that they should put more effort into finding a sponsor
  • ability to deal with VCS repositories in addition to uploading source packages
  • steal some features from http://fedoraproject.org/wiki/Features/ReviewOMatic

Proposed tools, modules and languages

  • Language: Python 2.4
  • Web framework: Pylons 0.9.6

  • Javascript library: jQuery

  • Database: PostgreSQL 8.1
  • Database layer/object-relational-mapper: SQLAlchemy 0.4

Assets (Database tables)

  • Users
    • package maintainers/sponsorees
    • Debian developers/sponsors
    • Database fields
      • Index
      • Real name
      • Email address (unique but can be updated)
      • PGP key (currently used for authenticating uploads)
      • IRC nickname on OFTC (so that users can be informed online if they wish)
      • Password (MD5 hashed)
  • User-Ratings (let other users voice their opinion on a certain maintainer)
    • Index
    • Timestamp
    • ID of the user this rating belongs to
    • ID of the submitter of this rating
    • Text
  • Files
    • Possible files are .dsc, .orig.tar.gz, .tar.gz, .diff.gz
    • These are the files in the actual repository that are stored in a diretory tree on disk
    • Database fields
      • Index
      • Timestamp (when the file was placed in the repository)
      • Filename
      • Size
      • MD5 checksum
      • Package ID the file belongs to
  • Packages
    • Index
    • User ID the package belongs to
    • Name (could also be used as the directory name in the repository tree)
    • Description
    • Programming Language
    • QA Warnings (lintian errors, piuparts errors, native package, etc.)
    • Closes bugs (list of bugs to be closed by the upload)
    • Logfile (textual output from the import process to debug problems)
    • Watch counter (how many times the package has been looked at)
    • Download counter (how many times the .dsc file has been downloaded)
    • Seek Sponsor Flag (set to "true" if the package needs a sponsor - otherwise a sponsor is found already)
    • Announced IRC Flag (will be set to "true" to mark that the upload has been announced on #debian-mentors)
    • Comment (can be set by user - e.g. "Security update" or "Replaces package foobar")
  • Package-History
    • Index
    • Timestamp
    • Type (set of 'newupload', 'sponsored', 'deleted', 'comment', 'seekingsponsor', etc.)
    • Text (the actual content of the history entry)
  • Package-Comments (let other users voice their opinion on a certain package)
    • Index
    • Timestamp
    • ID of the package this rating belongs to
    • ID of the submitter of this rating
    • Thumbs (thumbs up or thumbs down)
    • Text

Uploading/Importer

  • QA tests
    • syntax check of *.changes file
    • check if package is properly signed
    • check if maintainer is listed as registered user in the database
    • build test (perhaps through pbuilder - do we really need to spend so many resources? is it the scope of mentors?)
    • piuparts test (can we handle that with the resources?)
    • lintian checks (backport of lintian)
    • do the closed bugs belong to the package? (some packages accidentally closed other packages' bugs)
    • native package check
  • Uploads are done via HTTP with authentication (email address + password) so they can be received by the web framework (instead of an FTP server)
  • Removed files from previous package versions (only one version is kept)
  • Inform people who subscribed to the package
  • Pull orig.tar.gz from official Debian repositories if not included in the upload (is that really useful?)
  • maintainers should be forced to increase the package version/revision number to avoid confusion when different package contents had the same version/revision number (yes, this will be confusing at first)

Packages

  • Package list (http://mentors.debian.net/packages)

    • pagination (not displaying everything on the same page)
    • allow filtering (e.g. only show packages needing a sponsor)
  • Per-package page (http://mentors.debian.net/packages/foobar)

    • Closed bugs (clicking on a bug opens a popup getting information from the BTS over LDAP - ldap://bts2ldap.debian.net:10101 or SOAP)
    • Other packages from this maintainer
    • History
  • History (keep a history of uploads, comments, changes)
    • history is kept for all eternity - even for packages that were deleted already
  • Comments
    • if done comfortably this could replace the RFS-threads on the debian-mentors mailing list (duplication? some sponsors may prefer to retain RFS threads on the mailing list).
  • Check if the package is already in Debian
  • Creating RFS boilerplates (filling out as much information as possible with strong emphasis on maintainers adding meaningful content and not blindly filling in the fields)
  • Is there a new upstream version?
  • Popcon score
  • How long has the package been waiting for a sponsor?
  • display information on copyright, upstream homepage if possible

Maintainers/sponsorees

  • Information page
    • which packages does the sponsor have online
  • allow changing GPG key

Sponsors/DDs

  • Let DDs/sponsors register/have accounts, too
  • List of preferences (programming langue, CDBS/debhelper, ...) so that maintainers can try to find a matching sponsor

Repository access

  • don't store the packages in an Apache-index-style directory tree but serve the files through ?FileApp (for more flexibility)

    • (codehelp: I think that would be a mistake. Index style directories are more suitable for some sponsors.)
  • count downloads for statistics ("your package 'foobar' has been downloaded 23 times") (Nice to have but not essential.)
  • does anyone use mentors as a "deb-src" repository? or do we all use "dget -x"?

Background magic

  • monitor debian-devel-changes mailing list to see if packages were sponsored already
  • check if newer packages are availible in the Debian repository
  • announce new uploads on IRC (#debian-mentors on OFTC) - perhaps even /msg sponsors if they are online (codehelp: No thanks. email notifications are sufficient, sponsors are not at the beck and call of maintainers.)
  • warn maintainer about old packages (older than 3 months) / expire old packages (older than 6 months)

Metrics

Stuff that you would like to see checked automatically (or based on info provided by the maintainer) so you could use it to prioritise packages.

Package maintainer

  • maintainer is in NM
  • maintainer is willing to enter NM
  • approx date maintainer is planning to enter NM
  • maintainer is a DM
  • maintainer is willing to become a DM
  • approx date maintainer is planning to apply for DM
  • city maintainer lives in? (useful for determining if you can meet up)
  • timezone maintainer lives in?
  • maintainer has >1, >5, >10, >20, >100 packages in Debian already

  • has the maintainer read SC/DFSG? policy? devref? maint-guide? mentors FAQ?

Package metadata

  • this is a QA upload/NMU
  • is this an adopted package?
  • is this a hijacked package?
  • if this is an adopted/hijacked package, what is the rationale for that?
  • is this package available elsewhere (MergeDerivedDistributions/whohas)? what is the bugs/popcon/patches there?

  • is this is a new package?
    • does it close an ITP or other wnpp bug?
    • how long has it been waiting to be uploaded?
  • is this an update to an existing package?
    • what percentage/number of RC/important/normal/wishlist bugs does it close.
    • how long between the previous upload and this one
    • what proportion of existing lintian errors does it fix
    • who else have sponsored this package
    • does it have enough debtags? (define mandatory debtags facets and report ones missing from the package)
    • if the dependencies or debtags indicate some kind of UI, does it have a up-to-date screenshot (latest version available in the archive)?
    • how popular is it according to Debian popcon? Ubuntu popcon?
    • does it fail piuparts tests (for stable2unstable, testing2unstable, unstable, unstable2experimental)
    • is the version the same in testing/unstable?
      • if not, what are the excuses?
      • if so, is the testing version installable in stable? if not, is there a backport?
    • is a review available on debaday.debian.net?
    • does it have a history of or current security issues (CVE ids in security tracker)
  • number of upload attempts for this version of this source package
  • is there a homepage field in the control file and is it in the Source: stanza?
  • does the homepage and all the URLs in the Description give a HTTP 200 OK?
  • does the homepage return or redirect to a spam site or domain parking page?
  • are there VCS-* fields specified in the control file?
    • does the VCS-Browser URL return a web page?

    • does the VCS-foo URL contain a valid VCS repository, according to the $foo VCS client?

  • is it a native package?
  • which archive area (main, contrib, non-free) is the package intended for?
  • what section does the package specify in its Section control field? (mail, python, base, etc.)

  • are any of the dependencies transitional packages, and which ones?
  • are any of the dependencies dummy packages, and which ones?
  • are any of the dependencies not satisfiable, and which ones?
  • are any of the dependency alternatives not satisfiable, and which ones?
  • are any versioned dependencies asking for versions older than available in oldstable?
  • which programming languages is the code written in?
  • does the maintainer have commit/push access upstream?
  • what are your motivations for packaging this software?
  • how long do you intend to maintain the package? next release? x months?
  • if the Standards-Version was modified, what specific Policy changes affect this package since the last time this field was changed?
  • are the DMs in ?dm.txt for the package amongst the Maintainer/Uploaders?

Source package

  • package builds or FTBFS in sid on X architectures
    • including unofficial arches and with double builds, /bin/sh -> dash, -j2, DPKG_GENSYMBOLS_CHECK_LEVEL=4 etc

  • source package produces X lintian errors, Y lintian warnings, Z lintian infos, uses xx overrides
  • orig.tar.gz matches the tarball inside upstream's orig.tar.bz2/lzma/xz (this will be obsoleted by dpkg-source v3)
  • source package contains a ‘debian/watch’ file
    • ‘uscan’ gives no warnings or errors
    • ‘uscan --report’ reports the package is up-to-date
    • ‘uscan --download’ generates an original upstream source that matches the one provided in the source package
  • if there is no ‘debian/watch’ file, is there a README.Source file or a get-orig-source target
  • total size of the source package
  • size of the orig.tar.gz/diff.gz
  • is the diff.gz clean (only changes files in debian/)
  • does it require the -v option to debuild/dpkg-buildpackage?
  • does it use cdbs? or debhelper? or manoj-cdbs? or xsf-cdbs? or nothing?
  • are any files outside debian/ modified?
    • if so: how files many are modified?
    • if not: is a common patch system used?
      • if so: how many patches are there?
      • if so: which patch system is used? quilt? dpatch? cdbs-simplepatchsys? dbs?
      • if not: is there a patches/ dir, or are there any .diff or .patch files in debian/?
  • build logs:
    • number of warnings from gcc/python/etc
    • number of warnings corresponding to security or portability issues.
  • do any static analysis tools find issues? how many? (for eg: rats, flawfinder, cppcheck, splint, scan-build from clang/llvm)
  • code size (for people who audit) using wc -l? using ohcount or sloccount
  • what is the output of licensecheck?

  • if the package contains python files, what is the output of pymetrics, pychecker, pyflakes, and pylint?
  • if the package contains perl files, what is the output of Perl::Critic?
  • if the package contains Java files, what is the output of pmd, findbugs, jcsc, jlint?
  • if the package contains gettext translations, what is the output of gettext-lint, pofilter and Pootle from the pofilter tests?

  • if the package contains CSS files, what is the output of csslint

Resulting binary package

  • binary package produces X lintian errors, Y lintian warnings, Z lintian infos, uses xx overrides
  • if it is a GUI/console app, does it support i18n?
  • do input fuzzers affect it? (for eg: zzuf, fusil, fuzz, inguma, netsed, wapiti)
  • do any of the files conflict with an existing package? is there a Conflicts field?
  • all the lintian wontfix bugs (inc archived versions): http://bugs.debian.org/cgi-bin/pkgreport.cgi?archive=both;include=tags%3Awontfix;package=lintian

  • if this is a CPAN module, what is the Kwalitee level? >X? >Y? >Z?

  • if the package contains a ttf/otf font, what is the output of ftinfo, fontlint, ftvalid, ttfcoverage, otfinfo, openfontsanitiser?

  • if the package contains a font, what does the automated font sample image look like?
  • if the package contains library, is there an ABI breakage or unnecessary ABI bump compared to the archive?
  • if the package contains a FreeDesktop menu file, what is the output of desktop-file-validate?

  • if the package contains python files, what is the output of pymetrics?
  • if the package contains perl files, what is the output of Perl::Critic?
  • if the package contains MP3 files, what is the output of mp3diags?
  • if the package contains PNG files, what is the output of pngcheck?
  • if the package contains JPEG files, what is the output of jpeginfo --check?
  • if the package contains ?JavaScript, what is the output of http://www.javascriptlint.com/?

More ideas

Developers

This is a list of people involved in the 3.0 relaunch of the mentors site:

  • Christoph Haas
    Been there since the start. Founded the service years ago with Ivo Marino, Christoph Siess and D.K.Gebhart. Claims to always have done most of the work. Currently the only one working on the site, fixing bugs and doing user support

(add your name here)

The git repository is available. Public: read-only. Developers: full read-write with username/password.

Include this

Some documents from the repository need to be included in the above list.