Peer review for packages

Goal

Help package maintainers to get technical feedback about their work.

Note the more recent proposal in Mentors/BTS.

Principle

It is not always easy to get feedback about one's package on lists like debian-mentors, since calls for comments that are not quickly answered tend to be forgotten and eclipsed by newer requests. Moreover, there is no incentive for doing reviews: there is always the temptation to hope that somebody else will do the work.

This is a proposal for a simple peer review system of prospective or existing packages using the Debian BTS, with the double goal of avoiding loss of momentum and better sharing the workload. For each review the maintainer gets, he gives one to another maintainer.

The core idea is to represent the request for review as a wishlist bug, either against the package itself or its WNPP bug, and to track the requests with the BTS usertags:

If two reviews are not enough, the maintainer can request more by contributing a new review on another package, and changing the usertag for his own package back to one-package-review.

To be reviewed, the package must be published somewhere, for instance in a source code repository or on DebianMentorsNet.

Mode of operation

Usertags and package lists

BTS usertags need a user name. It is proposed to use debian-mentors@lists.debian.org, as it will not increase the traffic (no emails are sent). The BTS generates lists of bugs with a combination of user and usertags:

Packages waiting for their first review:

http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=package-review-requested;users=debian-mentors@lists.debian.org

Packages waiting for their second review:

http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=one-package-review;users=debian-mentors@lists.debian.org

Packages that have been reviewed at least two times:

http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=two-package-reviews;users=debian-mentors@lists.debian.org

Step one: review packages

Pick a one package that has no review, and another that was reviewed once, from the lists above. Try to take the first of the list, so that bigger packages do not get ignored.

Send your comments to the bug where the review is requested, and tag it accordingly so that it is displayed in the correct list (see above).

For example, the following control commands could be used when sending the second review for package whose bug number is 1234546:

user debian-mentors@lists.debian.org
usertags 1234546 two-packages-reviews
thanks

Step two: request review

Now you have given to the community, you can ask the community to give to you. To add you package to the list of review requests, open a wishlist bug on your package and tag it package-review-requested with the user debian-mentors@lists.debian.org. If your package is not yet part of Debian, tag its WNPP bug instead. Do not forget to indicate where and how the package can be downloaded (dget, debcheckout, …)

Discussion

Please discuss here about improvements.

See also

CopyrightReview, Mentors/BTS