Interpreting the release critical bug statistics

The unofficial release-critical bug tracking tool at http://udd.debian.org/bugs.cgi can be used to browse and filter release-critical bugs more flexibly than is possible with the official bug tracker, and can give us a clearer picture of an upcoming release.

For example, this interface can filter release-critical bugs by different Debian distributions. It currently lists 477 release-critical bugs in total for any distribution, but if you filter for release-critical bugs affecting Squeeze, you'll get only 378. That number's different from the one shown by the official bug tracker; I'm not entirely sure why, but a part of that difference is that both the official and the unofficial web pages are only synched periodically.

But let's play more: we can filter for bugs that apply only to Squeeze, and not to Sid. These bugs are already fixed in Sid, but the packages haven't yet migrated to Squeeze. There are currently 72 such bugs. We can furthermore ignore bugs only in Sid; when Squeeze is unaffected, we just don't migrate the broken package from Sid to Squeeze. The number that is most interesting for contributors is the number of bugs affecting both Sid and Squeeze, as these are the ones which really need to be fixed.

So filtering for bugs for both we are down to 306 at the moment.

But of these 306 remaining release-critical bugs, we can still ignore some (for now). For example, bugs concerning packages in non-free or contrib (13 currently) won't stop us from releasing, will they? There are also many bugs marked as pending (16), meaning that the maintainer is aware of the bug, has a fix prepared, and is just waiting a while before an upload. Some bugs already have a patch (46), but have not been uploaded (one possible reason being that it hasn't been reviewed). You'll also see that we can ignore merged bug reports (37) which are mere duplicates. Finally there are bug fixes already uploaded to the so-called "delayed" queue. These are bug fixes which were uploaded by someone other than the official maintainer, but to give the maintainer a chance to comment or react, the upload is delayed by some days before it will actually hit the archive. Currently 7 bugs are fixed by uploads to delayed. 3 bugs are "claimed", meaning that someone has volunteered to take care of them.

And we can ignore security related bugs as well (29). The bad thing is that this number will never hit 0 since security problems are detected all the time and thus these bugs come in all the time. The nice thing is that they can be fixed after the release via a security update, so we can (more or less) safely ignore them for our statistics, too.

We could also ignore the bugs which will vanish after the next migration of packages from Sid to Squeeze (tagged in the unofficial bugtracker as "invalidated by todays britney"), but as we are only looking at RC bugs in both Sid and Squeeze, this number will always be 0 ;)

That leaves "bugs marked as fixed some other way" which are bugs (27) where I honestly have no idea what they are... If someone can explain them, please do so ;)

Here is a small table showing the above numbers:

In Total:

477

Affecting Squeeze:

378

Squeeze only:

72

Remaining to be fixed in Squeeze:

306

Of these 306 bugs, the following tags are set:

Pending in Squeeze:

16

Patched in Squeeze:

46

Duplicates in Squeeze:

37

Contrib or non-free in Squeeze:

13

Claimed in Squeeze:

3

Delayed in Squeeze:

7

Fixable by a security update:

29

Otherwise fixed in Squeeze:

27

Or in other words: Release-critical bugs left in Squeeze, when ignoring all these: 171

That's a pretty number, isn't it? There are only 171 bugs left which need our immediate attention :)

But that is a rather optimistic view of the situation. We e.g. assume that all bugs having a patch are really fixed by the patch. We also ignored the fact that some bugs in Squeeze are already fixed in Sid, but the package can't migrate because the package in Sid contains a new bug - this seems to happen quite often.

So let's take off the rose-tinted spectacles and take a more pessimistic view (release managers must be pessimists to ensure high quality releases ;) ). We count the unmerged bugs in Squeeze main, and ignore only those with fixes on the way via an imminent scheduled migration, a delayed-queue upload, or a "hinted" override to the usual wait.

Seen from this release manager's viewpoint, 280 bugs remain to be fixed somehow before we can release.