Make Debile a first-class citizen in the Debian QA ecosystem

I am familiar with FLOSS, as an user since I installed Sarge in 2004, and as a developer since I made a few contributions to the Weboob framework, which allows easy interactions between console or graphical applications and various websites, in 2010.

I participated to the 2012 and 2014 GSoC:

I am now familiar with Debian packaging, QA tools, and communication within the project.

Project details

Debile, Firewoes and Debsources were originally envisioned as a tightly coupled group of applications for doing software quality assurance on Debian packages:

However, while Debsources bloomed and became a useful services to numerous users among the Debian and FLOSS communities as well as a powerful research tool, Debile has stagnated. Because of lingering issues in its distributed architecture, we haven't been able to exploit its potential.

Debile's very complicated setup process has made past developers as well as potential future contributors run away, because they couldn't even manage to run a development instance.

Meanwhile, without firehose reports produced by Debile to display, firewoes has lost its purpose, and it has no instance running anywhere.

This project aims to make Debile shine again. Firewoes will be brought back to life with an improved UI, more useful to its users, Debian packagers and upstream developers alike.

The main idea is to make both projects leverage Debsources' visibility and ability to gather statistics about packages.

Leverage Debsources in Firewoes

In Firewoes, I will make more use of Debsources' ability to insert "popup messages" into a source code view.

Leverage Debsources in Debile

Currently, debile runs all enabled plugins on all packages. Among other things, this means wasting time running scan-build on packages whose source code is not supported by clanganalyzer. A trivial inconvenience in the case of python or perl packages, but a huge cpu time sink for packages written in haskell, ocaml and other compiled languages.

On the other hand, Debsources knows, thanks to its sloccount plugin, what languages a package is developed with. We should retrieve that data in Debile.

Leverage Debile and Firewoes in Debsources

More than just a source code browsing web application, Debsources has become a research tool that can be used for running experiments. It might be useful to import data from Debile (via Firewoes) to extend its dataset.

Synopsis

Give Debile the love it deserves, by making it production-ready again, and leveraging Debsources' extensive dataset. Make Debile, Firewoes and Debsources interact with each other more than they currently do.

Benefits to Debian

Synergistic work on Debsources, Firewoes and Debile:

Deliverables

Project schedule

I am planning buffer periods for each subproject. They will be used for debugging, improving unit tests, documentation. Non-essential ("bonus") features will be developed within those when I have time to spare.

Community bonding

Setup a blog where I'll write shorts daily report. In addition to helping me write the weekly reports to soc-coordination, it'll allow me to quickly realize where I'm unproductive or get off track.

Improve code coverage in Debile, especially in areas where I'll break stuff.

Continue working on the changes I've started a few months ago, making Debile more easily installable.

If other GSoC students are accepted on a Debile-related project, all that will be driven by helping them setup Debile on their own machines.

Phase 1: make debile installable

Finalize what has been started on that subject.

Check that multiple instances of debile-slave can be run on the same machine, in order to fully exploit the multi-core architecture of the slave nodes. Some plugins prevent us from doing that, and we need adequate init scripts.

Run debile.debian.net again, with only a small subset of fast plugins enabled, in order to have real data for Phase 2.

Phase 2

Improve Firewoes using Debsources. Some work on Debsources' API might be needed.

Phase 3

Make Debile aware of packages' programming languages (database changes).

Retrieve programming language data from debsources. Some work on Debsources' API will probably be needed.

Optimize Debile job creation to avoid running jobs that cannot have results

Exams and other commitments

End of exams mid-May, no other commitment after that.

Why Debian?

Are you applying for other projects in SoC?

Yes