Make Debile a first-class citizen in the Debian QA ecosystem
Name: Clément Schreiner
Contact/Email: mailto:clement@mux.me - irc://irc.oftc.net/clemux
Background: I am a 23-year-old student in computer science at University of Strasbourg (France).
I am familiar with FLOSS, as an user since I installed Sarge in 2004, and as a developer since I made a few contributions to the Weboob framework, which allows easy interactions between console or graphical applications and various websites, in 2010.
I participated to the 2012 and 2014 GSoC:
- The former on debexpo (mentors.d.n), which allowed me to get better at programming, learn to use sqlalchemy, and get interested in software quality assurance.
- The latter on Debile, on which I am still working when I have some spare time, currently trying to make it installable without causing Sudden Hair Loss Syndrom to potential contributors, as well as improving the code coverage.
I am now familiar with Debian packaging, QA tools, and communication within the project.
Project title Make Debile a first-class citizen in the Debian QA ecosystem
Contents
Project details
Debile, Firewoes and Debsources were originally envisioned as a tightly coupled group of applications for doing software quality assurance on Debian packages:
- Debile: build packages, with default and alternative compilers, run static or dynamic analyzers on their code or binaries
- Firewoes: browse Debile's results
- Debsources: browse packages source code, provide context for displaying firewoes reports
However, while Debsources bloomed and became a useful services to numerous users among the Debian and FLOSS communities as well as a powerful research tool, Debile has stagnated. Because of lingering issues in its distributed architecture, we haven't been able to exploit its potential.
Debile's very complicated setup process has made past developers as well as potential future contributors run away, because they couldn't even manage to run a development instance.
Meanwhile, without firehose reports produced by Debile to display, firewoes has lost its purpose, and it has no instance running anywhere.
This project aims to make Debile shine again. Firewoes will be brought back to life with an improved UI, more useful to its users, Debian packagers and upstream developers alike.
The main idea is to make both projects leverage Debsources' visibility and ability to gather statistics about packages.
Leverage Debsources in Firewoes
In Firewoes, I will make more use of Debsources' ability to insert "popup messages" into a source code view.
Leverage Debsources in Debile
Currently, debile runs all enabled plugins on all packages. Among other things, this means wasting time running scan-build on packages whose source code is not supported by clanganalyzer. A trivial inconvenience in the case of python or perl packages, but a huge cpu time sink for packages written in haskell, ocaml and other compiled languages.
On the other hand, Debsources knows, thanks to its sloccount plugin, what languages a package is developed with. We should retrieve that data in Debile.
Leverage Debile and Firewoes in Debsources
More than just a source code browsing web application, Debsources has become a research tool that can be used for running experiments. It might be useful to import data from Debile (via Firewoes) to extend its dataset.
Synopsis
Give Debile the love it deserves, by making it production-ready again, and leveraging Debsources' extensive dataset. Make Debile, Firewoes and Debsources interact with each other more than they currently do.
Benefits to Debian
Synergistic work on Debsources, Firewoes and Debile:
- more QA on its packages, by making Debile usable
- real-word testing of Debsources' API (and improvement thereof)
- improved UI for Firewoes
Deliverables
- a easily installable release of Debile
- an improved UI for Firewoes
- improvements in Debile to stop wasting time running non-adequate plugins on packages
Project schedule
I am planning buffer periods for each subproject. They will be used for debugging, improving unit tests, documentation. Non-essential ("bonus") features will be developed within those when I have time to spare.
Community bonding
Setup a blog where I'll write shorts daily report. In addition to helping me write the weekly reports to soc-coordination, it'll allow me to quickly realize where I'm unproductive or get off track.
Improve code coverage in Debile, especially in areas where I'll break stuff.
Continue working on the changes I've started a few months ago, making Debile more easily installable.
- more unit tests on the simple authentication backend
- a new CLI tool, debile-auth, for managing authentication requests
- improve the docker scripts, with the help of our local Docker expert paultag.
If other GSoC students are accepted on a Debile-related project, all that will be driven by helping them setup Debile on their own machines.
Phase 1: make debile installable
Finalize what has been started on that subject.
Check that multiple instances of debile-slave can be run on the same machine, in order to fully exploit the multi-core architecture of the slave nodes. Some plugins prevent us from doing that, and we need adequate init scripts.
Run debile.debian.net again, with only a small subset of fast plugins enabled, in order to have real data for Phase 2.
Phase 2
Improve Firewoes using Debsources. Some work on Debsources' API might be needed.
Phase 3
Make Debile aware of packages' programming languages (database changes).
Retrieve programming language data from debsources. Some work on Debsources' API will probably be needed.
Optimize Debile job creation to avoid running jobs that cannot have results
Exams and other commitments
End of exams mid-May, no other commitment after that.
Why Debian?