Testing Debian with OpenQA
Description of the project: In order to test the Debian-Installer, we have installed an instance of OpenQA, which can be seen here: https://openqa.debian.net/ This is a very recent development (as it started doing successful tests yesterday). There is a vast amount of work left to do, which means that an intern would have quite a lot of freedom on where they could focus. One thing that would be very useful would be to develop tests, and document the proceedure for doing so, such that people other than me (Philip Hands) can be encouraged to test the bits of Debian that they work on. This is in no way limited to Debian-Installer.
The fact that the Debian-Installer tests now work means that it is rather easy to test other things, since one has a ready source of cleanly installed disk images. Developing tests involves writing a script for the system to apply to screen (so, look for something on the screen, then type something, or mouse-click something if you see it). This is done in perl, but it only really needs the simplest of perl, so lack of perl experience is no barrier. Once one has a (partial) script, the test can be run, at which point the screenshots will be found to be missing, and one can then use the Web UI to select them. As said, the important part of this process would be ensuring that it is easy for others to join in and add their own tests, as that will move that bottleneck.
There are also ways of streamlining the test development process. Discovering whether people can develop tests on their own systems, and how they can take advantage of these time-saving techniques, and documenting that, would also be very useful.
There are many other aspects of this project that are available for an intern to branch into, depending on what interests them most. We have a jenkins instance creates test images for Debian-Installer -- it should be able to trigger tests on OpenQA, and accept results back. This requires some scripting and co-ordination between systems. The CD team make daily images. At present we are probing to see if new images exist, but it would be better if they could trigger tests. User authentication of OpenQA is currently relying on OpenSUSE's single-sign-on, so this needs to be switched to using sso.debian.net or some such. etc. etc. This is a work in progress, so some of these points are likely to be addressed before the intern joins the team, but every fix is likely to throw up two new issues for quite a while, so there is plenty to do.
Confirmed Mentor: Philip Hands
How to contact the mentor: email@example.com or 'fil' on IRC (OFTC)
Confirmed co-mentors: Holger Levsen
Deliverables of the project: Documentation: for writers of new tests, and for people wishing to trigger tests from other systems. Demonstrations of the documented procedures working
Desirable skills: simple Perl scripting for writing the tests is needed, but should be easy to learn. The main skill we're looking for is a fresh perspective with which to notice which things don't work without cheating (i.e. using a root login on the server), and a willingness to work together in eliminating those cases.
What the intern will learn: Testing by hand is tedious and so often is not done comprehensively, so having the skills required to set up automated tests is useful in its own right. More important to Debian would be enabling others to do their own testing, as this should allow things to scale up to testing all of Debian. Being part of that will give opportunities to interact with developers across the project, using various parts of our infrastructure so this should be a good introduction to how the whole project works.
Application tasks: FIXME
Related projects: openqa from suse+fedora, ci.debian.net, jenkins.debian.net, tails testsuite (on roadmap)