Debian Smart Upload Server GSoC Proposal
Name: Matt Novenstern
Contact/Email: on Freenode, nick: fisxoj
Background: Physics student at Colgate university, studying abroad in Freiburg, Germany. Have contributed patches to rhythmbox and have done research in statistical mechanics with a simulation written in fortran.
Synopsis: Update the mechanism for uploading packages to debian.
Deliverables: server and a client that handle package uploads more intelligently
Project details: Create a simple client and server to handle a smarter upload of debian packages. The server should be HTTP based and accept a file at a time and run specified tests on the file and let the client know if it passes them. The testing section should be as general as possible, to allow more tests to be added at any time, preferably just dropped into place. Upon successful upload and check of files, push them down the line.
Project schedule: I can begin working right when the program begins, and continue working mostly regularly throughout its course. Here is a basic outline of the project schedule, though bullets will expand and contract as required:
- Implement server and client that can talk to each other in a mostly HTTP protocol
- Work on testing infrastructure - implement a generic testing system for all types of debian files
- Work out client/server exchanges in light of the possibility of failing a test, try and make the system able to fix trivial problems
- Backwards compatibility - teach the queue daemon to talk to the new server software and run packages uploaded through the old method through the smarter check system so they can be discarded efficiently if they don't pass tests and alert the uploader via e-mail
Travel: I live an hour north of New York City and will be there at that time of year. It would be trivial to come into the city by train.
Other summer plans: I am studying for a semester in Freiburg, Germany, and only have exams to deal with. The middle to end of July will be busy, as I take exams and move back to the US. Aside from that, there will just be a normal course load.
Exams and other commitments: See above.
If you are not a Debian Developer: No particular plans, but, I haven't been involved in the project before. We'll see what I find out about/find interesting.
The protocol should be based on a standard file transfer mechanism, to make use of already existing security software. Simple HTTP for a simple task seems best to me. The next section is testing the changes file to ensure it's ok to upload the other files. This should be a matter of porting existing tests to a new infrastructure.
Protocol
- Only requires a few commands/responses
- Client sends changes
- Server gives client go ahead if .changes passes whatever checks are present for .changes files
- Client sends other files and server checks them if required after each file is sent
- Server lets the client know how the transfer went
The general idea of this post provides a good deal of planning and interesting ideas including (but of course not limited to):
- Killing the connection immediately in the event of an abnormally large .changes file. It was pointed out in a followup that some .changes files are bigger than the suggested 16k, which is easy enough to fix to an arbitrary size.
- Killing the connection in the event of a bad signature.