Debusine: live log streaming
Description of the project: Debusine allows viewing logs for tasks after they have been completed, but it should also allow viewing logs of tasks that are in progress.
Confirmed Mentor: Stefano Rivera
How to contact the mentor: stefanor@debian.org
Confirmed co-mentors: None confirmed, but other people from the Debusine team can help if needed.
Difficulty level: Medium to hard
Project size: 175 hours (medium); this project can be split into multiple stages
Deliverables of the project:
- Debusine workers incrementally collect output from tasks as they run, and send it to the server which stores that output in a suitable backend database.
- (Stretch goal:) The Debusine web UI shows streaming output from running tasks.
Desirable skills: Python systems programming, Python async programming, websocket communication, Redis and/or PostgreSQL, Django application development. For later stages, HTML+CSS+?JavaScript development without ?JavaScript frameworks.
What the intern will learn: Development practices for a medium-sized real-world software development platform. Systems design architecture, with care for latency and scalability. Web user interface design.
Application tasks:
Pick a Quick fix issue from our list and try to fix it, including full test coverage for your changes.
- Write a "producer" command line tool that runs a program and streams its output to Redis.
- Write a "consumer" command line tool that takes output streamed to Redis and writes it on its output. Ideally, while one producer is running, one could start multiple consumers and see the same output streaming on both.
Additional details:
- Development would have multiple milestones. The first two would be required for successful completion of the project, and the remaining two would count towards rewarding user-visible bragging rights:
- Worker-side, collecting relevant output on the worker code and sending it back to the server via the websocket interface
- Server-side, collecting incoming output from workers and dispatching it to an appropriate Redis structure or a PostgreSQL table
- Server-side, providing a view to make incoming output lines available to code on the browser side
- Browser-side, subscribing to updates and viewing them streaming on the page
- Development would have multiple milestones. The first two would be required for successful completion of the project, and the remaining two would count towards rewarding user-visible bragging rights:
AI usage policy: We will not accept applications written using LLMs, and we expect accepted students to write code themselves, not via an LLM.