Differences between revisions 1 and 6 (spanning 5 versions)
Revision 1 as of 2014-02-07 16:11:57
Size: 2148
Comment:
Revision 6 as of 2017-03-05 03:51:00
Size: 3043
Editor: PaulWise
Comment: new udd-mirror.debian.net domain
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from SummerOfCode2014/ProjectProposals/DebianMetricsPortal
Line 3: Line 4:
 * '''Description of the project:''' To improve, one needs to measure, apply changes, and see how the changes affect the observed quality. In Debian, we have a lot of project-wide metrics, [[Statistics]], and graphs. Unfortunately they are scattered and maintained in a non-coordinated manner, resulting in recurrent shortages, and the lack of a uniform interface to view, add, and query Debian metrics. This project aims at building a Debian metrics portal with a uniform (Web) interface to peruse Debian metrics, as well as a uniform (programming) interface to maintain them.  * '''Description of the project:''' To improve, one needs to measure, apply changes, and see how the changes affect the observed quality. In Debian, we have a lot of project-wide metrics, [[Statistics|statistics]], and graphs. Unfortunately they are scattered and maintained in a non-coordinated manner, resulting in recurrent shortages, and the lack of a uniform interface to view, add, and query Debian metrics. This project aims at building a Debian Metrics Portal with a uniform (Web) interface to peruse Debian metrics, as well as a uniform (declarative) interface to add and maintain metrics.
Line 5: Line 6:
 * '''How to contact the mentor:''' see mentor wiki page
 * '''Confirmed co-mentors:''' ''none yet''
 * '''Confirmed co-mentors:''' Stuart Prescott
 * '''How to contact the mentor:''' via email at zack@debian.org + stuart@debian.org
Line 8: Line 9:
   * standardized interface to add/remove metrics to be graphed (possibly with different sampling rate)
   * integration of existing graphs in the metrics infrastructure
   * web interface to show daily (or more frequently) updated graphs of the various metrics
   * dynamic web interface to graph, on demand, specific metrics (possibly more than one at a time) over specific time periods
   * (optional) package the developed code to ease deployment on Debian-based machines
   * database structure to store all historical data points of all metrics
   * standardized declarative interface to add/remove metrics to be graphed. The interface should allow for both "local" metrics (e.g. data generated by scripts run on the machines hosting the metrics portal) and "remote" metrics (e.g. data generated by remote data sources which are then periodically gathered by the metrics portal)
   * cron jobs to periodically fetch new data and generate graphs
   * proof of concept: integration of (some of the) existing graphs in the metrics infrastructure
   * web interface to show updated graphs of the various metrics
   * client-side dynamic web interface to graph, on demand, specific metrics (possibly more than one at a time to look for correlations) over the desired time periods
   * (optional) produce a Debian Package of the portal code to ease deployment on Debian-based machines
Line 15: Line 18:
   * [[http://oss.oetiker.ch/rrdtool/|RRDtool]] and its [[http://oss.oetiker.ch/rrdtool/prog/rrdpython.en.html|Python bindings]]
   * some Python templating engine (e.g. genshi, jinjia2, etc.) might be useful
   * some JavaScript library to generate dynamic graphs might be useful as well (name one
!)
   * please '''include as part of your application a Python script''' that uses RRDtool to graph some of the metrics from [[http://udd.debian.org/schema/udd.html#history.table.sources-count|UDD history table]]
 * '''What the student will learn:''' deal with live legacy data and code; consolidate into a single, well-designed architecture existing functionalities; this will help Debian in monitoring its strength and deficiencies, and evaluate the usefulness of changes to project processes
   * [[http://www.sqlalchemy.org/|SQLAlchemy]] and [[https://code.google.com/p/sqlalchemy-migrate/|SQLAlchemy-migrate]]
   * [[http://matplotlib.org/|matplotlib]]
   * [[https://en.w
ikipedia.org/wiki/JavaScript|JavaScript]] (optional but desirable for the client-side interactive part)
   * some Python templating engine (e.g.
[[http://jinja.pocoo.org/|jinjia2]]) might be useful (name one you like in your application!)
   * some Javascript library to generate dynamic graphs (e.g. [[https://code.google.com/p/flot/|flot]]) might be useful as well (name one!)
   *
please '''include as part of your application a Python script''' that uses matplotlib to graph some metrics from [[http://udd.debian.org/schema/udd.html#history.table.sources-count|UDD sources history table]] via the [[http://udd-mirror.debian.net/|public UDD mirror]]
 * '''What the student will learn:''' deal with live legacy data and code; consolidate into a single, well-designed architecture existing functionalities; this will help Debian in monitoring its strength and deficiencies, and evaluate the usefulness of changes to project processes.

Debian metrics portal

  • Description of the project: To improve, one needs to measure, apply changes, and see how the changes affect the observed quality. In Debian, we have a lot of project-wide metrics, statistics, and graphs. Unfortunately they are scattered and maintained in a non-coordinated manner, resulting in recurrent shortages, and the lack of a uniform interface to view, add, and query Debian metrics. This project aims at building a Debian Metrics Portal with a uniform (Web) interface to peruse Debian metrics, as well as a uniform (declarative) interface to add and maintain metrics.

  • Confirmed Mentor: StefanoZacchiroli

  • Confirmed co-mentors: Stuart Prescott

  • How to contact the mentor: via email at zack@debian.org + stuart@debian.org

  • Deliverables of the project:

    • database structure to store all historical data points of all metrics
    • standardized declarative interface to add/remove metrics to be graphed. The interface should allow for both "local" metrics (e.g. data generated by scripts run on the machines hosting the metrics portal) and "remote" metrics (e.g. data generated by remote data sources which are then periodically gathered by the metrics portal)
    • cron jobs to periodically fetch new data and generate graphs
    • proof of concept: integration of (some of the) existing graphs in the metrics infrastructure
    • web interface to show updated graphs of the various metrics
    • client-side dynamic web interface to graph, on demand, specific metrics (possibly more than one at a time to look for correlations) over the desired time periods
    • (optional) produce a Debian Package of the portal code to ease deployment on Debian-based machines
  • Desirable skills:

  • What the student will learn: deal with live legacy data and code; consolidate into a single, well-designed architecture existing functionalities; this will help Debian in monitoring its strength and deficiencies, and evaluate the usefulness of changes to project processes.