Differences between revisions 12 and 15 (spanning 3 versions)
Revision 12 as of 2012-04-01 13:03:15
Size: 8035
Comment:
Revision 15 as of 2012-04-01 21:16:40
Size: 8468
Comment:
Deletions are marked like this. Additions are marked like this.
Line 10: Line 10:
  * senior undergraduate student at the Computer Science and Automatic Control Faculty, Politehnica University of Bucharest
  * knowledge in ''C'' (8 years), as well as ''C++'', ''Java'' and ''Python''
  * knowledge in ''Algorithm Design'', ''Data Structures'' and ''Project Development Workflow''
  * knowledge in ''Compilers'', ''Operating Systems'', ''Networking'', ''Distributed Systems''
  * highly dependable and efficiency oriented professional
  * ambitious, focused and enduring individual
  * f
amiliar with the concepts of Open Source Development - I've pursued an Open Source Development Course[1], organized by ROSEdu[2]
  * refactoring, optimizing and unifying the metadata acquire system for APT would significantly improve the whole Debian user experience, as well as improve the Package Management System's consistency. To me, this is both a thrilling challenge and a great opportunity to analyze the Debian OS internals and core features.
  * Senior undergraduate student at the Computer Science and Automatic Control Faculty, Politehnica University of Bucharest
  * Knowledge in ''C'' (8 years), as well as ''C++'', ''Java'' and ''Python''
  * Knowledge in ''Algorithm Design'', ''Data Structures'' and ''Project Development Workflow''
  * Knowledge in ''Compilers'', ''Operating Systems'', ''Networking'', ''Distributed Systems''
  * Highly dependable and efficiency oriented professional
  *
Ambitious, focused and enduring individual
  * Familiar with the concepts of Open Source Development - I've pursued an Open Source Development Course[1], organized by ROSEdu[2].
  * Refactoring, optimizing and unifying the metadata acquire system for APT would significantly improve the whole Debian user experience, as well as improve the Package Management System's consistency. To me, this is both a thrilling challenge and a great opportunity to analyze the Debian OS internals and core features.
Line 23: Line 23:
   * Several tools for package management - ''apt-get'', ''debtags'', ''apt-file''
   * Each one handles its own set of package metadata, therefore its own view of the remote Debian Archive
   * Each one requires individual updating and interaction
   * Several tools for package management - ''apt-get'', ''debtags'', ''apt-file''.
   * Each one handles its own set of package metadata, therefore its own view of the remote Debian Archive.
   * Each one requires individual updating and interaction.
Line 27: Line 27:
   * '''Performance''': minimal bandwidth usage by efficient diffs and broad local package metadata
   * '''Effectiveness''': the user defines which components the acquire system will use, and the system only uses these components
   * '''Scalability''': the system is pluggable
   * '''Forward compatibility''': the system will have a generic design, open for future development
   * '''Performance''': minimal bandwidth usage by efficient diffs and broad local package metadata.
   * '''Effectiveness''': the user defines which components the acquire system will use, and the system only uses these components.
   * '''Scalability''': the system is pluggable.
   * '''Forward compatibility''': the system will have a generic design, open for future development.
   * '''Backward compatibility''': it is desired that the system doesn't break the existing interfaces, and the transition to the new sistem is as transparent as possible to the user.
Line 32: Line 33:
   [[attachment:PluggableAcquireSystemforAPT-ArchitectureDiagram.jpg]] {{attachment:PluggableAcquireSystemforAPT-ArchitectureDiagram.png}}
Line 35: Line 36:
    * is responsible with fetching the metadata from the ''Debian Archive'', processing it, and retrieving it to ''apt-get update''
    * implements efficient transport mechanisms
    * implements security enforcement mechanisms
    * works according to a configuration file
    * is responsible with fetching the metadata from the ''Debian Archive'', processing it, and retrieving it to ''apt-get update''.
    * implements efficient transport mechanisms.
    * implements security enforcement mechanisms.
    * works according to a configuration file.
Line 40: Line 41:
    * read the sources.list file - now having a decorated format - and sends the sources to the Config File Generator, along with specific setting for each one of these     * reads the sources.list file - now containing information about the enabled / disabled plugins - and sends the sources to the Config File Generator, along with specific settings for each one.
Line 42: Line 43:
    * produces the configuration file for the backend
    * supports multiple plugin registration - each time a package tool is installed, it registers itself to this component, with a default configuration. The user may change the plugin configuration - as he would have been capable of changing it from the package tool's interface.
    * maps these plugin configurations to the resulting sources provided by the parser
    * produces the configuration file for the backend.
    * supports multiple plugin registration: each time a package tool is installed, it registers itself to this component, with a default configuration. The user may change the plugin configuration - as he would have been capable of changing it from the package tool's interface.
    * maps these plugin configurations to the resulting sources provided by the parser.
Line 46: Line 47:
  * Improving the package management system translates into improving the fundamental layer of the Debian OS
  * Better bandwidth usage
  * Less config and temporary files
  * Scalability of the metadata acquire system
  * Better metadata cohesion
  * Improving the package management system translates into improving the fundamental layer of the Debian OS.
  * Better bandwidth usage.
  * Less configuration and temporary files.
  * Scalability of the metadata acquire system.
  * Better metadata cohesion.
Line 52: Line 53:
  * Popularity through usability
  * Popularity through performance
  * Integration with other communities through [[AppStreamDebianProposal | AppStream]]
  * Popularity through usability.
  * Popularity through performance.
  * Integration with other communities through [[AppStreamDebianProposal | AppStream]].
Line 56: Line 57:
  * A consistent configuration file generator for all Debian package metadata tools (''config file generator'')
  * An insightful configuration interface (''config file generator interface'')
  * An efficient and secure acquire logic (''metadata backend'')
  * A parser for the sources.list file, to build the main config file (and others) (''parser'')
  * A generic, extensible model for a plugin - what it handles, how does it handle it, when does the information change (''generic plugin'')
  * Plugins for present tools - apt-get, apt-file, debtags, ... (''specific plugins'')
  * (Possible plugin for [[AppStreamDebianProposal | AppStream]])
  * All of the above would result in '''a powerful apt-get update tool capable of handling all OS package management metadata in a structured and coherent way'''
  * A consistent configuration file generator for all Debian package metadata tools (''config file generator'').
  * An insightful configuration interface (''config file generator interface'').
  * An efficient and secure acquire logic (''metadata backend'').
  * A parser for the sources.list file, to build the main config file (and others) (''parser'').
  * A generic, extensible model for a plugin - what it handles, how does it handle it, when does the information change (''generic plugin'').
  * Plugins for present tools - apt-get, apt-file, debtags, ... (''specific plugins'').
  * (Possible plugin for [[AppStreamDebianProposal | AppStream]]).
  * All of the above would result in '''a powerful apt-get update tool capable of handling all OS package management metadata in a structured and coherent way'''.
Line 67: Line 68:
    * get in touch with the mentor
    * install a local build environment
    * get familiar with the Debian community and development model
    * Get in touch with the mentor.
    * Install a local build environment.
    * Get familiar with the Debian community and development model.
Line 71: Line 72:
    * Debian source code structure
    * C++ is a very powerful language - how much of its cutting-edge features are used by DD, do I need to improve language knowledge cu cope with understanding the code?
    * Security issues - authentication, authorization, types of attacks, data integrity
    * Efficiency issues - responsiveness, bandwidth usage, caching
    * Debian source code structure.
    * C++ is a very powerful language - how much of its cutting-edge features are used by DD, do I need to improve language knowledge to cope with understanding the code?
    * Security issues - authentication, authorization, types of attacks, data integrity.
    * Efficiency issues - responsiveness, bandwidth usage, caching.
Line 76: Line 77:
    * the present package metadata acquire logic
    * t
he format of the configuration files
    * t
he relanshionships between different pieces of metadata
    * t
he Debian Archive format
    * t
he [[AppStreamDebianProposal | AppStream]] specs and metadata
    * The present package metadata acquire logic.
    * T
he format of the configuration files.
    * T
he relanshionships between different pieces of metadata.
    * T
he Debian Archive format.
    * T
he [[AppStreamDebianProposal | AppStream]] specs and metadata.
Line 82: Line 83:
    * configuration model
    * acquire-logic
    * plugins model
    * parser model
    * Configuration model.
    * Acquire-logic.
    * Plugins model.
    * Parser model.
Line 87: Line 88:
   * implement the configuration model and acquire logic - first draft
   * implement the metadata acquire backend - first draft
   * integrate apt-get update with the new metadata acquire backend - assert functionality
   * implement the first supported plugin
   * integrate apt-get update with the backend using this plugin - assert functionality
   * implement sources.list parser - first draft
   * generate configuration file using the parser - assert functionality
   * ''at this point there should be a complet proof of concept upon the functionality of the new model''
   * Implement the configuration model and acquire logic - first draft and unit tests.
   * Implement the metadata acquire backend - first draft and unit tests.
   * Integrate apt-get update with the new metadata acquire backend - assert functionality.
   * Implement the first supported plugin.
   * Integrate apt-get update with the backend using this plugin - assert functionality through integration testing.
   * Implement sources.list parser - first draft.
   * Generate configuration file using the parser - assert functionality.
   * ''At this point there should be a complet proof of concept upon the functionality of the new model and a set of tests for each module''.
Line 96: Line 97:
   * configuration interface for the user
   * s
upport for other plugins
   * s
ources.list parser - final implementation
   * c
onfiguration and acquire logic - final implementation
   * m
etadata acquire backend - final implementation
   * Configuration interface for the user.
   * S
upport for other plugins.
   * S
ources.list parser - final implementation.
   * C
onfiguration and acquire logic - final implementation.
   * M
etadata acquire backend - final implementation.
   * Final testing and refactoring.
Line 102: Line 104:
   * final touches    * Final touches.
Line 106: Line 108:
  * this is not certain, but I'm planning to leave the country for a few days on a trip this summer - a week at most. During this time, I won't be able to code at GSoc.   * This is not certain, but I'm planning to leave the country for a few days on a trip this summer - a week at most. During this time, I won't be able to code at GSoc.
Line 108: Line 110:
  * my first ''real'' contact with Linux - I had a first try with Slackware, but I failed miserably.   * My first ''real'' contact with Linux - I had a first try with Slackware and SUSE, but I didn't find them that easy to learn.
Line 110: Line 112:
  * used worldwide - currently one of the most popular Linux distros (along with Ubuntu, which is basically Debian as well).   * Used worldwide - currently one of the most popular Linux distros (along with Ubuntu, which is basically Debian as well).

Student Application Template

  • Name: Bogdan Purcăreață

  • Contact/Email:

    • IRC: dodgerblue on OFTC, Freenode

    • bogdan.purcareata AT gmail DOT com
  • Background:

    • Senior undergraduate student at the Computer Science and Automatic Control Faculty, Politehnica University of Bucharest
    • Knowledge in C (8 years), as well as C++, Java and Python

    • Knowledge in Algorithm Design, Data Structures and Project Development Workflow

    • Knowledge in Compilers, Operating Systems, Networking, Distributed Systems

    • Highly dependable and efficiency oriented professional
    • Ambitious, focused and enduring individual
    • Familiar with the concepts of Open Source Development - I've pursued an Open Source Development Course[1], organized by ROSEdu[2].
    • Refactoring, optimizing and unifying the metadata acquire system for APT would significantly improve the whole Debian user experience, as well as improve the Package Management System's consistency. To me, this is both a thrilling challenge and a great opportunity to analyze the Debian OS internals and core features.
  • Project title: Pluggable Acquire-System for APT

  • Project details:

    • Debian has developed several tools to manage packages, each one having its own way of handling metadata. This results in a mixed package management system, prone to inconsistencies and loss in overall OS performance. The aim of this project is to build a broad image of the package metadata locally, so all the information is kept in one place and is updated at the same time. The user, by choosing which tools to use, tells the manager what metadata to acquire, therefore how specific a Debian Archive local image he wishes to interact to.
  • Synopsis:

    • Current model:

      • Several tools for package management - apt-get, debtags, apt-file.

      • Each one handles its own set of package metadata, therefore its own view of the remote Debian Archive.
      • Each one requires individual updating and interaction.
    • Desired goals

      • Performance: minimal bandwidth usage by efficient diffs and broad local package metadata.

      • Effectiveness: the user defines which components the acquire system will use, and the system only uses these components.

      • Scalability: the system is pluggable.

      • Forward compatibility: the system will have a generic design, open for future development.

      • Backward compatibility: it is desired that the system doesn't break the existing interfaces, and the transition to the new sistem is as transparent as possible to the user.

    • Proposed Architecture:

PluggableAcquireSystemforAPT-ArchitectureDiagram.png

  • Main components:

    • The Backend:

      • is responsible with fetching the metadata from the Debian Archive, processing it, and retrieving it to apt-get update.

      • implements efficient transport mechanisms.
      • implements security enforcement mechanisms.
      • works according to a configuration file.
    • The parser:

      • reads the sources.list file - now containing information about the enabled / disabled plugins - and sends the sources to the Config File Generator, along with specific settings for each one.
    • The Config File Generator:

      • produces the configuration file for the backend.
      • supports multiple plugin registration: each time a package tool is installed, it registers itself to this component, with a default configuration. The user may change the plugin configuration - as he would have been capable of changing it from the package tool's interface.
      • maps these plugin configurations to the resulting sources provided by the parser.
  • Benefits to Debian OS:

    • Improving the package management system translates into improving the fundamental layer of the Debian OS.
    • Better bandwidth usage.
    • Less configuration and temporary files.
    • Scalability of the metadata acquire system.
    • Better metadata cohesion.
  • Benefits to Debian Community:

    • Popularity through usability.
    • Popularity through performance.
    • Integration with other communities through AppStream.

  • Deliverables:

    • A consistent configuration file generator for all Debian package metadata tools (config file generator).

    • An insightful configuration interface (config file generator interface).

    • An efficient and secure acquire logic (metadata backend).

    • A parser for the sources.list file, to build the main config file (and others) (parser).

    • A generic, extensible model for a plugin - what it handles, how does it handle it, when does the information change (generic plugin).

    • Plugins for present tools - apt-get, apt-file, debtags, ... (specific plugins).

    • (Possible plugin for AppStream).

    • All of the above would result in a powerful apt-get update tool capable of handling all OS package management metadata in a structured and coherent way.

  • Project schedule:

    • April 23 - May 21:

      • Administrative Tasks

        • Get in touch with the mentor.
        • Install a local build environment.
        • Get familiar with the Debian community and development model.
      • General Research

        • Debian source code structure.
        • C++ is a very powerful language - how much of its cutting-edge features are used by DD, do I need to improve language knowledge to cope with understanding the code?
        • Security issues - authentication, authorization, types of attacks, data integrity.
        • Efficiency issues - responsiveness, bandwidth usage, caching.
      • Research State of the Art

        • The present package metadata acquire logic.
        • The format of the configuration files.
        • The relanshionships between different pieces of metadata.
        • The Debian Archive format.
        • The AppStream specs and metadata.

      • Design Tasks

        • Configuration model.
        • Acquire-logic.
        • Plugins model.
        • Parser model.
    • May 21 - July 13

      • Implement the configuration model and acquire logic - first draft and unit tests.
      • Implement the metadata acquire backend - first draft and unit tests.
      • Integrate apt-get update with the new metadata acquire backend - assert functionality.
      • Implement the first supported plugin.
      • Integrate apt-get update with the backend using this plugin - assert functionality through integration testing.
      • Implement sources.list parser - first draft.
      • Generate configuration file using the parser - assert functionality.
      • At this point there should be a complet proof of concept upon the functionality of the new model and a set of tests for each module.

    • July 13 - August 13

      • Configuration interface for the user.
      • Support for other plugins.
      • Sources.list parser - final implementation.
      • Configuration and acquire logic - final implementation.
      • Metadata acquire backend - final implementation.
      • Final testing and refactoring.
    • August 13 - hard deadline

      • Final touches.
  • Exams and other commitments:

    • During this period I'm developing my Bachelor Thesis project, which is expected to be done until de 30th of May. After this date, my main interest will switch to GSoC.
  • Other summer plans:

    • This is not certain, but I'm planning to leave the country for a few days on a trip this summer - a week at most. During this time, I won't be able to code at GSoc.
  • Why Debian?:

    • My first real contact with Linux - I had a first try with Slackware and SUSE, but I didn't find them that easy to learn.

    • Open Source - powerful in learning new technologies and meeting new professionals.
    • Used worldwide - currently one of the most popular Linux distros (along with Ubuntu, which is basically Debian as well).
    • C/C++ - my first programming playground, and the one I'm most familiar with.
  • Are you applying for other projects in SoC? No.