Differences between revisions 52 and 53
Revision 52 as of 2011-04-09 12:18:43
Size: 12847
Editor: OsamuAoki
Comment:
Revision 53 as of 2011-04-09 13:00:05
Size: 12857
Editor: OsamuAoki
Comment: reordered with popcon and usage
Deletions are marked like this. Additions are marked like this.
Line 37: Line 37:


=== reprepro (formerly known as mirrorer) ===
 * Goals: Local Debian package repository storing files in a pool/ directory.
 * Pro: Strict checking of what comes in, no database server needed.
 * Cons: ?
 * Package: http://packages.qa.debian.org/r/reprepro.html
 * Distributions: [[http://packages.debian.org/stable/utils/reprepro|stable]], [[http://packages.debian.org/testing/utils/reprepro|testing]], [[http://packages.debian.org/unstable/utils/reprepro|unstable]] and [[http://packages.debian.org/etch-backports/reprepro|etch-backports]]
 * Dependencies: http://packages.debian.org/unstable/utils/reprepro
 * Automatic repositories: Yes
 * Incoming mechanism: Yes
 * Pools: Yes
 * GPG signing: Yes
 * Manual: [[http://alioth.debian.org/scm/viewvc.php/*checkout*/mirrorer/docs/manual.html?revision=HEAD&root=mirrorer|Yes]]
 * HOWTO: Setting up your own automatic Debian repository see [[http://www.debian-administration.org/articles/286|this arcticle on reprepro]]. The link is rather outdated, but still contains some useful information.
 * HOWTO: Setting up reprepro with apache2 and sign key(in italian language) [[http://www.biziowp.it/reprepro-repository-debian-329/]]

=== debpool ===
 * Goals: Lightweight replacement for dak using a pool layout.
 * Pro:
   * No external dependencies.
   * easy to use incoming mechanism
   * standard repository (can be pinned)
 * Cons:
   * only available from experimental
   * not actively maintained since 2008-10-30 (see [[http://git.debian.org/?p=debpool/debpool.git|latest development]])
   * no checking of older packages being replaced with new ones
   * no notification of what is going on (no mails when new packages are added)
 * Package: http://packages.qa.debian.org/d/debpool.html
 * Distributions: [[http://packages.debian.org/experimental/devel/debpool|experimental]]
 * Dependencies: http://packages.debian.org/experimental/devel/debpool
  * perl
  * gnupg (optional)
 * Automatic repositories: Yes
 * Incoming mechanism: Yes
 * Pools: Yes
 * GPG signing: Yes (with gnupg).
 * Wiki page: [[debpool]]

=== debarchiver ===
 * Goals: Make a simpler version of dak.
 * Pro:
  * easy to use incoming mechanism - even on remote systems - by using a cron-job
  * packages can be moved into a distribution by
   1. reading the Distribution value from .changes file or
   1. directly putting the whole package into a distributions-incoming directory.
  * standard repository (can be pinned)
 * Cons:
  * no Pool-architecture at the moment
  * some useful checks are missing
  * cleaning needs to be done manually
 * Package: http://packages.qa.debian.org/d/debarchiver.html
 * Distributions: [[http://packages.debian.org/oldstable/devel/debarchiver|oldstable]], [[http://packages.debian.org/stable/devel/debarchiver|stable]], [[http://packages.debian.org/unstable/devel/debarchiver|testing]], [[http://packages.debian.org/unstable/devel/debarchiver|unstable]]
 * Dependencies: http://packages.debian.org/unstable/devel/debarchiver
  * adduser
  * apt-utils (recommended) | dpkg-dev
  * opalmod (Perl modules)
  * gnupg (optional)
 * Automatic repositories: Yes
 * Incoming mechanism: Yes
 * Pools: No (but suggested somewhere at [[http://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=debarchiver|BTS]]).
 * GPG signing: Yes (with gnupg, post-Sarge feature).
 * A [[http://vipie.studentenweb.org/dev/debarchiver/|debarchiver how-to]]. An other nice [[http://debian.wgdd.de/debhowtos/howto-aptrep.de.html|debarchiver how-to (in German)]]. An [[http://foss.stat.unipd.it/mediawiki/index.php/Debian_Mirror_Setup|Italian howto]] for local Debian package mirroring (similar to apt-proxy).
 * An [[http://debian.wgdd.de/debian/|example of a repository]] [[http://debian.wgdd.de/repository|produced with debarchiver]].

=== mini-dinstall ===
 * Goals: Miniature version of dak.
 * Pro:
   * Doesn't require a PostgreSQL database.
   * small footprint
 * Package: http://packages.qa.debian.org/m/mini-dinstall.html
 * Distributions: [[http://packages.debian.org/stable/devel/mini-dinstall|stable]], [[http://packages.debian.org/testing/devel/mini-dinstall|testing]], [[http://packages.debian.org/unstable/devel/mini-dinstall|unstable]]
 * Dependencies: http://packages.debian.org/unstable/devel/mini-dinstall
  * apt-utils
  * python2.3
  * python-apt
 * Automatic repositories: Yes (?)
 * Incoming mechanism: Yes
 * Pools: No
 * GPG signing: Yes (external script and setup example provided in documentation)

=== apt-ftparchive ===
 * Goals: Superset of dpkg-scanpackages and dpkg-scansources.
 * Pro: Does not rely on any external programs aside from gzip. Creates Release and Contents files.
 * Cons:
 * Package: http://packages.qa.debian.org/a/apt.html
 * Distributions: [[http://packages.debian.org/oldstable/admin/apt-utils|oldstable]], [[http://packages.debian.org/stable/admin/apt-utils|stable]], [[http://packages.debian.org/testing/admin/apt-utils|testing]], [[http://packages.debian.org/unstable/admin/apt-utils|unstable]], [[http://packages.debian.org/experimental/admin/apt-utils|experimental]]
 * Dependencies: http://packages.debian.org/unstable/admin/apt-utils
 * Automatic repositories: No (Yes with dupload)
 * Incoming mechanism: No (Yes with custom move cron script with dupload)
 * Pools: Yes
 * GPG signing: No (Yes with dupload with script)

 * HOWTOs:
  * apt-ftparchive generate [[http://familiasanchez.net/~roberto/howtos/debrepository|Roberto Sanchez how-to]] -- he now recommend to use '''reprepro'''
  * [[http://www.debian.org/doc/manuals/debian-reference/ch02.en.html#_small_public_package_archive|Debian Reference (lenny)]] -- SecureAPT in combination with dupload (aimed at someone who has shell access to a web server)

=== dpkg-scanpackages and dpkg-scansources ===
 * Goals:
 * Pro:
 * Cons: Cannot create Release nor Contents files.
 * Package: http://packages.qa.debian.org/d/dpkg.html
 * Distributions: [[http://packages.debian.org/oldstable/utils/dpkg-dev|oldstable]], [[http://packages.debian.org/stable/utils/dpkg-dev|stable]], [[http://packages.debian.org/testing/utils/dpkg-dev|testing]], [[http://packages.debian.org/unstable/utils/dpkg-dev|unstable]]
 * Dependencies: http://packages.debian.org/unstable/utils/dpkg-dev
 * Automatic repositories: No
 * Incoming mechanism: No
 * Pools: No
 * GPG signing: No

 * HOWTO: [[http://www.debian.org/doc/manuals/repository-howto/repository-howto.en.html|Aaron Isotton how-to]]
Line 184: Line 74:
=== reprepro (formerly known as mirrorer) ===
 * Goals: Local Debian package repository storing files in a pool/ directory.
 * Pro: Strict checking of what comes in, no database server needed.
 * Cons: ?
 * Package: http://packages.qa.debian.org/r/reprepro.html
 * Distributions: [[http://packages.debian.org/stable/utils/reprepro|stable]], [[http://packages.debian.org/testing/utils/reprepro|testing]], [[http://packages.debian.org/unstable/utils/reprepro|unstable]] and [[http://packages.debian.org/etch-backports/reprepro|etch-backports]]
 * Dependencies: http://packages.debian.org/unstable/utils/reprepro
 * Automatic repositories: Yes
 * Incoming mechanism: Yes
 * Pools: Yes
 * GPG signing: Yes
 * Manual: [[http://alioth.debian.org/scm/viewvc.php/*checkout*/mirrorer/docs/manual.html?revision=HEAD&root=mirrorer|Yes]]
 * HOWTO: Setting up your own automatic Debian repository see [[http://www.debian-administration.org/articles/286|this arcticle on reprepro]]. The link is rather outdated, but still contains some useful information.
 * HOWTO: Setting up reprepro with apache2 and sign key(in italian language) [[http://www.biziowp.it/reprepro-repository-debian-329/]]

=== mini-dinstall ===
 * Goals: Miniature version of dak.
 * Pro:
   * Doesn't require a PostgreSQL database.
   * small footprint
 * Package: http://packages.qa.debian.org/m/mini-dinstall.html
 * Distributions: [[http://packages.debian.org/stable/devel/mini-dinstall|stable]], [[http://packages.debian.org/testing/devel/mini-dinstall|testing]], [[http://packages.debian.org/unstable/devel/mini-dinstall|unstable]]
 * Dependencies: http://packages.debian.org/unstable/devel/mini-dinstall
  * apt-utils
  * python2.3
  * python-apt
 * Automatic repositories: Yes (?)
 * Incoming mechanism: Yes
 * Pools: No
 * GPG signing: Yes (external script and setup example provided in documentation)

=== apt-ftparchive ===
 * Goals: Superset of dpkg-scanpackages and dpkg-scansources.
 * Pro: Does not rely on any external programs aside from gzip. Creates Release and Contents files.
 * Cons:
 * Package: http://packages.qa.debian.org/a/apt.html
 * Distributions: [[http://packages.debian.org/oldstable/admin/apt-utils|oldstable]], [[http://packages.debian.org/stable/admin/apt-utils|stable]], [[http://packages.debian.org/testing/admin/apt-utils|testing]], [[http://packages.debian.org/unstable/admin/apt-utils|unstable]], [[http://packages.debian.org/experimental/admin/apt-utils|experimental]]
 * Dependencies: http://packages.debian.org/unstable/admin/apt-utils
 * Automatic repositories: No (Yes with dupload)
 * Incoming mechanism: No (Yes with custom move cron script with dupload)
 * Pools: Yes
 * GPG signing: No (Yes with dupload with script)

 * HOWTOs:
  * apt-ftparchive generate [[http://familiasanchez.net/~roberto/howtos/debrepository|Roberto Sanchez how-to]] -- he now recommend to use '''reprepro'''
  * [[http://www.debian.org/doc/manuals/debian-reference/ch02.en.html#_small_public_package_archive|Debian Reference (lenny)]] -- SecureAPT in combination with dupload (aimed at someone who has shell access to a web server)

=== debarchiver ===
 * Goals: Make a simpler version of dak.
 * Pro:
  * easy to use incoming mechanism - even on remote systems - by using a cron-job
  * packages can be moved into a distribution by
   1. reading the Distribution value from .changes file or
   1. directly putting the whole package into a distributions-incoming directory.
  * standard repository (can be pinned)
 * Cons:
  * no Pool-architecture at the moment
  * some useful checks are missing
  * cleaning needs to be done manually
 * Package: http://packages.qa.debian.org/d/debarchiver.html
 * Distributions: [[http://packages.debian.org/oldstable/devel/debarchiver|oldstable]], [[http://packages.debian.org/stable/devel/debarchiver|stable]], [[http://packages.debian.org/unstable/devel/debarchiver|testing]], [[http://packages.debian.org/unstable/devel/debarchiver|unstable]]
 * Dependencies: http://packages.debian.org/unstable/devel/debarchiver
  * adduser
  * apt-utils (recommended) | dpkg-dev
  * opalmod (Perl modules)
  * gnupg (optional)
 * Automatic repositories: Yes
 * Incoming mechanism: Yes
 * Pools: No (but suggested somewhere at [[http://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=debarchiver|BTS]]).
 * GPG signing: Yes (with gnupg, post-Sarge feature).
 * A [[http://vipie.studentenweb.org/dev/debarchiver/|debarchiver how-to]]. An other nice [[http://debian.wgdd.de/debhowtos/howto-aptrep.de.html|debarchiver how-to (in German)]]. An [[http://foss.stat.unipd.it/mediawiki/index.php/Debian_Mirror_Setup|Italian howto]] for local Debian package mirroring (similar to apt-proxy).
 * An [[http://debian.wgdd.de/debian/|example of a repository]] [[http://debian.wgdd.de/repository|produced with debarchiver]].

=== dpkg-scanpackages and dpkg-scansources ===
 * Goals:
 * Pro:
 * Cons: Cannot create Release nor Contents files.
 * Package: http://packages.qa.debian.org/d/dpkg.html
 * Distributions: [[http://packages.debian.org/oldstable/utils/dpkg-dev|oldstable]], [[http://packages.debian.org/stable/utils/dpkg-dev|stable]], [[http://packages.debian.org/testing/utils/dpkg-dev|testing]], [[http://packages.debian.org/unstable/utils/dpkg-dev|unstable]]
 * Dependencies: http://packages.debian.org/unstable/utils/dpkg-dev
 * Automatic repositories: No
 * Incoming mechanism: No
 * Pools: No
 * GPG signing: No

 * HOWTO: [[http://www.debian.org/doc/manuals/repository-howto/repository-howto.en.html|Aaron Isotton how-to]]

=== debpool ===
 * Goals: Lightweight replacement for dak using a pool layout.
 * Pro:
   * No external dependencies.
   * easy to use incoming mechanism
   * standard repository (can be pinned)
 * Cons:
   * only available from experimental
   * not actively maintained since 2008-10-30 (see [[http://git.debian.org/?p=debpool/debpool.git|latest development]])
   * no checking of older packages being replaced with new ones
   * no notification of what is going on (no mails when new packages are added)
 * Package: http://packages.qa.debian.org/d/debpool.html
 * Distributions: [[http://packages.debian.org/experimental/devel/debpool|experimental]] --- removed
 * Dependencies: http://packages.debian.org/experimental/devel/debpool
  * perl
  * gnupg (optional)
 * Automatic repositories: Yes
 * Incoming mechanism: Yes
 * Pools: Yes
 * GPG signing: Yes (with gnupg).
 * Wiki page: [[debpool]]

This document summarises setting up a Debian package repository.

I have taken care to provide the most accurate information at the time of writing but should you find any mistakes, please fix them.

There are 2 kinds of repositories from user's perspective:

archive style

apt line

apt-pinning

secure APT

trivial archive

"deb http://example.org/debian ./"

No

Yes

official archive

"deb http://example.org/debian unstable main"

Yes

Yes

These have different meta-data structure, but both store actual package files. Many repository HOWTOs address creation of a "trivial archive". These are problematic since the "trivial archive" lacks support for apt-pinning and modern secure APT due to the collision of 2 types of Release files. (e.g., old "Debian Reference (sarge)" and "APT HOWTO (sarge)")

Even with an "official archive", you can create a much simpler archive than the real official one. This is explained in Debian Reference (lenny) using apt-ftparchive in apt-utils and dupload. All uploaded packages are located in a directory and no database server is needed. This may be good enough for people hosting a few packages.

For creating something similar to the official archive, there are some good packages to help you but they tend to require a database server.

The following sections contain more info about these applications.

Available Tools

dak (Debian Archive Kit)

  • Goals: Packaging of the tools handling the official Debian repositories.
  • Pro: Real stuff.
  • Cons: Depends on python and PostgreSQL (even if on an other host), lack of documentation, designed for large repositories.
  • Download: git repository.

  • Distributions: not in Debian
  • Dependencies:
    • python
    • postgresql (optional)
  • Automatic repositories: Yes
  • Incoming mechanism: Yes
  • ?Pools: Yes

  • GPG signing: Yes
  • Wiki Page: dak

mini-dak

  • Goals: Partial and lightweight reimplementation of dak in shell script and with no database dependencies, "designed" to host new Debian architectures.
  • Pro:
    • easy to setup: edit a config file and run a script to generate the whole structure
    • no database (the pool is the database)
    • all .changes files kept for later possible importing into the master repository
    • supports mail notifications and does extensive logging
    • auto package obsoleting
    • repository snapshotting
    • supports multiple suites from the Distribution field on the .changes file
    • additionally supports multipool (splitting each arch into its own pool, to ease partial mirroring)
    • supports upload ACLs based on gpg public keys
    • mirror push via ssh
  • Cons:
    • slow on huge repositories (due to not using a real db mainly)
    • has been written and tested mainly as a slave archive, so might have some hardcoded stuff which should be fixed to make it work as a master server
    • still has some quirks to be fixed
  • Download: http://www.hadrons.org/~guillem/debian/mini-dak/

  • Distributions: not in Debian
  • Dependencies: ('grep Requires: *' on the source tree)
    • apt-utils
    • procmail
    • gnupg
    • wget
    • ssh (optional)
    • bzip2 (optional)
    • quinn-diff (optional)
  • Automatic repositories: Yes
  • Incoming mechanism: Yes
  • ?Pools: Yes

  • GPG signing: Yes
  • Sites using it:

reprepro (formerly known as mirrorer)

mini-dinstall

apt-ftparchive

debarchiver

dpkg-scanpackages and dpkg-scansources

debpool

  • Goals: Lightweight replacement for dak using a pool layout.
  • Pro:
    • No external dependencies.
    • easy to use incoming mechanism
    • standard repository (can be pinned)
  • Cons:
    • only available from experimental
    • not actively maintained since 2008-10-30 (see latest development)

    • no checking of older packages being replaced with new ones
    • no notification of what is going on (no mails when new packages are added)
  • Package: http://packages.qa.debian.org/d/debpool.html

  • Distributions: experimental --- removed

  • Dependencies: http://packages.debian.org/experimental/devel/debpool

    • perl
    • gnupg (optional)
  • Automatic repositories: Yes
  • Incoming mechanism: Yes
  • Pools: Yes
  • GPG signing: Yes (with gnupg).
  • Wiki page: debpool

DebMarshal

  • Goals: Maintain multiple snapshots from upstream distros, to permit staging.
  • Pro: Fast, no database server needed (BerkeleyDB).
  • Cons: Lack of documentation. Hasn't been released (No version available, SVN repo has only trunk).
  • Download: http://code.google.com/p/debmarshal/

  • Distributions: not in Debian
  • Automatic repositories: Yes
  • Incoming mechanism: Yes
  • Pools: Yes
  • GPG signing: Yes

Built by Google for their use.

  • netselect selects the fastest mirrors from a list you give, and netselect-apt does the same from all existing mirrors.
  • apt-spy does something similar with a different method.
  • dput uploads one or more Debian packages into a repository.
  • parse-apt-files.inc PHP-script by Jarno Elonen produces a nice XHTML-summary of available packages in a repository - enhanced version for special usage with (but not limited to) debarchiver. There seems to be some efforts to develop a wordpress-plugin based on these scripts.

  • mkdebidx is a shell (mksh) script wrapping dpkg-scanpackages, dpkg-scansources, generating Release and Release.gpg files, and producing a nice XHTML/1.1 index (currently only package-centric view, but dist-/suite-centric views planned) of packages in a full, pinnable, repository with multiple dists and suites (only scales up to a hundred or two packages though; with 904 packages in a repository at the employer, 900 of them in one dist+suite, it takes a while to finish but still works). There may be plans to write a FusionForge plugin for repository handling based on this.

HowTos

How to setup a mini-dinstall repository on people.debian.org

See also


CategoryPackageManagement