Differences between revisions 124 and 127 (spanning 3 versions)
Revision 124 as of 2020-06-28 00:33:03
Size: 20096
Editor: donfede
Comment: update reprepro documentation URLs
Revision 127 as of 2020-12-04 19:39:06
Size: 20428
Editor: ?bauen1
Comment: Update outdated link to securing debian manual
Deletions are marked like this. Additions are marked like this.
Line 26: Line 26:
 * [[http://www.debian.org/doc/manuals/securing-debian-howto/ch7.en.html#s-deb-pack-sign|Package signing in Debian]] (DDP: Securing Debian Manual)
 * [[http://www.debian.org/doc/manuals/debian-reference/ch02.en.html#_debian_package_management_internals|Debian package management internals]] (DDP: Debian Reference)
 * [[https://www.debian.org/doc/manuals/securing-debian-manual/deb-pack-sign.en.html|Package signing in Debian]] (DDP: Securing Debian Manual)
 * [[https://www.debian.org/doc/manuals/debian-reference/ch02.en.html#_debian_package_management_internals|Debian package management internals]] (DDP: Debian Reference)
Line 104: Line 104:
   * [[http://ftp.ports.debian.org/debian-ports/|unofficial Debian ports]] ([[https://anonscm.debian.org/cgit/debian-ports/mini-dak.git/|mini-dak.git]])    * [[http://ftp.ports.debian.org/debian-ports/|unofficial Debian ports]] ([[https://salsa.debian.org/debian-ports-team/mini-dak|mini-dak.git]])
Line 126: Line 126:
  * Setting up your own automatic Debian repository see [[http://www.debian-administration.org/articles/286|this article on reprepro]]. The link is rather outdated, but still contains some useful information.
  * Setting up reprepro with apache2 and sign key (in italian language) [[http://www.fabreg.it/reprepro-repository-debian-43/]]
  * [[http://scotbofh.wordpress.com/2011/04/26/creating-your-own-signed-apt-repository-and-debian-packages/|Creating your own Signed APT Repository and Debian Packages]] by Jon Cowie
  * Setting up your own automatic Debian repository see [[https://debian-administration.org/article/286/Setting_up_your_own_APT_repository_with_upload_support|this article on reprepro]]. The link is rather outdated, but still contains some useful information.
  * Setting up reprepro with apache2 and sign key (in italian language) [[https://web.archive.org/web/20190427031534/http://www.fabreg.it/reprepro-repository-debian-43/]]
  * [[https://scotbofh.wordpress.com/2011/04/26/creating-your-own-signed-apt-repository-and-debian-packages/|Creating your own Signed APT Repository and Debian Packages]] by Jon Cowie
Line 150: Line 150:
  * [[http://upsilon.cc/~zack/blog/posts/2009/04/howto:_uploading_to_people.d.o_using_dput/|dput + mini-dinstall to ease people.d.o uploads]] by Stefano Zacchiroli.
   * This guides you to setup mini-dinstall on [[http://people.debian.org|people.debian.org]]. It supports [[SecureApt|secure apt]] and remote signing.
  * [[https://upsilon.cc/~zack/blog/posts/2009/04/howto:_uploading_to_people.d.o_using_dput/|dput + mini-dinstall to ease people.d.o uploads]] by Stefano Zacchiroli.
   * This guides you to setup mini-dinstall on [[https://people.debian.org|people.debian.org]]. It supports [[SecureApt|secure apt]] and remote signing.
Line 179: Line 179:
  * [[http://web.archive.org/web/20050410000129/http://vipie.studentenweb.org/dev/debarchiver/|debarchiver how-to]].
  * [[http://foss.stat.unipd.it/mediawiki/index.php/Debian_Mirror_Setup|Italian howto]] for local Debian package mirroring (similar to apt-proxy).
 * An [[http://debian.wgdd.de/debian/|example of a repository]] [[http://debian.wgdd.de/repository|produced with debarchiver]].
  * [[https://web.archive.org/web/20050410000129/http://vipie.studentenweb.org/dev/debarchiver/|debarchiver how-to]].
  * [[https://web.archive.org/web/20060510205006/http://foss.stat.unipd.it/mediawiki/index.php/Debian_Mirror_Setup|Italian howto]] for local Debian package mirroring (similar to apt-proxy).
Line 184: Line 183:
Line 336: Line 336:
=== laniakea ===

 * Source: [[https://github.com/lkorigin/laniakea|GitHub]]
Line 403: Line 407:
 * [[https://wiki.debian.org/Derivatives/CensusFull?action=fullsearch&context=180&value=title%3ADerivatives%2FCensus%2F+regex%3A%5B%5E%23%5D%5Cs%5C*%5CsArchive%5Cstool.%5Cs%5B%5E%2F*%5D&fullsearch=Text|Archive tools used by Debian derivatives]]

Translation(s): 한국어(Korean)


This document summarizes the process of setting up a Debian package repository. It does not describe the format of a Debian repository.

Care has been taken to provide the most accurate information at the time of writing. Please fix any identified mistakes.

Debian Repository Types

There are 2 kinds of repositories from user's perspective:

archive style

apt line

apt-pinning

secure APT

status

official archive

"deb http://example.org/debian unstable main"

Yes

Yes

preferred

trivial archive

"deb http://example.org/debian ./"

No

Yes

deprecated

These archives have different meta-data structure. Both archives can store actual package files. Many older repository HOWTOs (e.g. old "Debian Reference (sarge)" and "APT HOWTO (sarge)") address creation of a "trivial archive" and are problematic since the "trivial archive" lacks support for apt-pinning meta-data used by APT Preferences due to the collision of 2 types of Release files.

For the secure APT compatibility, the modern package archive must be signed by GPG.

References:

Repository instructions

The best practices for creating repository usage instructions are available in RepositoryInstructions.

Debian Repository Generation Tools

The full package archive similar to the official archive can be created using:

  • dak
  • mini-dak

The Private Package Archive (PPA) can be created on a web server with a shell account using:

  • reprepro
  • mini-dinstall

people.debian.org is installed with these packages. The PPA archives created on these hosts should only be used for small low-volume experimental archives only.

Do not run high-volume repositories without consulting the host server's maintainer(s).

dak (Debian Archive Kit)

  • Goals: Hosting the official Debian repositories.
  • Pros:
    • It's the official solution
  • Cons:
    • Depends on python and PostgreSQL
    • Lack of documentation
    • Designed for large repositories.
  • Download: git repository.

  • Distributions: not in Debian
  • Dependencies:
    • python
    • postgresql (optional)
  • Automatic repositories: Yes
  • Incoming mechanism: Yes
  • Pools: Yes
  • GPG signing: Yes
  • Inclusion of .deb without .changes: No
  • Several versions of each package: not in the same Packages instance
  • Wiki Page: dak

mini-dak

  • Goals: Hosting new Debian architectures (Partial and lightweight reimplementation of dak in shell script and with no database dependencies)
  • Pros:
    • Easy to setup: edit a config file and run a script to generate the whole structure
    • No database (the pool is the database)
    • All .changes files kept for later possible importing into the master repository
    • Supports mail notifications and does extensive logging
    • Auto package obsoleting
    • Repository snapshotting
    • Supports multiple suites from the Distribution field on the .changes file
    • Additionally supports multipool (splitting each arch into its own pool, to ease partial mirroring)
    • Supports upload ACLs based on gpg public keys
    • Mirror push via ssh
  • Cons:
    • Slow on huge repositories (due to not using a real db mainly)
    • Has been written and tested mainly as a slave archive, so might have some hardcoded stuff which should be fixed to make it work as a master server
    • Still has some quirks to be fixed
  • Download: https://www.hadrons.org/~guillem/debian/mini-dak/

  • Distributions: not in Debian
  • Dependencies: ('grep Requires: *' on the source tree)
    • apt-utils
    • procmail
    • gnupg
    • wget
    • ssh (optional)
    • bzip2 (optional)
    • quinn-diff (optional)
  • Automatic repositories: Yes
  • Incoming mechanism: Yes
  • Pools: Yes
  • GPG signing: Yes
  • Inclusion of .deb without .changes: No
  • Several versions of each package: ??
  • Sites using it:

reprepro

mini-dinstall

debarchiver

  • Goals: Make a simpler version of dak.
  • Pros:
    • Easy to use incoming mechanism - even on remote systems - by using a cron-job
    • Packages can be moved into a distribution by
      1. reading the Distribution value from .changes file or
      2. directly putting the whole package into a distributions-incoming directory.
    • Standard repository (can be pinned)
  • Cons:
    • No Pool-architecture at the moment
    • Some useful checks are missing
    • Cleaning needs to be done manually
  • Package: debarchiver

  • Distributions: oldstable, stable, testing, unstable

  • Dependencies: unstable/devel/debarchiver

    • adduser
    • apt-utils (recommended) | dpkg-dev
    • opalmod (Perl modules)
    • gnupg (optional)
  • Automatic repositories: Yes
  • Incoming mechanism: Yes
  • Pools: No (but suggested somewhere at BTS).

  • GPG signing: Yes (with gnupg, post-Sarge feature).
  • Inclusion of .deb without .changes: ??
  • Several versions of each package: ??
  • HOWTOs:

debpool

  • Goals: Lightweight replacement for dak using a pool layout.
  • Pros:
    • No external dependencies
    • Easy to use incoming mechanism
    • Standard repository (can be pinned)
  • Cons:
    • Not actively maintained since 2008-10-30 (see latest development)

    • no checking of older packages being replaced with new ones
    • no notification of what is going on (no mails when new packages are added)
  • Package: debpool (REMOVED)

  • Distributions: experimental

  • Dependencies:
    • perl
    • gnupg (optional)
  • Automatic repositories: Yes
  • Incoming mechanism: Yes
  • Pools: Yes
  • GPG signing: Yes (with gnupg).
  • Inclusion of .deb without .changes: ??
  • Several versions of each package: ??
  • Wiki page: debpool

DebMarshal

  • Goals: Maintain multiple snapshots from upstream distros, to permit staging.
  • Pros:
    • Fast
    • No database server needed (BerkeleyDB)
  • Cons:
    • Lack of documentation
    • Hasn't been released (No version available, SVN repo has only trunk).
  • Download: http://code.google.com/p/debmarshal/

  • Distributions: not in Debian
  • Automatic repositories: Yes
  • Incoming mechanism: Yes
  • Pools: Yes
  • GPG signing: Yes
  • Inclusion of .deb without .changes: ??
  • Several versions of each package: ??
  • Wiki page: debmartial

  • Presentation: ODP and PDF files

Built by Google for their use.

apt-ftparchive

  • Goals: Superset of dpkg-scanpackages and dpkg-scansources.
  • Pros:
    • Does not rely on any external programs aside from gzip.
    • Creates Release and Contents files without providing *.changes
  • Cons:
    • Can be slow on large repositories, unless the input file (?FileList) is sorted first (the sort command works).

    • ?
  • Package: apt

  • Distributions: oldstable, stable, testing, unstable, experimental

  • Dependencies: unstable/admin/apt-utils

  • Automatic repositories: No (Yes with dupload)
  • Incoming mechanism: No (Yes with custom move cron script with dupload)
  • Pools: Yes
  • GPG signing: No (Yes with dupload with script)
  • Inclusion of .deb without .changes: ??
  • Several versions of each package: ??
  • HOWTOs:

dpkg-scanpackages and dpkg-scansources

  • Goals: Generate Packages index files
  • Pros:
    • Use only bare bone tool provided by dpkg
    • Creates Release and Contents files without providing *.changes with additional scripts.
  • Cons:
    • Cannot create Release or Contents files by themselves
  • Package: dpkg

  • Distributions: oldstable, stable, testing, unstable

  • Dependencies: unstable/utils/dpkg-dev

  • Automatic repositories: No
  • Incoming mechanism: No
  • Pools: No
  • GPG signing: No
  • Inclusion of .deb without .changes: Yes
  • Several versions of each package: Yes
  • mkdebidx is a shell (mksh) script wrapping dpkg-scanpackages, dpkg-scansources, generating Release and Release.gpg files, and producing a nice XHTML/1.1 index (currently only package-centric view, but dist-/suite-centric views planned) of packages in a full, pinnable, repository with multiple dists and suites (only scales up to a hundred or two packages though; with 904 packages in a repository at the employer, 900 of them in one dist+suite, it takes a while to finish but still works). There may be plans to write a FusionForge plugin for repository handling based on this.

Quick instructions to create a trivial local archive with dpkg-scanpackages

We use /var/lib/mydebs as the example location

  • Add this line to /etc/apt/sources.list

      deb file:/var/lib/mydebs ./
  • Create the directory

      mkdir -p /var/lib/mydebs
  • Every time you put new files in the directory, you'll have to scan the directory and update aptitude:

      ( cd /var/lib/mydebs && dpkg-scanpackages . | gzip > Packages.gz )
      aptitude update
  • Now you can install the package normally, using aptitude or apt-get

aptly

  • Goals: manage local repositories, snapshot them and publish
  • Pros:
    • supports multiple versions of package in one repo
    • has supported for mirroring in the same tool
  • Download & Documentation: http://www.aptly.info/

  • Distributions: stable, testing, unstable

  • Source: GitHub

  • Automatic repositories: Yes (?)
  • Incoming mechanism: Yes (inoticoming -> aptly repo include)

  • Pools: Yes
  • GPG signing: Yes
  • Inclusion of .deb without .changes: Yes
  • Several versions of each package: Yes
  • HOWTOs:

debify

local-apt-repository

  • Goal: Create local repository from packages dropped into a certain directory
  • Pros:
    • Simple to use
  • Cons:
    • Repository available only to the local host, not to other machines
  • Distributions: testing, unstable

  • Automatic repositories: Yes
  • Incoming mechanism: Not needed. Packages are added by putting them into a fixed directory
  • Pools: No
  • GPG signing: No
  • Inclusion of .deb without .changes: Yes
  • Several version of each pacakge: ?

dr

  • Goals: managing small Debian repositories
  • Source: GitHub

  • Documentation: wiki

  • Distributions:
  • Automatic repositories:
  • Incoming mechanism:
  • Pools:
  • GPG signing:
  • Inclusion of .deb without .changes:
  • Several versions of each package:
  • HOWTO: https://radek.io/2015/06/21/dr-setup/

laniakea

Debian Repository Mirroring Tools

ftpsync

reprepro for partial mirroring

debmirror

  • Description: Debian partial mirror script, with ftp and package pool support
  • Package: debmirror

apt-mirror

  • Description: APT sources mirroring tool
  • Package: apt-mirror

debpartial-mirror

apt-move

  • Description: Maintain Debian packages in a package pool
  • Package: apt-move

aptly

  • Description: Mirror remote repositories and manage local repositories, take snapshots, merge them, pull packages from one snapshot to another, publish snapshots as repositories
  • Download & Documentation: http://www.aptly.info/

  • Distributions: testing, unstable

  • Source: GitHub

  • Presentation: PDF

anonftpsync (deprecated)

  • Description: The previous official mirroring tool
  • Pros:
    • Ease of use and few dependencies,
    • Bash script
  • Cons:
    • Lacks flexibility
    • Does not implement latest (2012 A.D.) mirror features (deprecated)
  • Download: anonftpsync

  • Package: not in Debian
  • Dependencies: bash, rsync

See also


CategoryPackageManagement | CategoryDeveloper