This document summarizes the process of setting up a Debian package repository. It does not describe the format of a Debian repository.
Care has been taken to provide the most accurate information at the time of writing. Please fix any identified mistakes.
Contents
- Debian Repository Types
- Repository instructions
- Debian Repository Generation Tools
- Debian Repository Mirroring Tools
- Related software
- See also
Debian Repository Types
There are 2 kinds of repositories from user's perspective:
archive style |
apt line |
apt-pinning |
secure APT |
status |
official archive |
"deb http://example.org/debian unstable main" |
Yes |
Yes |
preferred |
trivial archive |
"deb http://example.org/debian ./" |
No |
Yes |
deprecated |
These archives have different meta-data structure. Both archives can store actual package files. Many older repository HOWTOs (e.g. old "Debian Reference (sarge)" and "APT HOWTO (sarge)") address creation of a "trivial archive" and are problematic since the "trivial archive" lacks support for apt-pinning meta-data used by APT Preferences due to the collision of 2 types of Release files.
For the secure APT compatibility, the modern package archive must be signed using OpenPGP.
References:
Package signing in Debian (DDP: Securing Debian Manual)
Debian package management internals (DDP: Debian Reference)
Repository instructions
The best practices for creating repository usage instructions are available in RepositoryInstructions.
Debian Repository Generation Tools
The full package archive similar to the official archive can be created using:
- dak
- mini-dak
The Private Package Archive (PPA) can be created on a web server with a shell account using:
- reprepro
- mini-dinstall
people.debian.org is installed with these packages. The PPA archives created on these hosts should only be used for small low-volume experimental archives only.
Do not run high-volume repositories without consulting the host server's maintainer(s).
dak (Debian Archive Kit)
- Goals: Hosting the official Debian repositories.
- Pros:
- It's the official solution
- Cons:
- Depends on python and PostgreSQL
- Lack of documentation
- Designed for large repositories.
Download: git repository.
- Distributions: not in Debian
- Dependencies:
- python
- postgresql (optional)
- Automatic repositories: Yes
- Incoming mechanism: Yes
- Pools: Yes
- OpenPGP signing: Yes
- Inclusion of .deb without .changes: No
- Several versions of each package: not in the same Packages instance
Wiki Page: dak
mini-dak
- Goals: Hosting new Debian architectures (Partial and lightweight reimplementation of dak in shell script and with no database dependencies)
- Pros:
- Easy to setup: edit a config file and run a script to generate the whole structure
- No database (the pool is the database)
- All .changes files kept for later possible importing into the master repository
- Supports mail notifications and does extensive logging
- Auto package obsoleting
- Repository snapshotting
- Supports multiple suites from the Distribution field on the .changes file
- Additionally supports multipool (splitting each arch into its own pool, to ease partial mirroring)
- Supports upload ACLs based on OpenPGP public keys
- Mirror push via ssh
- Cons:
- Slow on huge repositories (due to not using a real db mainly)
- Has been written and tested mainly as a slave archive, so might have some hardcoded stuff which should be fixed to make it work as a master server
- Still has some quirks to be fixed
- Distributions: not in Debian
- Dependencies: ('grep Requires: *' on the source tree)
- apt-utils
- procmail
- gnupg
- wget
- ssh (optional)
- bzip2 (optional)
- quinn-diff (optional)
- Automatic repositories: Yes
- Incoming mechanism: Yes
- Pools: Yes
- OpenPGP signing: Yes
- Inclusion of .deb without .changes: No
- Several versions of each package: ??
- Sites using it:
reprepro
- (formerly known as mirrorer)
- Goals: Local Debian package repository storing files in a pool/ directory.
- Pros:
- Strict checking of what comes in, no database server needed.
- Cons:
- Lots of different options and commands, difficult to use for beginners.
Package: reprepro
Distributions: stable, testing, unstable and squeeze-backports
Dependencies: unstable/utils/reprepro
- Automatic repositories: Yes
- Incoming mechanism: Yes
- Pools: Yes
- OpenPGP signing: Yes
- Inclusion of .deb without .changes: Yes
- Several versions of each package: not in the same Packages instance
Manual: Yes
- HOWTOs:
Short HOWTO (official)
Setting up your own automatic Debian repository see this article on reprepro. The link is rather outdated, but still contains some useful information.
Setting up reprepro with apache2 and sign key (in italian language) https://web.archive.org/web/20190427031534/http://www.fabreg.it/reprepro-repository-debian-43/
Creating your own Signed APT Repository and Debian Packages by Jon Cowie
mini-dinstall
- Goals: Miniature version of dak.
- Pros:
- Doesn't require a PostgreSQL database.
- Small footprint
- Cons:
- ?
Package: mini-dinstall
Dependencies: unstable/devel/mini-dinstall
- apt-utils
- python
- python-apt
- Automatic repositories: Yes (?)
- Incoming mechanism: Yes
- Pools: No
- OpenPGP signing: Yes (external script and setup example provided in documentation)
- Inclusion of .deb without .changes: No
- Several versions of each package: Yes (see keep_old option)
- HOWTO:
dput + mini-dinstall to ease people.d.o uploads by Stefano Zacchiroli.
This guides you to setup mini-dinstall on people.debian.org. It supports secure apt and remote signing.
debarchiver
- Goals: Make a simpler version of dak.
- Pros:
- Easy to use incoming mechanism - even on remote systems - by using a cron-job
- Packages can be moved into a distribution by
- reading the Distribution value from .changes file or
- directly putting the whole package into a distributions-incoming directory.
- Standard repository (can be pinned)
- Cons:
- No Pool-architecture at the moment
- Some useful checks are missing
- Cleaning needs to be done manually
Package: debarchiver
Dependencies: unstable/devel/debarchiver
- adduser
- apt-utils (recommended) | dpkg-dev
- opalmod (Perl modules)
- gnupg (optional)
- Automatic repositories: Yes
- Incoming mechanism: Yes
Pools: No (but suggested somewhere at BTS).
- OpenPGP signing: Yes (with gnupg, post-Sarge feature).
- Inclusion of .deb without .changes: ??
- Several versions of each package: ??
- HOWTOs:
Italian howto for local Debian package mirroring (similar to apt-proxy).
debpool
- Goals: Lightweight replacement for dak using a pool layout.
- Pros:
- No external dependencies
- Easy to use incoming mechanism
- Standard repository (can be pinned)
- Cons:
Not actively maintained since 2008-10-30 (see latest development)
- no checking of older packages being replaced with new ones
- no notification of what is going on (no mails when new packages are added)
Package: debpool (REMOVED)
Distributions: experimental
- Dependencies:
- perl
- gnupg (optional)
- Automatic repositories: Yes
- Incoming mechanism: Yes
- Pools: Yes
- OpenPGP signing: Yes (with gnupg).
- Inclusion of .deb without .changes: ??
- Several versions of each package: ??
Wiki page: debpool
DebMarshal
- Goals: Maintain multiple snapshots from upstream distros, to permit staging.
- Pros:
- Fast
- No database server needed (BerkeleyDB)
- Cons:
- Lack of documentation
- Hasn't been released (No version available, SVN repo has only trunk).
Download: http://code.google.com/p/debmarshal/
- Distributions: not in Debian
- Automatic repositories: Yes
- Incoming mechanism: Yes
- Pools: Yes
- OpenPGP signing: Yes
- Inclusion of .deb without .changes: ??
- Several versions of each package: ??
Wiki page: debmartial
Presentation: ODP and PDF files
Built by Google for their use.
apt-ftparchive
- Goals: Superset of dpkg-scanpackages and dpkg-scansources.
- Pros:
- Does not rely on any external programs aside from gzip.
- Creates Release and Contents files without providing *.changes
- Cons:
Can be slow on large repositories, unless the input file (?FileList) is sorted first (the sort command works).
- ?
Package: apt
Distributions: oldstable, stable, testing, unstable, experimental
Dependencies: unstable/admin/apt-utils
- Automatic repositories: No (Yes with dupload)
- Incoming mechanism: No (Yes with custom move cron script with dupload)
- Pools: Yes
- OpenPGP signing: No (Yes with dupload with script)
- Inclusion of .deb without .changes: Yes
- Several versions of each package: Yes
- HOWTOs:
- Debian Reference (published around 2008 in lenny, this content is dropped in the current version) -- SecureAPT in combination with dupload (aimed at someone who has shell access to a web server)
apt-ftparchive generate Roberto Sanchez how-to -- he now recommends to use reprepro
Quick instructions to create a trivial local archive with apt-ftparchive
We use /var/lib/mydebs as the example location
Create the directory
mkdir -p /var/lib/mydebs
Every time you put new files in the directory, you'll have to scan the directory and update:
cd /var/lib/mydebs apt-ftparchive packages . > Packages apt-ftparchive release . > Release
Add this line to /etc/apt/sources.list
deb [allow-insecure=yes] file:/var/lib/mydebs ./
Now you can install the package normally. If apt-get asks "Install these packages without verification?", answer "Y" to install. That's because this local repository is not signed.
apt-get update apt-get install mydeb
dpkg-scanpackages and dpkg-scansources
- Goals: Generate Packages index files
- Pros:
- Use only bare bone tool provided by dpkg
- Creates Release and Contents files without providing *.changes with additional scripts.
- Cons:
- Cannot create Release or Contents files by themselves
Package: dpkg
Dependencies: unstable/utils/dpkg-dev
- Automatic repositories: No
- Incoming mechanism: No
- Pools: No
- OpenPGP signing: No
- Inclusion of .deb without .changes: Yes
- Several versions of each package: Yes
mkdebidx is a shell (mksh) script wrapping dpkg-scanpackages, dpkg-scansources, generating Release and Release.gpg files, and producing a nice XHTML/1.1 index (currently only package-centric view, but dist-/suite-centric views planned) of packages in a full, pinnable, repository with multiple dists and suites (only scales up to a hundred or two packages though; with 904 packages in a repository at the employer, 900 of them in one dist+suite, it takes a while to finish but still works). There may be plans to write a FusionForge plugin for repository handling based on this.
Quick instructions to create a trivial local archive with dpkg-scanpackages
We use /var/lib/mydebs as the example location
Add this line to /etc/apt/sources.list
deb file:/var/lib/mydebs ./
Create the directory
mkdir -p /var/lib/mydebs
Every time you put new files in the directory, you'll have to scan the directory and update aptitude:
( cd /var/lib/mydebs && dpkg-scanpackages . | gzip > Packages.gz ) aptitude update
- Now you can install the package normally, using aptitude or apt-get
aptly
- Goals: manage local repositories, snapshot them and publish
- Pros:
- supports multiple versions of package in one repo
- has supported for mirroring in the same tool
Download & Documentation: http://www.aptly.info/
Source: GitHub
- Automatic repositories: Yes (?)
Incoming mechanism: Yes (inoticoming -> aptly repo include)
- Pools: Yes
- OpenPGP signing: Yes
- Inclusion of .deb without .changes: Yes
- Several versions of each package: Yes
- HOWTOs:
aptly tutorials: https://www.aptly.info/tutorial/
Example of setup with incoming: http://oar.imag.fr/wiki:aptly
debify
- Goal: takes a directory full of Debian packages and creates a signed and properly-structured Debian repository (wrapper around aptly)
Download: https://hub.docker.com/r/spotify/debify/ (docker container)
Source: GitHub
local-apt-repository
- Goal: Create local repository from packages dropped into a certain directory
- Pros:
- Simple to use
- Cons:
- Repository available only to the local host, not to other machines
- Automatic repositories: Yes
- Incoming mechanism: Not needed. Packages are added by putting them into a fixed directory
- Pools: No
- OpenPGP signing: No
- Inclusion of .deb without .changes: Yes
- Several version of each pacakge: ?
dr
- Goals: managing small Debian repositories
Source: GitHub
Documentation: wiki
- Distributions:
- Automatic repositories:
- Incoming mechanism:
- Pools:
- OpenPGP signing:
- Inclusion of .deb without .changes:
- Several versions of each package:
laniakea
Source: GitHub
Pulp
Supports various packaging formats via plugins, including one for Debian.
Website: https://pulpproject.org/
Debian Repository Mirroring Tools
ftpsync
This is THE RECOMMENDED MIRRORING TOOL for public mirror sites.
- Goals: The official archive mirroring tool
Instruction: Setting up a Debian archive mirror
git repository: git clone https://ftp-master.debian.org/git/archvsync.git
- Mirroring with the rsync protocol is strongly recommend.
reprepro for partial mirroring
- Description: Debian package repository producer
Package: reprepro
- HOWTOs:
debmirror
- Description: Debian partial mirror script, with ftp and package pool support
Package: debmirror
apt-mirror
- Description: APT sources mirroring tool
Package: apt-mirror
debpartial-mirror
- Description: tools to create partial Debian mirrors
Package: debpartial-mirror
apt-move
- Description: Maintain Debian packages in a package pool
Package: apt-move
aptly
- Description: Mirror remote repositories and manage local repositories, take snapshots, merge them, pull packages from one snapshot to another, publish snapshots as repositories
Download & Documentation: http://www.aptly.info/
Source: GitHub
Presentation: PDF
anonftpsync (deprecated)
- Description: The previous official mirroring tool
- Pros:
- Ease of use and few dependencies,
- Bash script
- Cons:
- Lacks flexibility
- Does not implement latest (2012 A.D.) mirror features (deprecated)
Download: anonftpsync
- Package: not in Debian
- Dependencies: bash, rsync
Related software
netselect selects the fastest mirrors from a list you give, and netselect-apt does the same from all existing mirrors.
apt-spy does something similar with a different method.
dput and dupload: uploads one or more Debian packages into a repository.
parse-apt-files.inc PHP-script by Jarno Elonen produces a nice XHTML-summary of available packages in a repository - enhanced version for special usage with (but not limited to) debarchiver. There seems to be some efforts to develop a wordpress-plugin based on these scripts.
See also
CategoryPackageManagement | CategoryDeveloper | CategoryPackaging