Setting up the a unofficial repository
for DebianScience users.
The discussion has started from [http://lists.debian.org/debian-science/2006/07/msg00005.html this thread].
Motivation
Currently there are a fair number of repositories of science-related unofficial Debian packages out there. It might make sense to consolidate them into a single site.
Goals
- Permit convenient one-stop "shopping" for unofficial scientific .debs with only one sources.list entry (well, two counting deb-src).
- Better PR for the unofficial .debs. Instead of a dozen different unofficial repositories, now only one would need to be publicized, perhaps with an obvious name like "www.debian-science.org".
- Possibility of more complicated dependency graphs between unofficial packages
- Possibility (someday) to set up a buildd network that can autobuild all the packages for various architectures
- Testbed for packages that ultimately can enter Debian proper
- Convenient unofficial source for packages that can't enter Debian, for whatever reason, as long as the maintainers had permission to distribute them
Resources
Hosting considered
Brett Viren's site (see http://lists.debian.org/debian-science/2006/07/msg00005.html).
Potential contributors
[http://lists.debian.org/debian-science/2006/07/msg00005.html Kevin B. ?McCarty]
[http://lists.debian.org/debian-science/2006/07/msg00008.html Carlo U. Segre] including machines with mips, sparc, powerpc and m68k architectures for building
[http://lists.debian.org/debian-science/2006/07/msg00012.html Nil] hppa machine available for building
[http://lists.debian.org/debian-science/2006/07/msg00006.html Frédéric Lehobey] dak experience, powerpc, sparc and kfreebsd-i386 architectures
[http://lists.debian.org/debian-science/2006/07/msg00007.html Jordan Mantha]
[http://lists.debian.org/debian-science/2006/07/msg00013.html LI Daobing] building i386 and x86-64 packages
[http://lists.debian.org/debian-science/2006/07/msg00015.html Michael Hanke]
Archive tool considered so far
- dak
- reprepro
- mini-dinstall
Organisation of the archive
best way to organize the archive by section?
By fields?
The usual divisions "main contrib non-free" are fine for Debian, but one of the main reasons an unofficial repository is needed is the often-poor state of care to licenses in scientific software that makes them unsuitable for Debian's archive. Probably the only software in "main" in the repo would be either things undergoing testing on their way to the Debian official archive, or Free software that's too obscure to package for Debian. (Kevin B. ?McCarty is thinking of CERN's "patchy" as an example for the latter.)
So Kevin B. ?McCarty was thinking perhaps a division by field makes more sense - "analysis astronomy biology chemistry physics" etc. A typical sources.list line might then look something like
deb http://www.debian-science.org/ physics analysis
Maybe some packages could be made available under more than one field (e.g. ROOT under both physics and analysis)? After all, ROOT and (e.g.) PAW aren't intrinsically physics software (unlike say GEANT), they're just traditionally used by physicists.
What Debian versions will be supported (or what Debian derivatives)?
Michael Hanke maintains some unofficial packages related to experimental psychology and MRI data analysis. From user feedback and the download stats he knows that people seem use my packages with sarge, etch and Ubuntu breezy and dapper more or less equally often. Moreover, a single lab often uses a mixture of the above. Therefore Michael Hanke tries to provide binary packages for all those distributions.
Ubuntu
Michael Hanke knows that some people simply do not care about Ubuntu, but there is obviously a demand and most of the time porting a package to an Ubuntu release is just recompiling it.
What Michael Hanke wants to say is, that he would prefer a repository that provides packages for every distribution and platform that people (maintainers) are willing to support.
It makes sense to Kevin B. ?McCarty to provide both Debian and Ubuntu packages as long as someone can be found to build them. Maybe this could be a job for an automatic buildd network.
Jordan Mantha thinks Ubuntu Science team could perhaps help out. Right now, in Debian proper, he'd say about 1 in 10 packages from Debian need to get tweaked (mostly dependencies) to work on Ubuntu. Otherwise the Debian source packages are just rebuilt. If this idea takes off he'd imagine they could get some Ubuntu build machines. He's not sure if it would work (or even be wanted) to put them all in the same repo though.
What are the requirements a package has to meet to be included in the repository (e.g. license)?
If a package is perfect in any sense it could obviously go directly into the Debian archive. Therefore the repository will contain imperfect packages and the question is what kind of imperfection is tolerated (lintian error, minor/major licensing issues, ...)?
Relaxation on licensing restrictions
Kevin B. ?McCarty expects that the main relaxation would be on licensing restrictions. (Not so relaxed that we might get sued of course!) As long as someone could get permission from upstream for the repository (and its mirrors?) to distribute a package, it could be put in the archive. This would help in the case of things like Pythia where the author doesn't bother to give his work a specific license, but might be willing to permit redistribution in .deb format if the source code is unchanged.
Stuff that has too little demand per unit filesize
Kevin B. ?McCarty thinks that the unofficial repository could include stuff that has too little demand per unit filesize to go into Debian proper even if freely licensed. This might include obscure specialized programs like CERN's Patchy, or large but specialized data files like those of GEANT4 or the HIPPARCOS astronomical catalogs.
Package quality
Ought to remain high
In Kevin B. ?McCarty opinion the quality of the packaging ought to remain high, for the sake of the reputation of the repository as a whole. This could therefore be a good opportunity to teach packaging skills.
Who will be able to upload packages?
If only DDs are able to upload packages the number of contributors is (unecessarily?) limited. But if the Debian-science repository aims to provide the same quality and security as the main archive, there is no way around it.
More open than the Debian archive
everybody gets upload rights
This is simple, but might be the source of serious trouble.
a procedure similar to Alioth
Potential contributers explain what they want to provide and get upload rights if they provide a solid explanation. From that point on they have the right to upload new packages, but not to upload new versions of packages already in the archive where they are not (co-)maintainers. DDs might be an exception of the rule. This should not limit the number of contributors and introduces a minimal protection against bad guys.
The main disadvantage is that somebody has to implement this.
Kevin B. ?McCarty thinks this sounds very reasonable. He'd be scared to have anyone able to upload anything. It might even be worthwhile to implement a NEW queue of some kind to make sure that new uploads meet some minimum standard of quality.
people who build packages for the less common platforms
Every maintainer who contributes to the repository can't be expected to have a full set of {Debian, Ubuntu} x {stable, testing, unstable} x {i386, amd64, powerpc...} machines to build on. So there will have to be people who build packages for the less common platforms (either by admining a buildd, or else just manually by request as some have already volunteered for). These people will need to be highly trusted.
other best practices
Jordan Mantha suggests looking at other projects such as mentors.debian.net or even Ubuntu's REVU system might help.
Alternative propositions
Web page with sources.list lines for all the known repositories
Russell Shaw suggests an easier way might be just to make a web page that has all the necessary sources.list lines for all the known repositories, so the user can just paste them into their sources.list and apt-get update. If the page was a wiki, newly found repositories could be added easily.
a package doing this
Another way would be to make a debian package that updates your sources.list with new sources.list lines for unofficial repositories. That way, you could do: apt-get install debian-science, and the package installation script will run apt-get update for you.
debian-unofficial
http://www.debian-unofficial.org ?
Mario Fux thinks that's exactly what we are searching.
not really
Not really. Because:
Debian Unofficial should be *only* a repository for three types of packages, which:
1. do violating allegedly software-patents 2. requires a contract with the vendor prior distribution 3. too restrictive licenses (e.g. keeping sources 6 month after distribution etc.)
Nothing a all (using existing infrastructures)
Daniel Baumann sees no reason to not upload any dfsg-compliant package, whetever it is science related or not, to Debian directly. If the package is upstream buggy, then it should be uploaded to experimental, rather than an unofficial repository.
If we are lacking sponsors, tell Daniel Baumann and he can help with sponsoring packages.
Apart from autobuilding issue Carlo U. Segre agrees with Daniel Baumann that putting the packages in the main distribution is the best policy.
but non-free autobuilding?
The biggest problem with non-free packages, which many science-related packages would fall into, is that there is no automatic buildd system for them. Carlo U. Segre has been trying to get pgplot5 rebuilt with no success.