Differences between revisions 2 and 3
Revision 2 as of 2006-01-01 07:23:36
Size: 30797
Editor: AnthonyTowns
Comment:
Revision 3 as of 2006-01-01 07:24:38
Size: 59376
Editor: AnthonyTowns
Comment:
Deletions are marked like this. Additions are marked like this.
Line 360: Line 360:
14:17 <mlots> Aren't most SCC's not sensible until they become part of the archi 14:17 <mlots> Aren't most SCC's not sensible until they become part of the archive proper?
14:17 <neuro> shouldn't that be done before adding it to the archive?
14:17 <aj> nchip: no, they need to be sensible before being in the archive; that includes working out combinattions of kernel/ABI and assigning them to debian architectures in order to minimise the number of architectures, and also (hopefully) getting a stable ABI that won't need to be changed in future
14:18 <mlots> aj: Does Hurd, sparc etc meet that?
14:18 <aj> in so far as hurd doesn't meet it, it's questionable whether it should be in the archive
14:19 <aj> i don't think sparc's had any major abi issues; certainly it's avoided needing a sparc64 port in parallel (or a sparcel/sparceb port, etc)
14:19 <mlots> aj: It's a working port, it's useful to it's developers, it's useful to it's users.
14:20 <mlots> aj: SCC shouldn't need A
14:20 <mlots> ABI stability
14:20 <nchip> neuro: I guess that depends on who does the work of maintaining SCC archive servers.
14:20 <mlots> That's the point of having snapshots right?
14:21 <aj> mlots: useful to users ==> ABI stability. ABI instability ==> screwing your users over
14:21 <mlots> The ABI only has to be stable long enough to compile the relevant parts of the port's archive (granted, for c libaries that's most/all of it).
14:21 <aj> mlots: that's fine for development, but not for use
14:21 <nchip> neuro: but it would be a bit appalling to expect each porters to maintain the infrastructure outside debian
14:21 <mlots> aj: If your users are developers...
14:22 <aj> mlots: then your port isn't ready
14:22 <DavidS> uclibcs two last releases were a year apart
14:22 <mlots> aj: for scc?
14:22 <mlots> aj: Debian tools make it easier to work on a port.
14:22 <aj> mlots: yes; SCC == letting in other arches of at least m68k/hurd's standard; not lowering it further
14:22 <mlots> aj: I'd say API stability is more important.
14:23 <aba> mlots: look at e.g. amd64 - there are ports outside the main archive. Why not?
14:23 <aj> mlots: that's fine, dak and debbugs are free software, you can setup your own while you stabalise your software
14:23 <aba> aj: well, even bugs.d.o could be used to a large extend :)
14:23 <aj> mlots: API stability is hopefully a given, there's the C standard and all
14:24 <mlots> aj: so no SCC until a port can theoretically have two snapshots where one can upgrade from one to the other?
14:24 <aj> mlots: yes, definitely
14:25 <mlots> Should there be a requirement that users!=developers? (I hate to ask, but that seems to be the implication of what aj's saying).
14:25 * aj makes that explicit
14:26 <aj> even if your users are developers, you shouldn't screw them over with an unstable ABI
14:26 <Greek0> mlots: if you say all your users are the port developers, why do you want it to be an official debian arch (scc or not) already?
14:26 <mlots> aj: are current accepted ports following users!=developers for their user counts?
14:26 <Greek0> I mean, an alioth project or something should be enough anyway for that period of time
14:27 <mlots> Greek0: Infastructure.
14:27 <aba> mlots: all ports reviewed by the release team had more than enough users anyways
14:27 <mlots> Greek0: Didn't amd64 have problems using alioth?
14:27 <Ganneff> mlots: the size of it, yes.
14:27 <aba> mlots: and it was moved out due to too much space usage
14:27 <aj> mlots: if you're so wonderful that everyone of your millions of users learns C and contributes back patches, that's not a black mark
14:28 <aj> mlots: but if you then break the ABI for them, that is
14:28 <mlots> Gannelf: So isn't that a problem for other ports?
14:28 <Ganneff> mlots: only if they built all of debian, in testing and unstable and experimental
14:29 <mlots> Ganneff: so three ports building one of those would be too much too?
14:29 <aj> mlots: Stabilising your ABI isn't that tricky; at worst you just have to provide backwards compatability as Linux and glibc do
14:29 <Ganneff> mlots: maybe.
14:30 <neuro> making ftp-master run out of space for the port instead isn't really a win, either...
14:30 <mlots> So third class citizen's might use alioth, lists.d.o, perhaps bugs.d.o?
14:30 <aj> unstable ABIs also screw over mirrors and buildds
14:30 <braindmg> aj: well even linux cannot provide backwards compat in all cases
14:30 <aj> i'm not convinced arches that can't provide a stable ABI should be using shared resources at all
14:30 <Ganneff> mlots: or any other machine out there.
14:31 <Ganneff> mlots: if you have a machine and enough bandwith you just need the tools to run an archive.
14:31 <aj> braindmg: absolute perfection isn't a requirement; if you meet or exceed Linux's standards, you're fine
14:31 <Ganneff> mlots: and that is packaged stuff.
14:31 <aj> i'm sorry, Linux/i386's standards
14:32 <aj> i can't understand why anyone would want to try maintaining a port that randomly changes its exec format or libc regularly
14:32 <Yoe> aj: you'll only meet that rule for Linux/i386 anyhow
14:32 <Yoe> aj: other ports are by definition less tested
14:32 <Yoe> than the most popular hardware architecture in the world
14:33 <aj> Yoe: not breaking an ABI is easy -- don't remove stuff, only add to it
14:33 <aj> Yoe: bugs are an entirely different matter
14:33 <Yoe> aj: oh, right, missed that bit
14:33 <Greek0> mlots: I don't think it's that terrible difficult to setup the infrastructure you need compared to the other work to do for a successful port. and AIUI debian's infrastructure is currently quite near to being overloaded, so using that resources just because it's easier isn't probably the best idea
14:33 <mlots> aj: If you're stable per snapshot then there's use.
14:34 <aj> mlots: if you're stable permanently you can use it, you can upgrade it, and you can do 3rd party development on it
14:35 <mlots> aj: I think the key argument is upgrade.
14:35 <mlots> 3rd party development could be done against a snapshot.
14:35 <aj> it would have to be done against /every/ snapshot, is the problem
14:36 <mlots> aj: If snapshots are as far apart as releases (even at 18 months
14:36 <mlots> that might not be a problem.
14:36 <aj> just fix your abi once and for all
14:37 <Yoe> mlots: I thought the idea is for snapshots to /not/ be so far away from eachother
14:37 <aj> mlots: btw, who are you anyway?
14:37 <nyu> wrt archive size problems, have you seen http://www.linuks.mine.nu/sizematters/ ? it claims we can save 29.3 % of archive space by using p7zip in .deb format
14:37 <mlots> Did anyone have any comments about my installer comment?
14:37 <mlots> aj: Drew Scott Daniels
14:38 <mlots> nyu: Youc an save about 0.3% by reording the files inside the debs.
14:38 <nyu> interesting
14:38 <nyu> you mean in the tar?
14:38 <Greek0> nyu: as it was pointed out on debian-devel, p7zip is not an option as long as dpkg in stable doesn't support it. so we could only use it for etch+1
14:38 <mlots> nyu: yup. I've got code that proves it for the linux kernel.
14:38 <Yoe> mlots: there's a serious difference between 29.3% and 0.3%
14:39 <mlots> ppmd is better than p7zip for compression. See compression.ca or maximumcompression (.info?)
14:39 <Yoe> though of course the reordering doesn't require dpkg code changes to be able to unpack later on
14:39 <nyu> Greek0: right. i'm on it (i sent patch to tar already)
14:39 <nyu> mlots: URL?
14:39 <nyu> ah
14:39 <mlots> nyu: for my code?
14:40 <nyu> mlots: is it free?
14:40 <mlots> nyu: I haven't released my code yet.
14:40 <mlots> nyu: ppmd is in Debian.
14:41 <nyu> i'll have a look, thanks
14:42 <Q_> mlots: ppmd also doesn't work on 64 bit arches.
14:42 <mlots> aj: Is an arch specific installer good enough for first class?
14:43 <mlots> Q_: Yup. It could be ported though.
14:43 <nyu> aj: and if not, is it for second? :)
14:43 <mlots> The trouble with switching to 7zip or ppmd etc is that it might raise the mimimum system requirements.
14:45 <nyu> i think an i486 is still faster at uncompressing than average internet connection at downloading
14:45 <mlots> nyu: how about CD?
14:45 <nyu> CD access is usualy faster when the data you're reading is compressed, too
14:45 <mlots> nyu: See compression.ca's speed vs size comparison.
14:45 <aj> mlots, nyu: it needs to be a working installer, otherwise i don't see a problem
14:45 <Q_> nyu: So what is an average internet connection nowadays?
14:45 <nyu> maybe not for i486 though
14:47 <aj> you could always include p7zip and a (minimally) updated dpkg in a point release
14:47 <Q_> (And it doesn't download and decompress at the same time anyway.)
14:47 <mlots> More compression does not nessiarily mean more time needed to decompress.
14:48 <nyu> mlots: any idea how to contact this guy? FLAC is missing in the audio comparison
14:48 <mlots> It'd be worth considering p7zip's decompression requirements.
14:48 <mlots> nyu: Use his e-mail address or try it yourself.
14:49 <mlots> nyu: maximumcompression was more up to date last time I checked.
14:49 <nyu> no public email it seems
14:49 <mlots> I read a review showing that FLAC sucks compaired to some general purpose compressers.
14:49 <nyu> i'll dig around
14:50 <Greek0> aj: hmm. is this compatible with the "support updates from oldstable -> stable" policy? I mean what if someone installed sarge, never updated (ok.. probably unlikely and surely dumb, but anyway), and then wanted to update to etch when it's released?
14:50 <nyu> mlots: the sound comparison is not for general-purpose
14:50 <mlots> nyu: whois compression.ca?
14:50 <nyu> mlots: ah
14:50 <nyu> mlots: really? where's that
14:50 <Q_> Greek0: So they should first upgrade to sarge, then to etch.
14:50 <aj> Q_: they first upgrade to sarge's dpkg, and install p7zip
14:51 <mlots> Greek0: I think we violated that for woody->sarge.
14:51 <mlots> nyu: maximumcompression.info or something.
14:51 <Greek0> Q_: i.e. to "sarge+stable-security+stable-proposed-updates" or something like this?
14:51 <Q_> aj: You're saying we support upgrading from stable - 2 to stable?
14:51 <aj> i don't think we've got that sort of policy anyway -- we don't do security updates for people still running oldstable at that point, eg
14:52 <mlots> Greek0: I think that's the case for point releases only.
14:53 <aj> Q_: not particularly, but in this case how to do it by hand is pretty easy
14:54 <mlots> aj: Could a SCC or first class citizen be not "real", but embeded?
14:55 <nyu> mlots: i can't see any reference to 7zip there
14:56 <mlots> nyu: the files should be available for compression.ca so you could give it a try.
14:56 <pkern> nyu: "7-zip"?
14:56 <aj> mlots: i'm not sure what it would mean for an embedded architecture to be a release candidate
14:56 <aj> mlots: though udebs essentially make up a parallel set of "embedded" architectures
14:58 <mlots> aj: I guess it could be given that it'd meet the built requirement through not-for-us?
14:58 <mlots> aj: I'm not talking about udebs.
14:58 <nyu> pkern: www.7-zip.org, p7zip.sf.net
14:59 <nyu> btw, http://www.7-zip.org/ claims 7z outperforms ppmd (but that source is not impartial)
14:59 <aj> mlots: i'm not sure it wouldn't make more sense to have a different "built requirement" for "embedded" arches
15:00 <nyu> http://www.compression.ru/ybs/e&docs.htm
15:00 <mlots> nyu: what about paq5? There's new compressers all the time.
15:01 <mlots> aj: I'd say properly using not-for-us makes more sence than lowering the % requirement.
15:01 <mlots> aj: less buildd time wasted.
15:02 <aj> doing lots of not-for-us when the package could build is cheating for the archive %ge requirement
15:03 <aj> but for an embedded arch, building a certain set of packages might still be a useful requirement; i don't really know
15:03 <aj> until we've got an arch that it makes sense to consider "embedded" instead of "real", there's not much point trying to go into any detail
15:03 <mlots> nyu: If a case for a new compresser is made, one needs to consider: - space savings, - decompression time, -decompression memory requirements, - somewhat reasonable compression requirements such that devs are willing to use it.
15:03 <mlots> aj: m86k?
15:04 <mlots> 'r m68k?
15:04 <Yoe> mlots: we're not going to do that any time soon
15:04 <nyu> mlots: right
15:04 <nyu> mlots: but first, you have to consider patching dpkg ;)
15:05 <nyu> s/dpkg/tar/g
15:05 <aj> dpkg has already been patched for bz2 aiui; p7zip should be trivial
15:05 <nyu> then dpkg
15:05 <Q_> arm* looks more like an arch that might be an embedded one.
15:05 <nyu> aj: yes, but i don't want to send a patch that makes dpkg pipe() itself and exec tar/p7zip separately
15:05 <aj> there are actual arm workstations on the market today though
15:05 <nyu> tar needs fixing first
15:06 <aj> nyu: *shrug* it's not brain surgery
15:06 <Yoe> mlots: if you're interested in the specifics, we'd have to switch to something that works on ColdFire before making it an 'embedded' arch is going to be any useful
15:06 <mlots> nyu: what fixing? file reordering? putting the new compression into tar's code?
15:07 <Yoe> mlots: and once we reach that point, we might as well stick to being a full arch, since coldfire CPUs are much faster than the 'classic' m68k ones
15:07 <vorlon> are arm workstations anything more than glorified devel systems for embedded work, though?
15:08 <Yoe> vorlon: those were intended for being of more use, yes
15:08 <nchip> arm workstation I know of are riscpc replacements
15:08 <vorlon> ok
15:08 <vorlon> (somehow I doubt many people *use* them for anything other than devel systems for embedded work, but. :)
15:09 <nyu> mlots: p7zip support
15:09 <nyu> aj: no, it's more like planting rice
15:09 <nyu> you have to wait till it grows
15:09 <nchip> usually people crosscompile for embedded work. powerpc people are lucky enough they have a excuse to get nice devel systems from apple
15:10 <aj> nyu: huh? it's changing a few dozen lines of code, it's an undergrad exercise, geez.
15:10 <nyu> aj: it's ugly
15:10 <nyu> i want to get tar fixed first
15:10 <aj> that was including changing tar
15:11 <nyu> i have a patch for tar already
15:11 <nyu> the difficult part is getting it applied :)
15:11 <nyu> then send a patch for dpkg and wait again
15:13 <braindmg> aj: so are there plans to remove any arch from the archive?
15:15 <aj> braindmg: yes, sh.
15:15 <Yoe> yeah, that'd make total sense
15:15 <Yoe> is there actually still any binary in the archive which is sh and up-to-date?
15:15 <aj> i don't think there are any sh binaries in the archive, up to date or not
15:16 <aj> it's all arch:all
15:16 <mlots> scc requirements are basically: more useful than an existing scc, or signifigant(?) interest in a scc when there's theoreticaly extra infrastrucure space / man hours available?
15:17 <aj> "scc" isn't really a good term to use
15:17 <aj> requirements for going in the archive are basically: useful, maintained, stable
15:18 <aj> maybe s/maintained/supportable/
15:18 <nchip> "ports" would be quite neutral
15:18 <mlots> stable as in abi, and runtime?
15:19 <Yoe> aj: okay, and when the thing is stable, but would need an ABI update after having been in the archive for years. What's going to happen? It remains there, or it gets kicked out until the ABI's fixed again?
15:19 <braindmg> even linux-i386 may need that eventually
15:20 <aj> linux-i386 has gone through a few ABI changes (a.out to ELF, libc5 to libc6, libc6 to libc6 for that matter), it just maintains backwards compatability
15:21 <aj> if your arch isn't worth the effort to do the ABI change in a backwards compatible manner, you're giving yourself a hard time arguing that it's worth including in the archive
15:21 <aj> cf the hurd
15:21 <Yoe> aj: to get this a bit more specific,
15:21 <braindmg> aj: what about the hurd?
15:21 <Yoe> the example I gave about the ColdFire move earlier isn't just hypothetical
15:22 <Yoe> aj: I've been talking to rwhitby, who works at freescale, to see whether we'd be able to get a few eval boards to port Debian/m68k over to them
15:22 <nchip> aj: debian was *much* smaller and had only i386 then, so a completly library rename made sense.
15:22 <Yoe> they're a factor 5 or so faster than what we currently support
15:23 <aj> nchip: a library rename is how we do these things in a backwards compatible manner
15:23 <Yoe> but their instruction set is smaller, so we'd need to replace everything we have currently before it'll work.
15:23 <nchip> aj: most people are going to object renaming all libs beginning from libc6 only for a minor archs abi transition
15:23 <aj> nchip: so do it just for your arch, working out how to make it easy for everyone *shrug*
15:24 <aj> Yoe: that sounds like it'd be fairly straightforward to do compatibly?
15:25 <nchip> aj: with a new arch name. backward compatability if needed can be solved with multiarch.
15:25 <Yoe> aj: yes, but it might be fairly lot of work, with much binNMUs and such
15:25 <aj> nchip: a new arch name is a horrible idea
15:26 <Yoe> aj: also, because of the instruction set having new instructions as well as losing old ones, I'm not entirely sure it's going to be possible to do this in a backwards-compatible way
15:26 <mlots> aj: I'd like scc infrastructure to grow at the same rate as first class.
15:26 <Yoe> though I can't be sure before I've tested it on the hardware
15:26 <nchip> aj: that's your opinion. I think library rename for one arch is way worse.
15:27 <nchip> aj: new arch name only involves porters of the said arch. library transition affects every debian users and needs handwork for every library maintainer
15:27 <mlots> I'm not just talking about archive space. Also machiens, bandwidth, potentialy man hours...
15:27 <aj> nchip: new arch names involve every single user of the arch, and are a ridiculous waste of disk space and bandwidth, sorry, no.
15:28 <aj> mlots: stop thinking in terms of "scc"
15:28 <aj> mlots: the only actual difference looks likely to be between release candidates and non release candidates for the time being; and the only different infrastructure between those two will be the testing scripts and stable support
15:28 <nchip> aj: then we'll just stop caring about your official debian. sorry.
15:29 <vorlon> aj: only a ridiculous waste of disk space if you keep the old arch around in testing/unstable, which it doesn't seem you would want to do?
15:30 <nchip> ..since we are expected to maintain all the infra outside debian anyway, it does not seem like a big step anyway.
15:30 <mlots> aj: but then there's now the third, those that don't make it into the archive... I get your point though.
15:30 <aj> vorlon: you'd probably need to while the new arch was being repopulated, which would likely take a few weeks, then you've got to worry about supporting the people who aren't in a position to upgrade immediately
15:32 <Greek0> nchip: debian doesn't have infinite resources. I think noone objects your stuff going into the archive if you have enough users, are willing to maintain your port, and it doesn't add a huge burden of another kind on debian.
15:34 <vorlon> aj: would that really be much different, disk-wise, than changing the names of all the libraries, since the latter means recompiling everything and essentially having disparate package sets in testing and unstable for a period?
15:34 <Greek0> but whining about "debian doesn't support us" when you have hardly any users and there seems to be quite a cost involved for debian seems a bit questionable.
15:34 <aj> one of the major concerns is b/w from ftp-master; ABI bumping a port costs about 10-15GB; we usually try to limit pulses to about 2GB per day
15:34 <braindmg> and I can understand that for the mirrors infra but having a new ports (or second class or whatever) ports.debian.org would make life way easier
15:34 <Yoe> Greek0: uhh, Debian/arm does have quite some users, actually
15:35 <aj> vorlon: if you do it compatibly you don't need to do it all at once, and potentially don't need to do it at all
15:35 <aj> having a second set of infrastructure to maintain would make life twice as hard, thanks all the same
15:35 <Greek0> Yoe: is this what nchip is talking about? I didn't really get it what his "pet port" was, sorry
15:36 <nchip> ...potentially don't need to do it at all
15:36 <Yoe> Greek0: I happen to know what nchip works on. That helps :)
15:36 <vorlon> I'm not sure how you'd do it compatibly, though? There's certainly no precedent within Debian of maintaining such compatibility through an ABI transition
15:37 <aj> vorlon: eh? we did with libc5/libc6
15:37 <braindmg> aj: for all arches
15:37 <aj> vorlon: there were some rumours of an hppa ABI change a little while ago, iirc, what happened with it?
15:37 <nchip> 17:22 < nchip> aj: debian was *much* smaller and had only i386 then, so a completly library rename made sense.
15:37 <mlots> aj: I agree, a second set of infrastructure to maintain would make life twice as hard. That's the case with non-release and non-archive.
15:37 <vorlon> Package: libpam0g
15:37 <vorlon> Conflicts: libpam0 (<= 0.56-2), libpam
15:38 <aj> oh, debian didn't have only i386 then, it at least had m68k and i think maybe sparc and alpha?
15:38 <vorlon> aj: ^^^ doesn't seem like the kind of thing that happens piecemeal
15:38 <nchip> hrmh ok. but still it was necessary for all the archs.
15:38 <Greek0> Yoe: hehe. just that "stop caring about official debian" phrase reminded me of the gnu/solaris people (no offense)
15:38 <braindmg> mlots: and replicating the infra and work for each new/external port
15:38 <Yoe> Greek0: heh
15:40 <aj> mlots: no, non-release go with all the other architectures, they just don't release
15:40 <mlots> I can see the annoyance with getting bugs against your package for a port that would take time to maintain.
15:40 <aj> mlots: non-archive stuff just needs to have a stable ABI, which really isn't that hard
15:41 <mlots> aj: non-release wouldn't get the benifit of "testing" or the testing scripts... I guess they'd still be able to grab all the packages after release?
15:43 <aj> mlots: there are other things that could be done beyond unstable and experimental, but i'm not going to talk about that until there's an /actual example/
15:43 <nchip> what archs now have their own external infra? this stuff could benefit from co-operation.
15:43 <mlots> I guess non-release could still have "testing"/"stable" buildd, and security support?
15:43 <aj> at present, the only non-release architecture is hurd-i386, which currently has 4k packages in the archive, compared to ~24k for every other architecture in the archive
15:44 <nyu> Greek0: gnu/solaris is much worse than just "no official repository"
15:44 <mlots> aj: amd64 released sarge after Debian and didn't use the testing script?
15:45 <Q_> amd64 doesn't use britney, which makes it rather hard for us to have a real testing.
15:45 <Ganneff> mlots: we are on a different machine, we dont use britney.
15:45 <Ganneff> we have some own scripts, fetching the data from ftp-master and running a little "mini-britney", basically following i386 testing
15:45 <aj> had we had appropriate criteria in place, and changes to our mirroring, amd64 would've been in the archive and a release candidate years ago
15:45 <Greek0> nyu: I followed the whole tragedy. I also didn't mean to imply that anyone here works on that port or hase the same attitude as the people that do. just that single statement remembered me of this issue.
15:46 <Q_> aj: So what still needs to happen for the mirroring changes?
15:46 <mlots> That's my point. non-release could grab a list of packages from stable or testing and build them later right?
15:46 <braindmg> nchip: amd64 kfreebsd-i386 knetbsd-i386 hurd-i386 at least
15:47 <mlots> That's mini-britney? Or is mini-briteny something else?
15:48 <aj> Q_: work out exactly what mirrors should do; tell mirrors what they should do; wait for mirrors to do that
15:48 <Ganneff> mini-britney is a shell script.
15:48 <nyu> Greek0: they have an attitude? I thought they simply worked in their own, separate world
15:48 <nyu> Greek0: is there a single nexenta patch in bts? :)
15:48 <aj> mlots: if you're going to try mimicing the release stuff, you should just be a release arch. the requirements really aren't that challenging
15:48 <nyu> nchip: and armeb
15:49 <nyu> hurd-i386 is in sid
15:49 <nyu> and knetbsd-i386 is.. well, dead ;)
15:49 <mlots> aj: Doing it on time with limited man hours is a challenge.
15:49 <braindmg> nyu: it needs some stuff for which maintainers have not integrated the patches
15:49 <braindmg> nyu: the infra is there ;>
15:50 <nyu> braindmg: heh yes it is
15:50 <aj> mlots: so get more users, and thus more developers
15:50 <Greek0> nyu: attitude, as in: mindset, behavior, ..
15:50 <Q_> aj: The "work out what mirrors should do" part is rather vague.
15:51 <nyu> Greek0: yes. their behaviour from our POV can be cathegorised as non-behaviour ;)
15:51 <aj> mlots: the only real difficulty is when the hardware's difficult to come by and you can't get users, i guess
15:51 <mlots> aj: creating a "sarge" snapshot for a port is useful as it limits the non-port RC bugs that ones needs to deal with.
15:51 <Q_> They should have some easy script to select which aches they want to have on the mirror?
15:52 <Q_> aj: Your blog also said something about making dak scripts better to handle the load? Can you get into more specifics about that?
15:53 <aj> Q_: apt-ftparchive does too much stat'ing, rather than just using its cache; i think there was something else along those lines too
15:54 <mlots> Well I gotta go to work. Thanks for the discussion. I hope to read the transcript of the rest later. bye
15:54 <aj> Q_: there's already a reasonably easy script, it's more the issue of making it easy for people to find a (eg) m68k mirror when the country mirror only had i386 eg
15:54 <DavidS> aj: how much work would it be to split out non-mirrored .debs into /pool-non-mirrored/ or something, which can be easily excluded by those mirror -ops who don't want it?
15:55 <Q_> aj: Atleast in d-i, it lists which arches are on each mirror.
15:55 <aj> insanely painful, since it'd involve moving GBs of data around in a mirror pulse?
15:55 <braindmg> DavidS: and then people that want to mirror the non-mirrored? ;)
15:56 <aj> anyway, everything gets mirrored by someone, just not most people
15:56 <Q_> aj: And http://www.debian.org/mirror/list seems to list the arches too.
15:56 <DavidS> braindmg: well, nobody who wants to waste disk space should be hindered ;)
15:57 <aj> so are these questions just out of curiousity, or should i be taking down names for my "to be volunteered for mirror maintenance jobs" list?
15:57 <DavidS> aj: and only uploading new packages to the new locations? probably defeats the "simple" approach by too much complexity in programming :(
15:58 <aj> DavidS: well, that's what happened for the move from dists/ -> pool/, but i don't see the point?
15:58 <Q_> aj: Well, both curiousity, and I'd like to help if I can.
15:58 <aj> DavidS: there's already an rsync-based script that'll do partial mirrors, and with some archive support, something like rsync --delay-updates --files-from:indices/arch-i386.lst --delete-after # should work
15:59 <DavidS> aj: just brainstorming. point being: easy separation of seldom downloaded archs for easier capping of mirror disk usage and b/w
16:00 <DavidS> aj: then the whole discussion around "we haven't enough mirror b/w and disk" goes out the chimney, isn't it?
16:00 <aj> how so?
16:01 <aj> mirroring all architectures (which is what we currently expect of pushed mirrors and top level country mirrors) is 160GB or so, an arch-specific mirror including arch:all and source is about 20GB
16:02 <Q_> Seems to be at 164GB now.
16:03 <aj> yick
16:07 <vorlon> we need to get on jvw to remove more stuff
16:08 <nyu> unrar
16:08 * nyu hides
16:09 <aj> hey, how come the linux-kernel-m68k stuff is arch:all?
16:10 <Yoe> aj: hmm? isn't that just the patch you're talking about?
16:10 * Yoe checks
16:10 <aj> hrm, maybe it's just sources
16:11 <Yoe> aj: what's the exact package name? packages.d.o doesn't find linux-kernel-m68k
16:11 <aj> yeah, it was the patch and sources and for other arches too
16:11 <aj> sigh
16:11 <aj> toolchain-source is worse
16:11 <Yoe> right, thought as much
16:11 <Yoe> heh

Log

   1 11:50 <aj> heh
   2 11:50 -!- aj changed the topic of #debian-tech to: Channel info: http://wiki.debian.org/IRC/debian-tech | Now playing: http://lists.debian.org/debian-devel-announce/2005/12/msg00014.html
   3 11:54 <neuro> but it's playing early!
   4 11:58 <fs> hello
   5 11:59 <neuro> Hi!
   6 12:02  * rwhitby represents www.nslu2-linux.org, arm architecture (endian agnostic), installed linux firmware base of over 15K units, 100+ currently with debian installed, working towards integration of nslu2 into debian kernal and installer.
   7 12:02 <aj> hey, so it begins
   8 12:05 <aj> rwhitby: "endian agnostic" ? as in you use arm, don't care about armeb, but could use it if you flipped a coin and it came up tails?
   9 12:06 <neuro> aj: whichever the customer wants, I'd think...
  10 12:06 <neuro> mipsen is the same way for embedded stuff.
  11 12:06 <rwhitby> actually, those 100+ are mostly running armeb at the moment, but the one barrier to using arm(el) (the IXP ethernet driver) has been overcome
  12 12:06 <nchip> aj: long story :) originally it was not agnostic, it worked only properly in bigendian mode
  13 12:07 <rwhitby> The long story is at http://article.gmane.org/gmane.comp.misc.nslu2.linux/10693 if anyone is interested.
  14 12:07 <aj> elmo!
  15 12:09 <intero> hi
  16 12:10 <Q_> Are we waiting for something?
  17 12:10 <aj> not really
  18 12:10 <aj> i'm filling in http://ftp-master.debian.org/archive-criteria.html
  19 12:10 <rwhitby> I figured people would introduce themselves while waiting ...
  20 12:11 <aj> rwhitby: so can you comment on armeb's prospects for being an official port?
  21 12:12 <rwhitby> aj: I would leave that to lennert (who is driving the port).  nslu2 is now endian agnostic since getting the ixp driver working in both LE and BE.
  22 12:13 <Q_> rwhitby: Are there any others that might want to use armeb other than the nslu2?
  23 12:14 <rwhitby> 10% of unstable has been built by the two nslu2 buildd's but we would need more powerful armeb hardware to get ahead of the 95% required.
  24 12:14 <neuro> how fast are nslu2s?
  25 12:14 <aj> 95% is the release requirement, though
  26 12:14 <nchip> Q_: highend intel routing gear
  27 12:15 <rwhitby> Q_: most designs based on the IXDP425 reference board are armeb.  and the high end stuff that nchip is referring to (and lennert uses) requires armeb
  28 12:16 <rwhitby> nslu2's are 133MHz out of the box, but remove a resistor and they are de-underclocked to 266MHz.
  29 12:16 <neuro> and what's the I/O like?  IDE?  udma5?
  30 12:17 <rwhitby> nslu2 is USB to hard disks.  Iomega NAS 100d (our other potential debian target) is IDE.
  31 12:18 <Q_> usb2?
  32 12:19 <aj> okay, so http://ftp-master.debian.org/archive-criteria.html has what i can think of for linux arches
  33 12:19 <rwhitby> Q_: yes.  nslu2 performance is at http://www.nslu2-linux.org/wiki/Info/Performance, nas100d is at http://www.nslu2-linux.org/wiki/NAS100d/Performance
  34 12:20 <aba> aj: the 50 admin criteria is tougher than the 50 user release criteria
  35 12:21 <aj> hrm
  36 12:22 <nyu> it'd be useful to know what is the criteria when there are multiple lists (e.g. popcon and accounts in a shell server), and it's not possible to weed out duplicates
  37 12:22 <aj> i'm tempted to leave it as 50 admins, except popcon probably won't measure that
  38 12:22 <nyu> do we need a single >=50 list?
  39 12:22 <Q_> rwhitby: For the nas100d, what's hda and sda?
  40 12:23 <rwhitby> Q_: hda is internal IDE, sda is external USB2
  41 12:25 <rwhitby> aj: Is a list like http://www.debonaras.org/wiki/Info/OpenDebianSlugUsers acceptable?
  42 12:25 <nchip> what do people think of handling ABI transition with a new arch name?
  43 12:26 <nchip> and accepting such "new" archs to debian official?
  44 12:26 <Mithrandir> nchip: it was mentioned as a possibility which multiarch will give us in the discussions at debconf4, iirc.
  45 12:26 <Q_> nchip: And do what with the old one?
  46 12:27 <aba> nchip: I think best would be to make a seamless migration, if possible.
  47 12:27 <aba> Mithrandir: I would like to see multiarch working before building plans on it
  48 12:27 <nchip> Q_: maintain it as long as it has users who are willing to maintain it
  49 12:28 <aj> nyu: if you've got two lists, you could make a single list using "cat", or at least "sort -u"; so probably just depends on the lists
  50 12:28 <neuro> having users who are willing to maintain it isn't enough, if those users can't deal with all the similar points of release criteria.
  51 12:28 <nchip> aba: agreed, but without multiarch it is not really simple. renaming all libs for a abi change in one arch is clearly even worse..
  52 12:29 <aj> if m68k can get 100 (according to its release qual page on the wiki) i don't see the problem :)
  53 12:29 <nyu> aj: yes, we did that for wiki userlists, but popcon is anonymous
  54 12:29 <nyu> so we can't really tell
  55 12:29 <aba> nchip: in that case, making a new arch is the only option left AFAICS
  56 12:29 <aj> popcon counts machines, so if you're doing popcon + shell accounts, you can just subtract 1
  57 12:29 <aba> (I think you speak about arm's eabi)
  58 12:30 <nyu> uhm right
  59 12:30 <aj> renaming all libs is what we're almost doing for the C++ changes every other month
  60 12:30 <nchip> yes, and mips nubi (althoug I think NUBI is more controversial)
  61 12:30 <aj> we also did it for libc5->6
  62 12:30 <Mithrandir> aj: it would be considerably worse if we had to rename _all_ libs, wouldn't it?
  63 12:30 <nyu> aj: what if a user has a shell account at the server, plus a machine at home?
  64 12:30 <aba> aj: but only the c++-ones, not *all* packages
  65 12:30 <nyu> then he's being counted twice
  66 12:30 <aba> aj: and we even have to rename all packages, if the kernel cannot cope with old and new abi at the same time (which is true for arm IIRC)
  67 12:31 <aj> the main difference is "apt-get dist-upgrade" won't work if you change the arch name, which isn't remotely acceptable for release candidates
  68 12:31 <Q_> aj: But a port can't go and rename binary packages.
  69 12:31 <aj> if you've got an arch that's not remotely releasable yet, rm -rf'ing the port and starting from scratch isn't /completely/ out of the question, i guess
  70 12:31 <aba> aj: well, why not have both variants in parallel for the time of one release, and then kill the old one?
  71 12:31 <neuro> aj: and has happened at least once in the past...
  72 12:32 <aj> nyu: if a user has a workstation at work and a pc at home, he gets counted twice with popcon too
  73 12:32 <nchip> aj: you cant distupgrade a i386 installation to amd64 installation, if you installed i386 originally on your amd64 machine
  74 12:32 <Mithrandir> nchip: people keep dreaming about being able to, though.
  75 12:32 <aba> (anyways, I don't see the arm's eabi will get stable enough very soon)
  76 12:32 <nyu> aj: ah i see.  very nice, then
  77 12:32 <aba> Mithrandir: if the problem is solved for i386->amd64, it should be able to solve it for arm->earm
  78 12:32 <nchip> Mithrandir: what's ubuntu's plans for multiarch (since it probably happens there first, right?)
  79 12:33 <aj> aba: well, why not fix the kernel to support both abis?
  80 12:33 <aba> aj: I'm don't have enough insight to answer to this question.
  81 12:33 <aj> aba: epoching arch names is a horrible, horrible idea
  82 12:33 <aba> aj: but currently, eabi doesn't really run at all, leaving alone the question of "how should a migration look like"
  83 12:34 <aba> aj: frankly speaking, I'd prefer to being able to migrate soft with e.g. multiarch or so. Well, if I look at eabi's speed, multiarch could be ready in time :)
  84 12:34 <nchip> aba: I have a eabi root filesystem that boots upto X, so "doesn't really run" is not true
  85 12:34 <Mithrandir> nchip: I've been thinking about just pushing for a transition to new-style paths before deciding on how to do the dpkg/apt/archive bits of it, since it seems that's how far consensus has stretched so far.
  86 12:35 <aba> nchip: oh, there has been progress? My latest understanding was that there are lots of issues still open.
  87 12:35 <Mithrandir> nchip: and if so, I'd like the changes to happen in Debian first, to not end up with a totally insane merge in ubuntu.
  88 12:35 <aj> okay, so do we have any m68k people around?
  89 12:36 <aba> aj: I've seen Yoe on #d-devel a few minutes ago
  90 12:36 <Q_> Invite Yoe?  He seems to be around.
  91 12:36 <nchip> aba: the eabi bits just have not been in mainline toolchains until now.
  92 12:36 <aj> okay, how about hurd people?
  93 12:37 <aba> .oO(azeem has even the op-bit here)
  94 12:37 <aj> azeem's on holidays
  95 12:38 <aj> okay, so another question is "what should we do about non release candidate ports?"
  96 12:38 <aj> default answer: unstable + experimental + a pat on the head
  97 12:39 <nyu> any chance we can get testing?
  98 12:39 <aba> nyu: I don't see that, as far as you're away from being a release arch
  99 12:39 <aj> sure, if you meet the release criteria
 100 12:39 <neuro> nyu: testing is where the next release is being prepared, since it's not release candidate, testing makes little sense?
 101 12:39 <nyu> ok
 102 12:40 <Q_> Maybe a snapshot of unstable?
 103 12:40 <aj> i mean, there are two options: try to release; or say "releasing isn't appropriate for our arch, ______ would be instead"
 104 12:40 <aba> (it would make sense if there are few things left, and we expect that it would become an release arch soon enough)
 105 12:40 <nyu> yes, sounds reasonable
 106 12:40 <nyu> for kfreebsd-i386 we expect to be release-capable soon, but not yet
 107 12:40 <aj> arm's problem is apparently upstream toolchain support; and some underpoweredness
 108 12:41 <nyu> for unstable, what are the requirements specific to non-Linux ports? (other than license issues)
 109 12:41 <rwhitby> I thought upstream toolchain support was solved for arm (according to the requal wiki page)
 110 12:41 <aj> do any of the arm folks (or armeb folks) think it makes sense to do anything other than fix those problems and be a release arch?
 111 12:41 <kyllikki> aj: the upstream toolchain support is a very red herring
 112 12:41 <neuro> that's a good question.
 113 12:41 <nchip> aj: the upstream toolchain part is not true.
 114 12:42 <aba> aj: on the short term, I think that's the only way (now speaking as someone looking closely at arm*)
 115 12:42 <kyllikki> aj: and all were missing is the open acess box which there is a suitable one sat at black cat networks which elmo has turned down
 116 12:42 <aj> nchip, kyllikki: you should poke mr aba here to update the etch table then
 117 12:43  * aba hides his release manager's hat
 118 12:43 <aj> aba: (the reason you wear the hat is so you can pull it down over your forehead so no one sees the "bloody idiot" branded there)
 119 12:43 <nchip> aj: I sent a mail to debian-release about the issue. can we get a contact address on that page or a bts virtual package?
 120 12:43 <neuro> kyllikki: umm, I haven't seen any request made to DSA.  elmo is not the appropriate email address for DSA requests.  I'm aware of one system pending atm, and it isn't at black cat.
 121 12:43 <rwhitby> kyllikki: there have also been offers of nslu2 boxes with hosting for armel or armeb developer access
 122 12:44 <aj> nchip: just whine at aba
 123 12:44  * aba looks something up now
 124 12:44 <kyllikki> tbh Im getting pissed off that I do all this stuff for the arch and others create issues which wernt there before
 125 12:44 <kyllikki> neuro: email addr to forward the mail from him after the last round of DSA mail?
 126 12:45 <neuro> debian-admin@lists.debian.org is the debian-admin contact address.
 127 12:45 <neuro> debian-admin@d.o if it shouldn't go to local admins
 128 12:46 <aba> nchip: sorry for not fixing it. Indeed, this mail was prepared for some time, and last when I looked, it was not there.
 129 12:47 <aba> nchip: web page updated
 130 12:48 <nchip> aba: thanks. it was our fault too.. we knew binutils was supported, but we where not sure by who officially so that part of page was left empty waiting for confirmation.
 131 12:49 <aba> nchip: well, having a "binutils: yes [looking by whom personally, nchip/2005-...]" would have helped much, because that would have increased chances of asking you before sending out the mail.
 132 12:51 <elmo> kyllikki: umm, I turned down a box that didn't have hosting, not one "sat at black cat"
 133 12:51 <aj> sat on the mat at black cat?
 134 12:52 <kyllikki> elmo: I thought it was made clear to you when you said Sledges place wasnt apropriate noodles and huggie said they would find a home for it
 135 12:53 <elmo> kyllikki: if that's what happened it happened after I was on vacation without internet access (announced on -private); so I haven't even read that email yet, never mind had a chance to turn anything down
 136 12:54 <kyllikki> elmo: you said you were gonna go with a box provided by joey hess?
 137 12:54 <neuro> "find a home" != "found a home, host up and ready for DSA setup?"
 138 12:54 <elmo> kyllikki: if the alternative was a box on sledge's ADSL, yes
 139 12:54 <aba> do we still need a hosting place for that box?
 140 12:55 <elmo> kyllikki: also, bear in mind (and I think I mentioned this in the email) the last time I asked for hosting in the UK, I couldn't get hosting for an arm box a) as a buildd and b) even from BlackCat
 141 12:55 <kyllikki> elmo: shrug, as usual it seems were destined to get wires crossed and make no progress with this box...maybe its cursed?
 142 12:55 <aj> ooo
 143 12:55 <elmo> aba: not if blackcat are happy to host it
 144 12:56 <aj> there's a good question to add: "Is port cursed?"
 145 12:56 <aba> elmo: good. Otherwise, I have 2-4 places in .de/.at where we might have quite good changes
 146 12:56  * aj wonders what ia64's answer would've been
 147 12:56 <kyllikki> elmo: huggie said it was ok
 148 12:56 <neuro> how about the standards that the (lib)C implementations conforms to?
 149 12:57 <Q_> That's for non-linux ports I guess?
 150 12:57  * maswan grumbles about the relevant people at his uni not answering his inquiries about hosting devel machines
 151 12:57 <neuro> yes
 152 12:57 <kyllikki> elmo: if that is acceptable I will get it so its up and powered when everyones back in the new year? atm people are getting inebriated for th new year
 153 12:58 <kyllikki> elmo: and drunk in charge of root login shouldnt be allowed
 154 12:58 <Q_> From what I understand, there are 2 netbsd ports, but they use a different libc?
 155 12:59 <aj> okay, reload, there's some OS-specific questions too
 156 12:59 <elmo> kyllikki: let me catch up on email - there's about 3 different discussions going on about arm machines atm - and get back to you today or tomorrow
 157 12:59 <kyllikki> elmo: fil offered to do the admin too, would that be helpful?
 158 12:59 <elmo> kyllikki: fil's already DSA, that's somewhat redundant?
 159 12:59 <aj> fil's DSA? who's fil?
 160 12:59 <kyllikki> elmo: yeah but he offered to do the setup specificaly if others were too busy
 161 12:59 <elmo> phil hands
 162 13:00 <elmo> kyllikki: if he does, then, great
 163 13:00 <aj> wow, he ircs and everything
 164 13:01 <nyu> Q_: yes, but none of them is active
 165 13:02 <nyu> Q_: (besides, most of the ongoing work on gnu/kfreebsd can be reused for gnu/knetbsd)
 166 13:03 <aj> oh, biarch needs a mention
 167 13:03 <aj> why shouldn't gnu/k*bsd be bi/triarch anyway?
 168 13:03 <nyu> tri?
 169 13:04 <nyu> we have Linux abi emulation, with some caveats.  biarch can be provided if someone works on it
 170 13:04 <aj> free, net, open
 171 13:05 <nyu> abi is different
 172 13:05 <nyu> oh, right
 173 13:05 <nyu> free doesn't emulate the other *bsd afaik
 174 13:05 <nyu> net emulates free
 175 13:05 <mlots__> dragonfly bsd?
 176 13:05 <nyu> i don't know about open
 177 13:06 <aj> the *bsd stuff don't seem to rate on the "What's this provide over upstream *BSD distros" or "What's it provide over Debian Linux?" -- two architectures even moreso
 178 13:06 <nyu> no idea about dragon either.  perhaps kernel interface is so similar we could just provide a "kernel of dragonfly" package instead
 179 13:07 <nyu> aj: http://wiki.debian.org/Debian_GNU/kFreeBSD_why
 180 13:08 <aj> " kFreeBSD offers an alternative in case Linux is branded illegal by the SCO case or other threats."
 181 13:08 <aj> *cough*
 182 13:09 <mlots____> As to the freeness of the win32 port, ReactOS is GPL/LGPL. It'd be useful to have second class status for it when it gets packaged.
 183 13:09 <mlots____> My appologies for the disconnects. I'm not sure what's happening.
 184 13:10 <aj> having a single (Debian GNU) userspace that lets you run either freebsd or netbsd kernel (possibly with different basic tools, firewalling whatever) at your whim would be more plausible
 185 13:10 <aj> mlots____: you're still pretty laggy
 186 13:11 <mlots> This connection seems to be pretty unusable. Sorry aj
 187 13:11 <aba> in fact, I'd like to be able to choose between different kernels (re *bsd). Being able to choose is one of the good things in open source
 188 13:12 <neuro> that's a huge undertaking, however.
 189 13:12 <nyu> aj: doesn't sound easy
 190 13:12 <nyu> although there are plans for it
 191 13:13 <nyu> but not short term
 192 13:13 <nyu> besides, there's only one active *bsd port atm
 193 13:13 <neuro> look at how much work has to be done in say, d-i, to go from 2.4 to 2.6.
 194 13:13 <nyu> heh yes
 195 13:13 <nyu> we have problems with 5.x vs 6.x too
 196 13:14 <neuro> and all of the places we provide glue just by installing a package, which would have to be made to work on the other OS' as well
 197 13:14 <aj> nyu: managing 15G of debs isn't easy either
 198 13:15 <neuro> it also makes packaging more complex, which tends to make it more fragile
 199 13:15 <nyu> aj: i didn't mean it was
 200 13:16 <aj> okay, http://ftp-master.debian.org/archive-criteria.html - updated again
 201 13:17 <neuro> (the patches I've had for !linux archs that break linux are almost at a 1:1 ratio, for example...)
 202 13:17 <nyu> but, again, the other *bsd haven't gathered enough interest to the date for someone to make a functional port.  I don't think you should be so much concerned about the space these ports would take since they don't exist yet
 203 13:17 <nyu> Guillem Jover had a document about kernel-independant userland if you're interested, though
 204 13:18 <aj> saying "yes" to GNU/kFreeBSD serves as a precedent for saying "yes" to GNU/kNetBSD; so before you say the first yes, you need to be clear on why the second one should be "no", or willing to accept both being "yes"
 205 13:20 <nyu> I'm not sure about that.  If GNU/kFreeBSD meets the requirements (whatever they are), and there was a GNU/kNetBSD port, how can one tell beforehand if the second would meet the requirements too?
 206 13:20 <neuro> nyu: the point is in making clear what the requirements are
 207 13:20 <neuro> and why they exist
 208 13:20 <nyu> ah, ok
 209 13:21 <nyu> ah, found it
 210 13:21 <nyu> http://www.hadrons.org/~guillem/debian/ports/gnu-any/
 211 13:21 <aj> nyu: because the alternatives are "whoever asks first wins, even if that's not best for our users" or "whoever asks gets in, even if we can't cope with the load"
 212 13:22 <neuro> you can't, as a distribution, integrate all of the components well if your packages limit themselves to gnu-any, however...
 213 13:22 <nyu> yes
 214 13:22 <nyu> it's a difficult issue
 215 13:22 <nyu> but "gnu-any" is just a long-term plan
 216 13:23 <nyu> it needs huge work to even get hello.c, and hasn't even started yet
 217 13:23 <aj> for k*BSD, gnu-any + a smaller number of apps compiled for the specific kernel might be worthwhile (separated into kfreebsd, knetbsd sections say)
 218 13:23 <nyu> I don't think we should center the discussion on this
 219 13:23 <aj> the biarch question needs to be answered before you get a "yes", though
 220 13:24 <aj> (same as for mip/mips, sh*, ppc/ppc64, etc)
 221 13:25 <nyu> biarch is no problem.  the problem is making a base system that can boot/work with any of them
 222 13:25 <nyu> we can't handle all kernel-specific apps or libraries in a case-by-case basis
 223 13:25 <aj> that doesn't seem very important? have a base system for kfreebsd, a different one for knetbsd, and common apps
 224 13:26 <nyu> uhm
 225 13:26 <nyu> we could use kfreebsd ABI, and then get knetbsd to run *most* of kfreebsd userland
 226 13:27 <nyu> but is that technicaly possible with biarch?
 227 13:27 <nyu> i thought both arches had to be complete
 228 13:27 <aj> everything's technically possible
 229 13:27 <nyu> well, right
 230 13:28 <nyu> then if someone wants to do a knetbsd port, they'd have to:
 231 13:28 <nyu> - extend biarch to support this combination
 232 13:28 <nyu> - build a base system specific to knetbsd, plus kernel-specific apps/libs
 233 13:28 <nyu> - use kfreebsd-i386 packages for the rest
 234 13:28 <nyu> that sounds plausible?
 235 13:29 <aj> if it was going to support netbsd kernels, it'd make sense to call it kbsd-i386
 236 13:29 <nyu> yes
 237 13:30 <nyu> but:
 238 13:30 <nyu> - what about abi emulation for kfreebsd in non-bsd systems?
 239 13:31 <nyu> - kfreebsd-i386 is not necessarily a wrong name even if knetbsd was added, it would describe the kernel abi
 240 13:31 <nyu> (and it wouldn't be less descriptive than, say, i386)
 241 13:31 <aj> those are things you should note down for the related arches question
 242 13:32 <aj> probably doing some analysis of the differences between freebsd ABI, netbsd ABI and potential gnu-bsd ABI
 243 13:33 <nyu> well, at kernel level we use the same abi as upstream
 244 13:33 <nyu> coming up with something new (and incompatible) doesn't sound like a good idea
 245 13:34 <nyu> btw, will there be a public log of this?
 246 13:34 <aj> "We can support NetBSD kernels using the FreeBSD-compatability stuff in the kernel, the disadvantages for NetBSD users are _____" would be helpful
 247 13:34 <aj> yeah, i'll probably stick it up on wiki.d.o
 248 13:34 <nyu> i see
 249 13:35 <nyu> i take it we're expected to go through the list of questions in http://ftp-master.debian.org/archive-criteria.html and send a response?
 250 13:35 <aj> "we can support netbsd well" ==>. "no need for a knetbsd arch" ==>. reassuring
 251 13:35 <aj> "freebsd rocks, who cares about netbsd" ==>. "if we add this will we be adding netbsd next week? _why??_" ==>. concerning
 252 13:36 <aj> put a response up on the wiki as per the release qualification stuff i think
 253 13:36 <aj> i'm not sure the page is done yet, though; any other comments or additions anyone?
 254 13:37 <nyu> alright
 255 13:37 <nyu> would a proof-of-concept package of knetbsd image for kfreebsd-i386 help?
 256 13:37 <nyu> i mean, rather, a base system
 257 13:38 <aj> numbers on how much it sucks compared to a real freebsd and netbsd system would help too
 258 13:39 <nyu> benchmarks?
 259 13:39 <aj> yeah
 260 13:39 <nyu> i don't think it'll be significantly different
 261 13:40 <nyu> but we can do it, yes..
 262 13:42 <mlots> Ugg, bad DI-624... sorry
 263 13:44 <mlots> I was saying that the win32 port could meat DFSG due to ReactOS (as was the original argument). ReactOS is usable these days...
 264 13:44 <mlots> but it's dev's don't consider it stable.
 265 13:45 <Q_> What still needs to happen before the mirror split can happen?
 266 13:45 <mlots> It certainly isn't from a packaging point of view. From a running point of view, sometimes it is.
 267 13:47 <mlots> Sorry if I missed discussion about installers. Fwiw, I think it makes some sence to allow arch specific installers from upstream... for the first part anyway (booting, creating partition, setting up devices...)
 268 13:47 <aj> ah, you're here now?
 269 13:47 <aj> so what's this win32 port and reactos stuff?
 270 13:48 <mlots> I figure it's possible given mingw.
 271 13:48 <mlots> ReactOS can deal with the elf format, I'm not sure if that means running linux apps though.
 272 13:49 <aj> what is reactos?
 273 13:49 <mlots> http://www.reactos.org
 274 13:49 <Ganneff> aj: free nt kernel
 275 13:49 <mlots> It uses wine code for some of the userspace.
 276 13:50 <aj> so effectively a windows clone?
 277 13:50 <nchip> Daniel Rouso is not around but is there any issues with uclibc debian ports?
 278 13:50 <mlots> aj: mostly. They diverge in places. They aim for binary compatibility where it makes sence.
 279 13:51 <neuro> nchip: it would have to have a lot of reasons for why we would want it in addition to the glibc port
 280 13:51 <aj> mlots: eg?
 281 13:52 <mlots> aj: The internal parts of the kernel and low level things can be REactOS specific.
 282 13:53 <aj> mlots: so would there be any point to a win32 port that only ran on reactos, but not on (say) XP? i assume not?
 283 13:54 <mlots> aj: Targeting ReactOS only makes sence. It'd be hard to get upstream support from Microsoft (or at least that's my impression).
 284 13:54 <nchip> neuro: well there is only one technical reason to have uclibc, it uses a low less memory
 285 13:54 <mlots> aj: Think of drivers that can't run on Linux...
 286 13:54 <aba> mlots: why is ReactOS better as kernel than say Linux?
 287 13:54 <mlots> aba: re above. ;-)
 288 13:55 <mlots> aba: and wine has some limitations because of Linux and XWindows.
 289 13:55 <DavidS> re uclibc: it's really nice for embedded systems where space is tight; and it'd be hard to have all necessary packages compiled against glibc _and_ uclibc...
 290 13:55 <neuro> nchip: why does a port of debian make sense for that?  should it be instead of the glibc port, if that's how it's most often used?  what functionality is lost by not using glibc?
 291 13:55 <aj> mlots: think of drivers that can't run on reactos? seems like linux would be more likely to have quality drivers than reactos?
 292 13:56 <mlots> aj: It's hard to quantify that.
 293 13:57 <aj> "wine has limitations because of linux and xwindows" ?
 294 13:57 <aj> reactos + wine is more capable than linux + wine?
 295 13:57 <nchip> neuro: I don't know. 
 296 13:58 <mlots> aj: for userspace, sometimes. There are scheduler issues, graphics limitations, convertion requirements...
 297 13:58 <neuro> nchip: that's the sort of things that would need to be known before making a port of it.
 298 13:58 <aj> interesting
 299 13:58 <DavidS> neuro: see http://uclibc.org/FAQ.html#doesnt_suck for a short paragraph vs glibc
 300 13:59 <aj> mlots: so the obvious other purpose of a win32 port is so people running windows can "apt-get" a nice unix environment; i presume you could do that and support reactos at the same time?
 301 13:59 <mlots> aj: I beleive so. I'm not sure whether PE or ELF is the right way to go.
 302 13:59 <nchip> neuro: I'm not porting to atm. I'm interested in general issues about alternative libc's/userlands on the same kernel/physical arch
 303 14:00 <aj> DavidS, nchip: having a full port (kde, gnome, tetex) would seem insane; having a micro port (like udebs provide) may be interesting, but would need thought
 304 14:00 <mlots> aj: I'd rather just concentrait on ReactOS. Gives people more insentive to develop it.
 305 14:01 <nchip> here's some motivation behind it: https://www.debconf.org/comas/general/proposals/11
 306 14:01 <mlots> aj: where would one draw the line for ulibc Not For Us?
 307 14:01 <DavidS> aj: yes, saving 30MB in the libc and then adding 300MB for KDE makes not much sense
 308 14:01 <aj> mlots: *shrug* that's one of the things that need thought :)
 309 14:02 <nchip> aj: with the same argument compiling kde for m68k is insane.
 310 14:03 <aj> nchip: m68k is (at least currently) a real port, so compiling kde is pretty much expected
 311 14:03 <mlots> aj: Why?
 312 14:03 <DavidS> another problem with uclibc is, that they are never ABI compatible between releases ...
 313 14:03 <aj> mlots: because that's what real ports do -- work the same no matter what hardware they're run on
 314 14:04 <nchip> DavidS: that's a big problem
 315 14:04 <mlots> aj: There's no memory or speed limitations for "real ports"?
 316 14:04 <aj> you could do patches to make "debian/rules binary ULIBC=yes" work for the apps people're actually interested in
 317 14:05 <aj> mlots: "can keep up with building kde" :)
 318 14:05 <mlots> aj: It seems that a port with limited resources (cpu, memory...) would still be useful. e.g. embeded systems.
 319 14:05 <mlots> aj: Perhaps that's  different "class" of ports other than "real" though?
 320 14:05 <aj> mlots: (having the win32 port be usable by people running XP would pass the "is the port likely to be used by anyone?" criterion much more easily than a "must run reactos kernel" port)
 321 14:06 <mlots> aj: Yes, can the two be seperate though?
 322 14:06 <aj> mlots: yes, that would be an "embedded" port instead of a "real" one, which is entirely hypothetical at this point
 323 14:06 <mlots> It's hard to say has upstream support if the users aren't using the thing supported by upstream.
 324 14:06 <aj> mlots: not really -- an XP port would violate the SC; a reactos port might not pass the "useful?" test
 325 14:07 <nchip> aj: with that sentiment, embedded debian does not make much sense. it will be just easier to compile your own distro.
 326 14:07 <DavidS> nchip: libuclibc0 could be parallel-installed in different versions and the new bin-NMU capabilities of the buildd network mitigate the abi troubles
 327 14:07 <aj> nchip: huh?
 328 14:07 <aj> nchip: i'm all for m68k being an "embedded" or some other !real port, it just isn't that currently
 329 14:07 <mlots> nchip: Debian is one of the most used distro's for creating embeded enviromnents (or so I hear). Why not make it more easy?
 330 14:08 <aj> oh, right, debian/rules ym
 331 14:08 <mlots> aj: ReactOS is used by a reasonable sized community right now.
 332 14:09 <mlots> aj: I'd more like to get a win32 port to "second class citizen" at this point.
 333 14:09 <aj> mlots: XP is used by a much bigger community :) i mean, that's not a "no", but it seems silly not to make it useful for XP users if it can be
 334 14:09 <DavidS> nchip: but yes, testing migration would be a pain anyways :)
 335 14:10 <aj> mlots: can't you make XP run ELF binaries or whatever anyway?
 336 14:10 <nchip> mlots: I would like to make it easier
 337 14:10 <mlots> aj: I'm not sure. If it can't then there's a good reason for only supporting ReactOS (i.e. archive space).
 338 14:11 <aj> is PE that much bigger than ELF?
 339 14:11 <mlots> aj: sure if you need to recompile.
 340 14:11 <aj> ...?
 341 14:11 <nchip> mlots: but currently it seems that the familiar approach is gaining more momentum
 342 14:11 <aj> you have to recompile everything for a new arch anyway?
 343 14:11 <aj> or are you saying "just run the linux binaries" ?
 344 14:12 <mlots> aj: I'm saying just run the linux binaries in the same way that kfreebsd might.
 345 14:12 <aj> kfreebsd was going to have all its own binaries ttbomk
 346 14:13 <mlots> They're not linux binaries neccesarily?
 347 14:13 <aj> we were talking about the freebsd abi earlier, not the linux abi
 348 14:13 <aj> i didn't think any of them were linux binaries
 349 14:13 <aj> i may be mistaken
 350 14:13 <braindmg> mlots: we don't use the linux compat layer
 351 14:14 <nchip> In away, I'm more interested in how to get new archs to SCC, than debating if a certain combination of a arch/kernel/libc is sensible or not
 352 14:15 <neuro> nchip: you don't see how the two are related?
 353 14:16 <mlots> I guess I'm not sure if syscalls leak into binaries when things aren't compiled statically.
 354 14:16 <mlots> Anyway I agree with nchip, that's getting a bit into implementation.
 355 14:16 <nchip> neuro: does and arch to be included in SCC need to be known to be sensible? Isn't better to find out in the SCC phase how usefull it is
 356 14:17 <neuro> nchip: why does it need to be on the debian archive server to determine if it is sensical/useful or not?
 357 14:17 <mlots> Aren't most SCC's not sensible until they become part of the archive proper?
 358 14:17 <neuro> shouldn't that be done before adding it to the archive?
 359 14:17 <aj> nchip: no, they need to be sensible before being in the archive; that includes working out combinattions of kernel/ABI and assigning them to debian architectures in order to minimise the number of architectures, and also (hopefully) getting a stable ABI that won't need to be changed in future
 360 14:18 <mlots> aj: Does Hurd, sparc etc meet that?
 361 14:18 <aj> in so far as hurd doesn't meet it, it's questionable whether it should be in the archive
 362 14:19 <aj> i don't think sparc's had any major abi issues; certainly it's avoided needing a sparc64 port in parallel (or a sparcel/sparceb port, etc)
 363 14:19 <mlots> aj: It's a working port, it's useful to it's developers, it's useful to it's users.
 364 14:20 <mlots> aj: SCC shouldn't need A
 365 14:20 <mlots> ABI stability
 366 14:20 <nchip> neuro: I guess that depends on who does the work of maintaining SCC archive servers.
 367 14:20 <mlots> That's the point of having snapshots right? 
 368 14:21 <aj> mlots: useful to users ==> ABI stability. ABI instability ==> screwing your users over
 369 14:21 <mlots> The ABI only has to be stable long enough to compile the relevant parts of the port's archive (granted, for c libaries that's most/all of it).
 370 14:21 <aj> mlots: that's fine for development, but not for use
 371 14:21 <nchip> neuro: but it would be a bit appalling to expect each porters to maintain the infrastructure outside debian
 372 14:21 <mlots> aj: If your users are developers...
 373 14:22 <aj> mlots: then your port isn't ready
 374 14:22 <DavidS> uclibcs two last releases were a year apart
 375 14:22 <mlots> aj: for scc?
 376 14:22 <mlots> aj: Debian tools make it easier to work on a port.
 377 14:22 <aj> mlots: yes; SCC == letting in other arches of at least m68k/hurd's standard; not lowering it further
 378 14:22 <mlots> aj: I'd say API stability is more important.
 379 14:23 <aba> mlots: look at e.g. amd64 - there are ports outside the main archive. Why not?
 380 14:23 <aj> mlots: that's fine, dak and debbugs are free software, you can setup your own while you stabalise your software
 381 14:23 <aba> aj: well, even bugs.d.o could be used to a large extend :)
 382 14:23 <aj> mlots: API stability is hopefully a given, there's the C standard and all
 383 14:24 <mlots> aj: so no SCC until a port can theoretically have two snapshots where one can upgrade from one to the other?
 384 14:24 <aj> mlots: yes, definitely
 385 14:25 <mlots> Should there be a requirement that users!=developers? (I hate to ask, but that seems to be the implication of what aj's saying).
 386 14:25  * aj makes that explicit
 387 14:26 <aj> even if your users are developers, you shouldn't screw them over with an unstable ABI
 388 14:26 <Greek0> mlots: if you say all your users are the port developers, why do you want it to be an official debian arch (scc or not) already?
 389 14:26 <mlots> aj: are current accepted ports following users!=developers for their user counts?
 390 14:26 <Greek0> I mean, an alioth project or something should be enough anyway for that period of time 
 391 14:27 <mlots> Greek0: Infastructure.
 392 14:27 <aba> mlots: all ports reviewed by the release team had more than enough users anyways
 393 14:27 <mlots> Greek0: Didn't amd64 have problems using alioth?
 394 14:27 <Ganneff> mlots: the size of it, yes.
 395 14:27 <aba> mlots: and it was moved out due to too much space usage
 396 14:27 <aj> mlots: if you're so wonderful that everyone of your millions of users learns C and contributes back patches, that's not a black mark
 397 14:28 <aj> mlots: but if you then break the ABI for them, that is
 398 14:28 <mlots> Gannelf: So isn't that a problem for other ports?
 399 14:28 <Ganneff> mlots: only if they built all of debian, in testing and unstable and experimental
 400 14:29 <mlots> Ganneff: so three ports building one of those would be too much too?
 401 14:29 <aj> mlots: Stabilising your ABI isn't that tricky; at worst you just have to provide backwards compatability as Linux and glibc do
 402 14:29 <Ganneff> mlots: maybe.
 403 14:30 <neuro> making ftp-master run out of space for the port instead isn't really a win, either...
 404 14:30 <mlots> So third class citizen's might use alioth, lists.d.o, perhaps bugs.d.o?
 405 14:30 <aj> unstable ABIs also screw over mirrors and buildds
 406 14:30 <braindmg> aj: well even linux cannot provide backwards compat in all cases
 407 14:30 <aj> i'm not convinced arches that can't provide a stable ABI should be using shared resources at all
 408 14:30 <Ganneff> mlots: or any other machine out there.
 409 14:31 <Ganneff> mlots: if you have a machine and enough bandwith you just need the tools to run an archive.
 410 14:31 <aj> braindmg: absolute perfection isn't a requirement; if you meet or exceed Linux's standards, you're fine
 411 14:31 <Ganneff> mlots: and that is packaged stuff.
 412 14:31 <aj> i'm sorry, Linux/i386's standards
 413 14:32 <aj> i can't understand why anyone would want to try maintaining a port that randomly changes its exec format or libc regularly
 414 14:32 <Yoe> aj: you'll only meet that rule for Linux/i386 anyhow
 415 14:32 <Yoe> aj: other ports are by definition less tested
 416 14:32 <Yoe> than the most popular hardware architecture in the world
 417 14:33 <aj> Yoe: not breaking an ABI is easy -- don't remove stuff, only add to it
 418 14:33 <aj> Yoe: bugs are an entirely different matter
 419 14:33 <Yoe> aj: oh, right, missed that bit
 420 14:33 <Greek0> mlots: I don't think it's that terrible difficult to setup the infrastructure you need compared to the other work to do for a successful port. and AIUI debian's infrastructure is currently quite near to being overloaded, so using that resources just because it's easier isn't probably the best idea
 421 14:33 <mlots> aj: If you're stable per snapshot then there's use.
 422 14:34 <aj> mlots: if you're stable permanently you can use it, you can upgrade it, and you can do 3rd party development on it
 423 14:35 <mlots> aj: I think the key argument is upgrade.
 424 14:35 <mlots> 3rd party development could be done against a snapshot.
 425 14:35 <aj> it would have to be done against /every/ snapshot, is the problem
 426 14:36 <mlots> aj: If snapshots are as far apart as releases (even at 18 months
 427 14:36 <mlots>  that might not be a problem.
 428 14:36 <aj> just fix your abi once and for all
 429 14:37 <Yoe> mlots: I thought the idea is for snapshots to /not/ be so far away from eachother
 430 14:37 <aj> mlots: btw, who are you anyway?
 431 14:37 <nyu> wrt archive size problems, have you seen http://www.linuks.mine.nu/sizematters/ ?  it claims we can save 29.3 % of archive space by using p7zip in .deb format
 432 14:37 <mlots> Did anyone have any comments about my installer comment?
 433 14:37 <mlots> aj: Drew Scott Daniels
 434 14:38 <mlots> nyu: Youc an save about 0.3% by reording the files inside the debs.
 435 14:38 <nyu> interesting
 436 14:38 <nyu> you mean in the tar?
 437 14:38 <Greek0> nyu: as it was pointed out on debian-devel, p7zip is not an option as long as dpkg in stable doesn't support it. so we could only use it for etch+1
 438 14:38 <mlots> nyu: yup. I've got code that proves it for the linux kernel.
 439 14:38 <Yoe> mlots: there's a serious difference between 29.3% and 0.3%
 440 14:39 <mlots> ppmd is better than p7zip for compression. See compression.ca or maximumcompression (.info?)
 441 14:39 <Yoe> though of course the reordering doesn't require dpkg code changes to be able to unpack later on
 442 14:39 <nyu> Greek0: right. i'm on it (i sent patch to tar already)
 443 14:39 <nyu> mlots: URL?
 444 14:39 <nyu> ah
 445 14:39 <mlots> nyu: for my code?
 446 14:40 <nyu> mlots: is it free?
 447 14:40 <mlots> nyu: I haven't released my code yet.
 448 14:40 <mlots> nyu: ppmd is in Debian.
 449 14:41 <nyu> i'll have a look, thanks
 450 14:42 <Q_> mlots: ppmd also doesn't work on 64 bit arches.
 451 14:42 <mlots> aj: Is an arch specific installer good enough for first class?
 452 14:43 <mlots> Q_: Yup. It could be ported though.
 453 14:43 <nyu> aj: and if not, is it for second? :)
 454 14:43 <mlots> The trouble with switching to 7zip or ppmd etc is that it might raise the mimimum system requirements.
 455 14:45 <nyu> i think an i486 is still faster at uncompressing than average internet connection at downloading
 456 14:45 <mlots> nyu: how about CD?
 457 14:45 <nyu> CD access is usualy faster when the data you're reading is compressed, too
 458 14:45 <mlots> nyu: See compression.ca's speed vs size comparison.
 459 14:45 <aj> mlots, nyu: it needs to be a working installer, otherwise i don't see a problem
 460 14:45 <Q_> nyu: So what is an average internet connection nowadays?
 461 14:45 <nyu> maybe not for i486 though
 462 14:47 <aj> you could always include p7zip and a (minimally) updated dpkg in a point release
 463 14:47 <Q_> (And it doesn't download and decompress at the same time anyway.)
 464 14:47 <mlots> More compression does not nessiarily mean more time needed to decompress.
 465 14:48 <nyu> mlots: any idea how to contact this guy?  FLAC is missing in the audio comparison
 466 14:48 <mlots> It'd be worth considering p7zip's decompression requirements.
 467 14:48 <mlots> nyu: Use his e-mail address or try it yourself.
 468 14:49 <mlots> nyu: maximumcompression was more up to date last time I checked.
 469 14:49 <nyu> no public email it seems
 470 14:49 <mlots> I read a review showing that FLAC sucks compaired to some general purpose compressers.
 471 14:49 <nyu> i'll dig around
 472 14:50 <Greek0> aj: hmm. is this compatible with the "support updates from oldstable -> stable" policy? I mean what if someone installed sarge, never updated (ok.. probably unlikely and surely dumb, but anyway), and then wanted to update to etch when it's released?
 473 14:50 <nyu> mlots: the sound comparison is not for general-purpose
 474 14:50 <mlots> nyu: whois compression.ca?
 475 14:50 <nyu> mlots: ah
 476 14:50 <nyu> mlots: really?  where's that
 477 14:50 <Q_> Greek0: So they should first upgrade to sarge, then to etch.
 478 14:50 <aj> Q_: they first upgrade to sarge's dpkg, and install p7zip
 479 14:51 <mlots> Greek0: I think we violated that for woody->sarge.
 480 14:51 <mlots> nyu: maximumcompression.info or something.
 481 14:51 <Greek0> Q_: i.e. to "sarge+stable-security+stable-proposed-updates" or something like this?
 482 14:51 <Q_> aj: You're saying we support upgrading from stable - 2 to stable?
 483 14:51 <aj> i don't think we've got that sort of policy anyway -- we don't do security updates for people still running oldstable at that point, eg
 484 14:52 <mlots> Greek0: I think that's the case for point releases only.
 485 14:53 <aj> Q_: not particularly, but in this case how to do it by hand is pretty easy
 486 14:54 <mlots> aj: Could a SCC or first class citizen be not "real", but embeded?
 487 14:55 <nyu> mlots: i can't see any reference to 7zip there
 488 14:56 <mlots> nyu: the files should be available for compression.ca so you could give it a try.
 489 14:56 <pkern> nyu: "7-zip"?
 490 14:56 <aj> mlots: i'm not sure what it would mean for an embedded architecture to be a release candidate
 491 14:56 <aj> mlots: though udebs essentially make up a parallel set of "embedded" architectures
 492 14:58 <mlots> aj: I guess it could be given that it'd meet the built requirement through not-for-us?
 493 14:58 <mlots> aj: I'm not talking about udebs.
 494 14:58 <nyu> pkern: www.7-zip.org, p7zip.sf.net
 495 14:59 <nyu> btw, http://www.7-zip.org/ claims 7z outperforms ppmd (but that source is not impartial)
 496 14:59 <aj> mlots: i'm not sure it wouldn't make more sense to have a different "built requirement" for "embedded" arches
 497 15:00 <nyu> http://www.compression.ru/ybs/e&docs.htm
 498 15:00 <mlots> nyu: what about paq5? There's new compressers all the time.
 499 15:01 <mlots> aj: I'd say properly using not-for-us makes more sence than lowering the % requirement.
 500 15:01 <mlots> aj: less buildd time wasted.
 501 15:02 <aj> doing lots of not-for-us when the package could build is cheating for the archive %ge requirement
 502 15:03 <aj> but for an embedded arch, building a certain set of packages might still be a useful requirement; i don't really know
 503 15:03 <aj> until we've got an arch that it makes sense to consider "embedded" instead of "real", there's not much point trying to go into any detail
 504 15:03 <mlots> nyu: If a case for a new compresser is made, one needs to consider: - space savings, - decompression time, -decompression memory requirements, - somewhat reasonable compression requirements such that devs are willing to use it.
 505 15:03 <mlots> aj: m86k?
 506 15:04 <mlots> 'r m68k?
 507 15:04 <Yoe> mlots: we're not going to do that any time soon
 508 15:04 <nyu> mlots: right
 509 15:04 <nyu> mlots: but first, you have to consider patching dpkg ;)
 510 15:05 <nyu> s/dpkg/tar/g
 511 15:05 <aj> dpkg has already been patched for bz2 aiui; p7zip should be trivial
 512 15:05 <nyu> then dpkg
 513 15:05 <Q_> arm* looks more like an arch that might be an embedded one.
 514 15:05 <nyu> aj: yes, but i don't want to send a patch that makes dpkg pipe() itself and exec tar/p7zip separately
 515 15:05 <aj> there are actual arm workstations on the market today though
 516 15:05 <nyu> tar needs fixing first
 517 15:06 <aj> nyu: *shrug* it's not brain surgery
 518 15:06 <Yoe> mlots: if you're interested in the specifics, we'd have to switch to something that works on ColdFire before making it an 'embedded' arch is going to be any useful
 519 15:06 <mlots> nyu: what fixing? file reordering? putting the new compression into tar's code?
 520 15:07 <Yoe> mlots: and once we reach that point, we might as well stick to being a full arch, since coldfire CPUs are much faster than the 'classic' m68k ones
 521 15:07 <vorlon> are arm workstations anything more than glorified devel systems for embedded work, though?
 522 15:08 <Yoe> vorlon: those were intended for being of more use, yes
 523 15:08 <nchip> arm workstation I know of are riscpc replacements
 524 15:08 <vorlon> ok
 525 15:08 <vorlon> (somehow I doubt many people *use* them for anything other than devel systems for embedded work, but. :)
 526 15:09 <nyu> mlots: p7zip support
 527 15:09 <nyu> aj: no, it's more like planting rice
 528 15:09 <nyu> you have to wait till it grows
 529 15:09 <nchip> usually people crosscompile for embedded work. powerpc people are lucky enough they have a excuse to get nice devel systems from apple
 530 15:10 <aj> nyu: huh? it's changing a few dozen lines of code, it's an undergrad exercise, geez.
 531 15:10 <nyu> aj: it's ugly
 532 15:10 <nyu> i want to get tar fixed first
 533 15:10 <aj> that was including changing tar
 534 15:11 <nyu> i have a patch for tar already
 535 15:11 <nyu> the difficult part is getting it applied :)
 536 15:11 <nyu> then send a patch for dpkg and wait again
 537 15:13 <braindmg> aj: so are there plans to remove any arch from the archive?
 538 15:15 <aj> braindmg: yes, sh.
 539 15:15 <Yoe> yeah, that'd make total sense
 540 15:15 <Yoe> is there actually still any binary in the archive which is sh and up-to-date?
 541 15:15 <aj> i don't think there are any sh binaries in the archive, up to date or not
 542 15:16 <aj> it's all arch:all
 543 15:16 <mlots> scc requirements are basically: more useful than an existing scc, or signifigant(?) interest in a scc when there's theoreticaly extra infrastrucure space / man hours available?
 544 15:17 <aj> "scc" isn't really a good term to use
 545 15:17 <aj> requirements for going in the archive are basically: useful, maintained, stable
 546 15:18 <aj> maybe s/maintained/supportable/
 547 15:18 <nchip> "ports" would be quite neutral
 548 15:18 <mlots> stable as in abi, and runtime?
 549 15:19 <Yoe> aj: okay, and when the thing is stable, but would need an ABI update after having been in the archive for years. What's going to happen? It remains there, or it gets kicked out until the ABI's fixed again?
 550 15:19 <braindmg> even linux-i386 may need that eventually
 551 15:20 <aj> linux-i386 has gone through a few ABI changes (a.out to ELF, libc5 to libc6, libc6 to libc6 for that matter), it just maintains backwards compatability
 552 15:21 <aj> if your arch isn't worth the effort to do the ABI change in a backwards compatible manner, you're giving yourself a hard time arguing that it's worth including in the archive
 553 15:21 <aj> cf the hurd
 554 15:21 <Yoe> aj: to get this a bit more specific,
 555 15:21 <braindmg> aj: what about the hurd?
 556 15:21 <Yoe> the example I gave about the ColdFire move earlier isn't just hypothetical
 557 15:22 <Yoe> aj: I've been talking to rwhitby, who works at freescale, to see whether we'd be able to get a few eval boards to port Debian/m68k over to them
 558 15:22 <nchip> aj: debian was *much* smaller and had only i386 then, so a completly library rename made sense.
 559 15:22 <Yoe> they're a factor 5 or so faster than what we currently support
 560 15:23 <aj> nchip: a library rename is how we do these things in a backwards compatible manner
 561 15:23 <Yoe> but their instruction set is smaller, so we'd need to replace everything we have currently before it'll work.
 562 15:23 <nchip> aj: most people are going to object renaming all libs beginning from libc6 only for a minor archs abi transition
 563 15:23 <aj> nchip: so do it just for your arch, working out how to make it easy for everyone *shrug*
 564 15:24 <aj> Yoe: that sounds like it'd be fairly straightforward to do compatibly?
 565 15:25 <nchip> aj: with a new arch name. backward compatability if needed can be solved with multiarch.
 566 15:25 <Yoe> aj: yes, but it might be fairly lot of work, with much binNMUs and such
 567 15:25 <aj> nchip: a new arch name is a horrible idea
 568 15:26 <Yoe> aj: also, because of the instruction set having new instructions as well as losing old ones, I'm not entirely sure it's going to be possible to do this in a backwards-compatible way
 569 15:26 <mlots> aj: I'd like scc infrastructure to grow at the same rate as first class.
 570 15:26 <Yoe> though I can't be sure before I've tested it on the hardware
 571 15:26 <nchip> aj: that's your opinion. I think library rename for one arch is way worse.
 572 15:27 <nchip> aj: new arch name only involves porters of the said arch. library transition affects every debian users and needs handwork for every library maintainer
 573 15:27 <mlots> I'm not just talking about archive space. Also machiens, bandwidth, potentialy man hours...
 574 15:27 <aj> nchip: new arch names involve every single user of the arch, and are a ridiculous waste of disk space and bandwidth, sorry, no.
 575 15:28 <aj> mlots: stop thinking in terms of "scc"
 576 15:28 <aj> mlots: the only actual difference looks likely to be between release candidates and non release candidates for the time being; and the only different infrastructure between those two will be the testing scripts and stable support
 577 15:28 <nchip> aj: then we'll just stop caring about your official debian. sorry.
 578 15:29 <vorlon> aj: only a ridiculous waste of disk space if you keep the old arch around in testing/unstable, which it doesn't seem you would want to do?
 579 15:30 <nchip> ..since we are expected to maintain all the infra outside debian anyway, it does not seem like a big step anyway.
 580 15:30 <mlots> aj: but then there's now the third, those that don't make it into the archive... I get your point though.
 581 15:30 <aj> vorlon: you'd probably need to while the new arch was being repopulated, which would likely take a few weeks, then you've got to worry about supporting the people who aren't in a position to upgrade immediately
 582 15:32 <Greek0> nchip: debian doesn't have infinite resources. I think noone objects your stuff going into the archive if you have enough users, are willing to maintain your port, and it doesn't add a huge burden of another kind on debian.
 583 15:34 <vorlon> aj: would that really be much different, disk-wise, than changing the names of all the libraries, since the latter means recompiling everything and essentially having disparate package sets in testing and unstable for a period?
 584 15:34 <Greek0> but whining about "debian doesn't support us" when you have hardly any users and there seems to be quite a cost involved for debian seems a bit questionable.
 585 15:34 <aj> one of the major concerns is b/w from ftp-master; ABI bumping a port costs about 10-15GB; we usually try to limit pulses to about 2GB per day
 586 15:34 <braindmg> and I can understand that for the mirrors infra but having a new ports (or second class or whatever) ports.debian.org would make life way easier
 587 15:34 <Yoe> Greek0: uhh, Debian/arm does have quite some users, actually
 588 15:35 <aj> vorlon: if you do it compatibly you don't need to do it all at once, and potentially don't need to do it at all
 589 15:35 <aj> having a second set of infrastructure to maintain would make life twice as hard, thanks all the same
 590 15:35 <Greek0> Yoe: is this what nchip is talking about? I didn't really get it what his "pet port" was, sorry
 591 15:36 <nchip> ...potentially don't need to do it at all
 592 15:36 <Yoe> Greek0: I happen to know what nchip works on. That helps :)
 593 15:36 <vorlon> I'm not sure how you'd do it compatibly, though?  There's certainly no precedent within Debian of maintaining such compatibility through an ABI transition
 594 15:37 <aj> vorlon: eh? we did with libc5/libc6
 595 15:37 <braindmg> aj: for all arches
 596 15:37 <aj> vorlon: there were some rumours of an hppa ABI change a little while ago, iirc, what happened with it?
 597 15:37 <nchip> 17:22 < nchip> aj: debian was *much* smaller and had only i386 then, so a completly library rename made sense.
 598 15:37 <mlots> aj: I agree, a second set of infrastructure to maintain would make life twice as hard. That's the case with non-release and non-archive.
 599 15:37 <vorlon> Package: libpam0g
 600 15:37 <vorlon> Conflicts: libpam0 (<= 0.56-2), libpam
 601 15:38 <aj> oh, debian didn't have only i386 then, it at least had m68k and i think maybe sparc and alpha?
 602 15:38 <vorlon> aj: ^^^ doesn't seem like the kind of thing that happens piecemeal
 603 15:38 <nchip> hrmh ok. but still it was necessary for all the archs.
 604 15:38 <Greek0> Yoe: hehe. just that "stop caring about official debian" phrase reminded me of the gnu/solaris people (no offense)
 605 15:38 <braindmg> mlots: and replicating the infra and work for each new/external port
 606 15:38 <Yoe> Greek0: heh
 607 15:40 <aj> mlots: no, non-release go with all the other architectures, they just don't release
 608 15:40 <mlots> I can see the annoyance with getting bugs against your package for a port that would take time to maintain.
 609 15:40 <aj> mlots: non-archive stuff just needs to have a stable ABI, which really isn't that hard
 610 15:41 <mlots> aj: non-release wouldn't get the benifit of "testing" or the testing scripts... I guess they'd still be able to grab all the packages after release?
 611 15:43 <aj> mlots: there are other things that could be done beyond unstable and experimental, but i'm not going to talk about that until there's an /actual example/
 612 15:43 <nchip> what archs now have their own external infra? this stuff could benefit from co-operation.
 613 15:43 <mlots> I guess non-release could still have "testing"/"stable" buildd, and security support?
 614 15:43 <aj> at present, the only non-release architecture is hurd-i386, which currently has 4k packages in the archive, compared to ~24k for every other architecture in the archive
 615 15:44 <nyu> Greek0: gnu/solaris is much worse than just "no official repository"
 616 15:44 <mlots> aj: amd64 released sarge after Debian and didn't use the testing script?
 617 15:45 <Q_> amd64 doesn't use britney, which makes it rather hard for us to have a real testing.
 618 15:45 <Ganneff> mlots: we are on a different machine, we dont use britney.
 619 15:45 <Ganneff> we have some own scripts, fetching the data from ftp-master and running a little "mini-britney", basically following i386 testing
 620 15:45 <aj> had we had appropriate criteria in place, and changes to our mirroring, amd64 would've been in the archive and a release candidate years ago
 621 15:45 <Greek0> nyu: I followed the whole tragedy. I also didn't mean to imply that anyone here works on that port or hase the same attitude as the people that do. just that single statement remembered me of this issue.
 622 15:46 <Q_> aj: So what still needs to happen for the mirroring changes?
 623 15:46 <mlots> That's my point. non-release could grab a list of packages from stable or testing and build them later right?
 624 15:46 <braindmg> nchip: amd64 kfreebsd-i386 knetbsd-i386 hurd-i386 at least
 625 15:47 <mlots> That's mini-britney? Or is mini-briteny something else?
 626 15:48 <aj> Q_: work out exactly what mirrors should do; tell mirrors what they should do; wait for mirrors to do that
 627 15:48 <Ganneff> mini-britney is a shell script.
 628 15:48 <nyu> Greek0: they have an attitude?  I thought they simply worked in their own, separate world
 629 15:48 <nyu> Greek0: is there a single nexenta patch in bts? :)
 630 15:48 <aj> mlots: if you're going to try mimicing the release stuff, you should just be a release arch. the requirements really aren't that challenging
 631 15:48 <nyu> nchip: and armeb
 632 15:49 <nyu> hurd-i386 is in sid
 633 15:49 <nyu> and knetbsd-i386 is.. well, dead ;)
 634 15:49 <mlots> aj: Doing it on time with limited man hours is a challenge.
 635 15:49 <braindmg> nyu: it needs some stuff for which maintainers have not integrated the patches
 636 15:49 <braindmg> nyu: the infra is there ;>
 637 15:50 <nyu> braindmg: heh yes it is
 638 15:50 <aj> mlots: so get more users, and thus more developers
 639 15:50 <Greek0> nyu: attitude, as in: mindset, behavior, ..
 640 15:50 <Q_> aj: The "work out what mirrors should do" part is rather vague.
 641 15:51 <nyu> Greek0: yes.  their behaviour from our POV can be cathegorised as non-behaviour ;)
 642 15:51 <aj> mlots: the only real difficulty is when the hardware's difficult to come by and you can't get users, i guess
 643 15:51 <mlots> aj: creating a "sarge" snapshot for a port is useful as it limits the non-port RC bugs that ones needs to deal with.
 644 15:51 <Q_> They should have some easy script to select which aches they want to have on the mirror?
 645 15:52 <Q_> aj: Your blog also said something about making dak scripts better to handle the load?  Can you get into more specifics about that?
 646 15:53 <aj> Q_: apt-ftparchive does too much stat'ing, rather than just using its cache; i think there was something else along those lines too
 647 15:54 <mlots> Well I gotta go to work. Thanks for the discussion. I hope to read the transcript of the rest later. bye
 648 15:54 <aj> Q_: there's already a reasonably easy script, it's more the issue of making it easy for people to find a (eg) m68k mirror when the country mirror only had i386 eg
 649 15:54 <DavidS> aj: how much work would it be to split out non-mirrored .debs into /pool-non-mirrored/ or something, which can be easily excluded by those mirror -ops who don't want it?
 650 15:55 <Q_> aj: Atleast in d-i, it lists which arches are on each mirror.
 651 15:55 <aj> insanely painful, since it'd involve moving GBs of data around in a mirror pulse?
 652 15:55 <braindmg> DavidS: and then people that want to mirror the non-mirrored? ;)
 653 15:56 <aj> anyway, everything gets mirrored by someone, just not most people
 654 15:56 <Q_> aj: And http://www.debian.org/mirror/list seems to list the arches too.
 655 15:56 <DavidS> braindmg: well, nobody who wants to waste disk space should be hindered ;)
 656 15:57 <aj> so are these questions just out of curiousity, or should i be taking down names for my "to be volunteered for mirror maintenance jobs" list?
 657 15:57 <DavidS> aj: and only uploading new packages to the new locations? probably defeats the "simple" approach by too much complexity in programming :(
 658 15:58 <aj> DavidS: well, that's what happened for the move from dists/ -> pool/, but i don't see the point?
 659 15:58 <Q_> aj: Well, both curiousity, and I'd like to help if I can.
 660 15:58 <aj> DavidS: there's already an rsync-based script that'll do partial mirrors, and with some archive support, something like rsync --delay-updates --files-from:indices/arch-i386.lst --delete-after # should work
 661 15:59 <DavidS> aj: just brainstorming. point being: easy separation of seldom downloaded archs for easier capping of mirror disk usage and b/w
 662 16:00 <DavidS> aj: then the whole discussion around "we haven't enough mirror b/w and disk" goes out the chimney, isn't it?
 663 16:00 <aj> how so?
 664 16:01 <aj> mirroring all architectures (which is what we currently expect of pushed mirrors and top level country mirrors) is 160GB or so, an arch-specific mirror including arch:all and source is about 20GB
 665 16:02 <Q_> Seems to be at 164GB now.
 666 16:03 <aj> yick
 667 16:07 <vorlon> we need to get on jvw to remove more stuff
 668 16:08 <nyu> unrar
 669 16:08  * nyu hides
 670 16:09 <aj> hey, how come the linux-kernel-m68k stuff is arch:all?
 671 16:10 <Yoe> aj: hmm? isn't that just the patch you're talking about?
 672 16:10  * Yoe checks
 673 16:10 <aj> hrm, maybe it's just sources
 674 16:11 <Yoe> aj: what's the exact package name? packages.d.o doesn't find linux-kernel-m68k
 675 16:11 <aj> yeah, it was the patch and sources and for other arches too
 676 16:11 <aj> sigh
 677 16:11 <aj> toolchain-source is worse
 678 16:11 <Yoe> right, thought as much
 679 16:11 <Yoe> heh