The following discussion of multiarch support in Debian and in general, took place between 00:35 and 03:42 UTC on 2005/09/27 on #debian-tech.


Summary and Log

Discussion began with DavidNusinow asking what was going on with biarch/multiarch support in Debian, as a result of requests for X to include biarch support (in the form of lib*64 packages for powerpc).

   1 00:35 <gravity> So who's got the scoop on biarch vs multiarch?
   2 00:45 <aj> biarch = same userspace for 32b/64b, different kernels, maybe glibc tweaks?
   3 00:46 <gravity> I don't know. svenl is asking what the XSF plans are for supporting one of the two. I don't want to commit to anything until there's some sort of decision made
   4 00:47 <aj> multiarch continues to not exist, and the whole point of it is that packages don't have to make any special decision
   5 00:47 <gravity> Last I heard, the biarch and multiarch people were fighting to the death
   6 00:47 <aj> i don't know what biarch actually is, i don't know if anyone but svenl's actually used the term
   7 00:47 <aj> but if it's what sparc already does, presumably X already deals
   8 00:48 <aj> and if it's not what sparc already does, it probably doesn't exist yet either
   9 00:48 <gravity> Well, he wants me to apply some patch for ppc64 of all things
  10 00:48 <dato> gravity: not really fighting. multiarch is the way to go long term, short term if somebody goes and does it in time.
  11 00:48 <dato> if not, biarch would be the interim solution, afaik.
  12 00:49 <aj> all the plans for multiarch i've seen have been "completely redo every package", which makes "long term" something of an underestimate :-/

?JamesTroup clarified what biarch support actually means -- namely providing 64 bit variants of packages without changing the Architecutre:. This was contrasted with multiarch support, which requires two separate architectures in the distribution, but allows systems of one architecture to make use of packages built for the other (as per i386 and ia64 or amd64).

   1 00:49 <gravity> Well, this thing is weird. Why do we need special 64 bit versions of the x libs now? There's been plenty of 64 bit arches before.
   2 00:49 <dato> aj: nah, biarch is a real term. our toolchain is even getting it.
   3 00:49 <elmo> aj: biarch is doing libfoo64 for relevant libraries
   4 00:49 <dato> (but I've read vorlon say it's an utter hack.)
   5 00:50 <elmo> the thing about biarch is that it requires modifying core libraries to do biarch for each biarch architecture
   6 00:50 <gravity> The idea of producing even more packages from the X source makes me nauseous, so I'm a little hesitant to do this
   7 00:51 <elmo> the thing about multiarch is that requires modifying every single part of the core toolchain in utterly drastic manners
   8 00:51 <aj> elmo: what defines "relevant" ?
   9 00:51 <azeem> you could always punt it until modular X is around
  10 00:51 <gravity> azeem: I'm very tempted to do so
  11 00:51 <elmo> aj: good question, but generally speaking, base/common libraries, e.g. curses, zlib, glibc etc.
  12 00:51 <gravity> Progress towards modular X in Debian appears to be slowing down though
  13 00:51 <elmo> it pre-assumes that you generally want to default to one architecture and only switch to the other for specific apps
  14 00:51 <aj> elmo: as opposed to "libraries where the speed difference is noticable" ?
  15 00:52 <dondelelcaro> gravity: aren't you guys primarily waiting on upstream there?
  16 00:52 <aj> elmo: is biarch == {arch=powerpc, some apps run as 64 bit} ?
  17 00:52 <elmo> aj: more libraries you'd probably need to run or build whatever you want to run in 64-bit mode (for ppc, sparc, etc.) or 32-bit mode (for amd64)
  18 00:52 <gravity> dondelelcaro: Not really. The Ubuntu packages exist to steal from yet again.
  19 00:52 <elmo> aj: yes, basically
  20 00:52 <dondelelcaro> ah
  21 00:52 <aj> ah, right, gross
  22 00:52 <gravity> dondelelcaro: I need to get a workable 6.9 ready to go first though, so if 7.0 takes too long on our end, at least etch can ship with up to date drivers
  23 00:53 <dondelelcaro> gravity: right
  24 00:53 <elmo> aj: well, yes and no.  it depends on how much gain multiarch actually gives real world users
  25 00:53 <elmo> rather than "hypothetical bob who wants to run hppa code on his Itanium"
  26 00:53 <elmo> biarch solves 99.9% of people's problems
  27 00:54 <elmo> if you've got an amd64 system, you _want_ everything to be 64-bit, you only need 32-bit for legacy apps
  28 00:54 <aj> well, yes, but modifying random libs to build a 64 bit package because someone might want to use it is suck
  29 00:54 <elmo> and if you've got ppc64 system and your name isn't Andreas Iminsane Jochens, you want everything to be 32-bit and you only need 64-bit for your database or whatever
  30 00:55 <elmo> aj: so do it on demand?  it should only ever be like 5% of libraries, never mind packages
  31 00:55 <gravity> elmo: Yeah... that's part of the issue.
  32 00:55 <aj> elmo: yeah, but it's going to be random insane people who demand it
  33 00:55  * gravity opts to punt it to 7.0, per azeem 

With the original issue punted, discussion continues on the more general issue of handling multiarch support.

   1 00:55 <mjg59> elmo: But with kfreebsd actually getting some users, it's likely that they'll want to be able to install Linux binaries
   2 00:56 <mjg59> Which multiarch basically gives you for free
   3 00:56 <elmo> there's nothing whatsoever FREE about multiarch
   4 00:56 <aj> nothing about multiarch is free
   5 00:56 <aj> heh
   6 00:56 <mjg59> Well, yeah
   7 00:56 <gravity> What exactly would multiarch entail that's so different?
   8 00:57 <aj> i still don't get why we can't just use alien to install _i386.debs on _amd64 natively now
   9 00:57 <mjg59> It's like "If you ram this rusty nail through your hand, we'll give you tetanus FOR FREE"
  10 00:57  * gravity laughs
  11 00:57 <mjg59> But from a technical aesthetics point of view, multiarch appeals to me
  12 00:57 <elmo> gravity: it involves radical (depending on the variant of multiarch) changes to the entire packaging toolchain, the archive software and anything that makes fairly basic assumptions about packages
  13 00:58 <elmo> (like e.g. relying on, err, I dunno, the Architecture field, f.e.)
  14 00:58 <aj> it's the technical aesthetics of multiarch that're the main problem
  15 00:58 <gravity> Ouch
  16 00:58 <mjg59> elmo: The archive software needs changes?
  17 00:58 <elmo> mjg59: for most of the multi-arch variants I've seen yes
  18 00:58 <aj> hey, that's right, it was multiarch's "make pkg's unique by name+arch instead of name" that makes me wary about having debbugs distinguish by name/ver/arch instead of just name/ver
  19 00:59 <elmo> mjg59: it breaks fundamental assumptions, like pkg uniqueness
  20 00:59 <aj> multiarch is a red hat plot to EAT YOUR BRANE
  21 01:00 <aj> can anyone explain why we can't do multiarch via alien, anyway?
  22 01:00 <mjg59> elmo: Surely that's only on the system, rather than in the archive?
  23 01:00 <aj> assuming you want to run i386 binaries on some supported non-i386 arch, anyway?
  24 01:00 <mjg59> aj: What about stuff with hard-coded paths?
  25 01:01 <elmo> mjg59: dude, there are multiarch proposals that involve dropping the Architecture field
  26 01:01 <mjg59> Ah
  27 01:01 <aj> mjg59: unhardcode the paths just as you would if you were doing multiarch?
  28 01:01 <elmo> mjg59: you've got to see how that'd mess with the archive's head?
  29 01:01 <mjg59> elmo: Yes, I can imagine that
  30 01:01 <mjg59> But. Uhm. I don't really understand why that would be desirable.
  31 01:02  * mjg59 should really reread these proposals
  32 01:02 <mjg59> aj: Yeah, so you end up with a lot of the pain anyway
  33 01:02 <aj> mjg59: you don't end up changing either dpkg or the archive
  34 01:02 <aj> mjg59: which is what causes a lot of the pain
  35 01:02 <elmo> mjg59: because if you put the architecture information in Depends, you can do things like "Depends: sse" !!!!!!!!!!!!!!!!!! so useful!!!
  36 01:03 <mjg59> elmo: Nnngh.
  37 01:03 <elmo> ahem.  I should not talk about multiarchitecture, it makes me appear as bitter as I actually am
  38 01:03 <mjg59> Maybe I should write a multiarch proposal that isn't on crack.
  39 01:03 <aj> you can't
  40 01:03 <mjg59> aj: I, uh, semi-implemented one
  41 01:03 <aj> it's a paradox
  42 01:03 <dondelelcaro> it's got fundamental crack usage.
  43 01:04 <mjg59> Using a hacked NetBSD kernel
  44 01:04 <aj> see? crack
  45 01:04 <mjg59> Haha
  46 01:04 <mjg59> But the Debian side of it was non-crack
  47 01:04 <gravity> My God... this svenl ajochens request thread degenerates in to what to name the fucking arch. wtf?
  48 01:05 <helix> you should listen to both of them
  49 01:05 <dondelelcaro> hey, naming the arch is a serious technical questions
  50 01:05 <aj> hrm
  51 01:05 <dondelelcaro> $DEITY forbid that we have to visit the ctte to decide that
  52 01:05 <aj> elmo: could we assign colours to packages/arches/etc? then we could have /real/ debates...
  53 01:05 <aj> elmo: or maybe logos instead of colours?
  54 01:05 <aj> dondelelcaro: (i thought it already visited the ctte; it was on dpkg already at least)
  55 01:06 <dondelelcaro> aj: ;-)
  56 01:06 <azeem> I'd guessed this is about ppc64 now?
  57 01:06 <gravity> azeem: Yeah
  58 01:07 <aj> where is a current multiarch proposal?
  59 01:07 <elmo> I'll ask taggart
  60 01:07 <elmo> he was pestering me today
  61 01:11 -!- taggart [~taggart@colo.lackof.org] has joined #debian-tech
  62 01:13 <taggart> multiarch?
  63 01:13 <elmo> 02:07 <@aj> where is a current multiarch proposal?
  64 01:13 <taggart> http://www.linuxbase.org/futures/ideas/multiarch/index.html

The LSB multiarch proposal MattTaggart points to basically covers how the filesystem should behave, leaving the question of how the files get there in the first place to the distributions. That is, it specifies what biarch and multiarch should result in, but leaves the question of which approach to use up to the distributor.

MattTaggart goes on to note that that proposal is likely to be extended further to differentiate architectures by processor, kernel and userspace, rather than just processor and kernel, and discussion of how that works (and why it's worthwhile) ensues. In summary it's probably not necessary, but it's not harmful and makes some things clearer. Essentially the arch triple defines an ABI for running and linking binary applications -- and each aspect of the triple is important for doing that.

   1 01:14 <taggart> but that needs to be updated for the new TRIPLE idea
   2 01:14 <taggart> instead of $arch-$os it's now $arch-$kernel-$userspace, just like the gnu config.guess triple
   3 01:15 <taggart> that change was based on some discussions with Keybuk at debconf
   4 01:15 <Lunar^> about kfreebsd and such?
   5 01:15 <neuro> oh, so we can have i386-linux-debian, and hppa-hpux-debian and that sort of crack?
   6 01:15 <taggart> if people haven't watched Keybuk's dpkg2.0 talk from debconf I recommend it
   7 01:16 <neuro> lunar^: amd64-freebsd-debian
   8 01:16 <aj> taggart's a stalker!
   9 01:16 <taggart> neuro: yes
  10 01:16 <Keybuk> small shorts^Warchitectures ...
  11 01:16 <Lunar^> taggart: watching it is still on my todo list (with some other talks) :)
  12 01:17 <taggart> Lunar^: yes this would allow for kfreebsd sorts of stuff too
  13 01:17 <dato> heh, I read bad and it seemed to me it was Keybuk who said Lunar^'s line
  14 01:17 <taggart> aj: nah, just a fanboy :)
  15 01:17 <neuro> of course, just because we can make every *-*-debian under the sun, doesn't mean we should.
  16 01:18 <aj> elmo: (erm, am i confused, or is taggart making multiarch even /more/ crackful?)
  17 01:18 <aj> Keybuk: btw, debian's planet is crashing while parsing branden's current rss feed
  18 01:19 <taggart> aj: I think matching the gnu triple probably makes the most sense. now matching the quad would be crackful
  19 01:19 <Keybuk> aj: yeah, elmo's ping'd me <--
  20 01:20 <aj> pung should be the past tense of ping
  21 01:20 <aj> "oh, yes, i was pung"
  22 01:20 <mjg59> elmo: The toolchain changes are pretty basic - it ought to mostly just be hacks to the specs file
  23 01:21 <aj> if you were the victim of a flood ping, you could say "my server was pung drunk"
  24 01:21 <elmo> mjg59: packaging toolchain
  25 01:21 <elmo> i.e. dpkg, apt, aptitude, dselect etc.
  26 01:22 <mjg59> Oh, I see what you mean
  27 01:22 <aj> taggart: multiarch support for things like qemu or wine assumes the kernel has some binfmt-type handling to make it look like they're running completely natively, right?
  28 01:22 <Keybuk> dpkg depends on keybuk having time
  29 01:22 <Keybuk> and/or motivation :p
  30 01:22 <mjg59> aj: Yeah, but that's trivial
  31 01:22 <aj> Keybuk: but dpkg is essential, so that should be pre-depends
  32 01:24 <aj> how can binfmt possibly handle the -dist component of an arch-os-dist tuple?
  33 01:25 <taggart> aj: that's sorta separate from the FHS proposal. but yeah if you want to be able to just run shit on the command line then yes. but the proposal doesn't require that, you could invoke qemu by hand for example.
  34 01:26 <aj> if you're invoking qemu by hand, why not dump it all in /srv somewhere, instead of cluttering up ld's paths?
  35 01:26 <aj> or /usr/lib/qemu or whatever
  36 01:26 <taggart> aj: I guess we've never had to deal with more than one userspace under the same kernel yet
  37 01:26 <mjg59> aj: Magic ELF flags. Uhm.
  38 01:26 <aj> what magic elf flags are there between distros?
  39 01:26 <helix> I love magical elves
  40 01:26 <mjg59> aj: The ones that we standardise on
  41 01:27 <mjg59> (CRACK CRACK LOVELY CRACK)
  42 01:27 <aj> ...
  43 01:27 <aj> why is the -dist desirable?
  44 01:27 <taggart> aj: the point of the FHs proposal is to fix the namespace issue, why dump the qemu case in another dir when you can use one consistant solution?
  45 01:27 <aj> taggart: if you can use /srv already, what's wrong with the namespace that it needs fixing?
  46 01:28 <aj> taggart: (one thing that's wrong with it is it won't work for binfmt/ld type execution)
  47 01:28 <aj> taggart: (but that's not an issue if you're runningstuff through some special envrionment like chroot/qemu afaics)
  48 01:28 <taggart> aj: the part under /srv in your example. qemu can support various arch-os's where do you put them under /srv?
  49 01:29 <aj> taggart: wherever you like, /srv/redhat/i386/ eg
  50 01:29 <aj> why is the -dist desirable?
  51 01:30 <taggart> mainly so the second field means kernel I think
  52 01:30 <aj> why would you distinguish -redhat and -debian, but not -fedora-core2 -fedora-core3?
  53 01:30 <neuro> that's dist vs. dist version ?
  54 01:31 <taggart> aj: oh we don't, the userspace would be -gnu -bsd -solaris etc
  55 01:31 <elmo> hum, so what are my chances of running an X app in a remote chroot?
  56 01:31 <aj> elmo: xauth + ssh tunneling?
  57 01:32 <neuro> along with copying the .Xauthority files into the right spots, pretty good?
  58 01:32 <aj> taggart: i386-linux-gnu, i386-linux-ulibc?
  59 01:32 -!- mentor [~matthew@] has joined #debian-tech
  60 01:33 <taggart> aj: is ulibc's ABI different than glibc? if so then that might make sense
  61 01:33 <mjg59> Are libc's symbols not versioned?
  62 01:33 <elmo> mjg59: seven ways to sunday
  63 01:33 <neuro> glibc's are
  64 01:33 <mjg59> Right
  65 01:33 <neuro> other libcs?  I don't think so.
  66 01:33 <aj> taggart: you might be able to install glibc and ulibc simultaneously, but we can pretend you can't
  67 01:34 <mjg59> I think a situation where different libcs aren't installable in parallel should be a bug...
  68 01:34 <taggart> so here's a concrete example, ia64-hpux-hpux has support for running ia64-linux-gnu
  69 01:34 <aj> are there any a-k-u1 / a-k-u2 pairs where u1 != u2?
  70 01:34 <taggart> and maybe i386-solaris-solaris does too?
  71 01:35 <taggart> aj: ^^^
  72 01:35 <aj> those have different kernels
  73 01:35 <aj> same arch, same kernel, different userspace
  74 01:35 <azeem> aj: i386-netbsd-gnu vs. i386-netbsd-netbsd, I guess
  75 01:35 <taggart> yeah what azeem said
  76 01:36 <neuro> i386-solaris-gnu, for a more practical one
  77 01:36 <taggart> neuro: does that exist?
  78 01:36 <aj> solaris-gnu?
  79 01:36 <neuro> wanting to run i386-solaris-solaris for what has to be from solaris
  80 01:36 <neuro> yes, there was a debian port to solaris
  81 01:36 <vorlon> gravity: wow, biarch people are fighting to... defend kludges that require us to keep multiple copies of glibc in the archive?
  82 01:37 <aj> neuro: is that meaningfully different to just installing lots of GNU toolson solaris-solaris?
  83 01:37 <mjg59> azeem: The aim with i386-netbsd-gnu was always to have it be compatible with i386-netbsd-netbsd
  84 01:37 <taggart> btw I talked to some apple darwin people at OSCON and they had no interest in ppc-linux-gnu apps running on ppc-mach-macos(or whatever it is)
  85 01:37 <gravity> vorlon: Is that what they're fighting over? I thought it was just the name
  86 01:37 <neuro> aj: it's giving a place for the two to exist, which can make a way to say which you prefer easier?
  87 01:38 <taggart> but then apple employees might not, but darwin hackers might
  88 01:38 <mjg59> I don't entirely understand why i386-solaris-solaris and i386-solaris-gnu should be incompatible
  89 01:38 <mjg59> You're not talking about being able to install more than one copy of the same binary, are you?
  90 01:39 <neuro> for that example, you would want to, yes.
  91 01:39 <azeem> mjg59: ugh, do you trust nyu on that?
  92 01:39 <mjg59> azeem: He's not working on NetBSD
  93 01:39 <azeem> ok, I thought he had his hands on that as well
  94 01:39 <mjg59> neuro: That sounds like a significantly more complicated situation
  95 01:40 <mjg59> azeem: He toyed with it briefly, but, well
  96 01:42 <mjg59> I doubt that we'll ever have a situation where arch-kernel-1 isn't binary compatible with arch-kernel-2
  97 01:43 <mjg59> There'd never be any real incentive to produce that - people would want to be able to run existing binaries
  98 01:43 <aj> mjg59: right, but you can treat "kernel-userspace" as just a more descriptive tag of "kernel" then, no big deal
  99 01:44 <aj> mjg59: as long as we don't get magic elves and i586-linux-gentoo
 100 01:44 <taggart> mjg59: well linux has better support for running arch-native-os binaries than the arch-native-os has for running the linux binaries? (confusing)
 101 01:45 <mjg59> taggart: Mm. But why would that matter?
 102 01:45 <mjg59> And surely that would already be disambiguated by the kernel?
 103 01:46 <taggart> mjg59: well if they use the same namespace it might be an ok thing to do one direction and a disaster to do the other
 104 01:47 <mjg59> taggart: But they're different kernels, so you're already separating the namespace
 105 01:47 <taggart> mjg59: ah I see, I was thinking two kernels, nm

As the preceeding conversation finished, AnthonyTowns went back to first principles, and the topic of what the point of multiarch is. This included discussing how foreign binaries get launched (in Linux, the kernel uses binfmt to do so), where the loader (ld.so) goes, and where foreign libraries should go. Discussion of whether multiarch support should be in /usr or /lib also took place.

   1 01:40 <aj> the point is to have /usr/bin/mozilla be able to run as-if native, even though it happens to need stuff from some other arch, right?
   2 01:41 <taggart> aj: yes
   3 01:43 <aj> so you need binfmt to say "ah, this is a ppc-darwin-gnu binary, my libraries and such for that are in /usr/lib/ppc-darwin-gnu"
   4 01:43 <taggart> aj: loader and libs, yes
   5 01:44 <vorlon> gravity: well, I really don't see any reason to *like* biarch; reasons to dislike multiarch as an alternative are plentiful and have been enumerated nicely in this channel, but that doesn't mean biarch is warm and fuzzy.
   6 01:44 <Keybuk> aha!  water boils faster if I turn the hob on
   7 01:45 <gravity> vorlon: That definitely makes me feel better about punting it for a while.
   8 01:45 <gravity> vorlon: The whole thing strikes me as odd, given that we've gone without it for some time despite having 64bit arches about
   9 01:45 <neuro> biarch causes FTBFS for the 32bit arch for any biarch package
  10 01:46 <vorlon> gravity: <cough> no we haven't
  11 01:46 <gravity> We haven't?
  12 01:46 <neuro> no, we haven't
  13 01:46 <gravity> Sparc and ia64?
  14 01:46 <vorlon> gravity: look in the ia32-libs package.  I'll be minding the toilet ready to hold your hair back for you afterwards.
  15 01:47 <aj> "The program interpreter will be /lib/arch-os/ld.so.version"
  16 01:47 <aj> why does it make sense to ahve the loader be in /lib instead of /usr/lib?
  17 01:47 <neuro> sparc is the older example, and stopped as far as enough to build a kernel.
  18 01:47 <neuro> so you can have it before /usr is mounted ?
  19 01:47 <taggart> aj: IMHO it doesn't and I'd like to clean that up too
  20 01:47 <vorlon> aj: so that you can use the same binary regardless of whether it's the native ABI for the system you're running on
  21 01:47 <aj> neuro: but why would you want to be able to run mozilla before /usr is mounted?
  22 01:48 <aj> okay
  23 01:48 <vorlon> aj: linker path is embedded in the ELF binary, that's how the kernel tells them apart
  24 01:48 <taggart> aj: and if we put them in the multiarch dir we don't need them to have different names either
  25 01:48 <aj> taggart: where's i386 glibc go on amd64?
  26 01:48 <neuro> aj: what if it's something on /bin ?
  27 01:48 <vorlon> so you want to be able to have one consistent linker path for the ABI across all archs that use it
  28 01:48 <azeem> aj: so you can use this mininit or whatever it's called with i386-linux-ulibc, *duck*
  29 01:48 <taggart> aj: /lib/i386-linux-gnu/libc6....
  30 01:48 <aj> neuro: sure; but then you're crazy anyway :)
  31 01:49 <aj> huh?
  32 01:49 <aj> existing i386 binaries don't have /lib/i386-linux-gnu embedded in them
  33 01:49 <vorlon> nope, which means those binaries aren't multiarch-safe.
  34 01:50 <taggart> vorlon: in my proposal the linker path put in the binary would be /usr/lib/TRIPLE/ld.so.1
  35 01:50 <aj> vorlon: huh?
  36 01:50 <neuro> but if you do that for everything, how do you boot before /usr is mounted?
  37 01:50 <taggart> but we'll have links for legacy stuff
  38 01:50 <vorlon> taggart: hrm.  So /usr/lib/foo has to be part of the root fs?
  39 01:50 <aj> vorlon: if file can tell what arch they are, so can binfmt, and they're completely safe
  40 01:50 <taggart> vorlon: er sorry /lib/TRIPLE/ld.so.1
  41 01:51 <aj> taggart: what's with changing binaries?
  42 01:51 <aj> taggart: can't we already handle this??
  43 01:52 <taggart> aj: I'm just saying future binaries should point to the multiarch location as a way of cleaning things up
  44 01:52 <taggart> aj: existing binaries will work
  45 01:52 <vorlon> aj: hmm.  yes, in theory binfmt could tell them apart.  I don't think anyone's written anything to let binfmt do that, though?
  46 01:53 <vorlon> taggart: they will work on the native arch where the compat symlinks point to the linker for the matching ABI...?
  47 01:53 <taggart> vorlon: yep
  48 01:53 <aj> taggart: if the existing thing works fine, why change it? if it ain't broke...
  49 01:53 <aj> taggart: changing the ld path will mean existing systems can't run future binaries, why break that?
  50 01:54 <aj> file can certainly identify them
  51 01:54 <taggart> vorlon: and since all the linkers have different names currently, you could have compat symlinks for non-native existing binaries too
  52 01:54 <vorlon> taggart: hrm?
  53 01:55 <taggart> aj: yeah I guess so, but existing systems would only need to add a symlink the other way to support future binaries. and they might not be able to support the future binaries if they don't have the new libs anyway
  54 01:56 <taggart> vorlon: the current ld names are unique per arch
  55 01:56 <aj> they don't seem unique?
  56 01:56 <taggart> oh wait no they aren't
  57 01:57 <aj> heh
  58 01:57 <taggart> only on the common bi-arch cases they are
  59 01:57 <vorlon> taggart: you may be able to do that for particular pairs of archs that we already have biarch running on, but afaik most linkers are /lib/ld-linux.so.2 
  60 01:57 <aj> coz i386 uses ld-linux
  61 01:57 <taggart> http://sources.redhat.com/cgi-bin/cvsweb.cgi/~checkout~/libc/shlib-versions?content-type=text/plain&cvsroot=glibc
  62 01:57 <taggart> see the "# The dynamic loader also requires different names." section
  63 01:58 <taggart> vorlon: yep, you're right. so for those bi-arch cases you could also do a symlink for the non-native target
  64 01:59 <aj> taggart: see the "# We use the ELF ABI standard name for the default." ?
  65 02:00 <aj> anyway, different ld names are just a special case of the general binfmt hack
  66 02:01 <taggart> aj: yeah I'll separate the ld stuff off into a separate proposal (that depends on the multiarch FHS proposal)
  67 02:01 <aj> so, afaics, personally, i would want everything to go under /usr, with symlinks from /lib as necessary (which is the case for ld.so in the absence of clever binfmt hacks)
  68 02:02 <taggart> aj: I just saw it as an opportunity to clean the weird linker names up
  69 02:02 <aj> the only use case for sticking anything else under / seems to be multiarch at boottime, which seems crazy mad?
  70 02:02 <taggart> aj: yeah I guess that's an implementation detail
  71 02:02 <vorlon> aj: well, at that point you're requiring a newer kernel to support multiarch, which is less likely anyway to be something people with existing systems today would *need* to upgrade in order to run newer binaries, as opposed to glibc...
  72 02:02 <neuro> aj: 32-bit drivers on a 64-bit system?
  73 02:02 <aj> 32 bit userspace drivers ym?
  74 02:02 <neuro> yah.
  75 02:02 <aj> that're needed to mount /usr?
  76 02:03 <aj> that still says crazy/mad to me..?
  77 02:03 <taggart> yeah my proposal is supposed to be about userspace only, kernel drivers are way out
  78 02:03 <neuro> i'm just thinking non-free binary-only drivers for your disk array or something
  79 02:03 -!- faw [~felipe@] has joined #debian-tech
  80 02:03 <aj> can't you do binfmt hacks in userspace?
  81 02:04 <aj> or at least, modularly?
  82 02:04 <neuro> but i guess those would be kernel modules for the most part, anyways.
  83 02:04 <aj> vorlon: ^^
  84 02:04 <taggart> aj: are you still talking about this driver thing, or pointing out something else?
  85 02:05 <taggart> aj: btw the "# We use the ELF ABI standard name for the default." is what I'm proposing changing all of them too, and then they would just live in multiarch dirs to avoid collision
  86 02:05 <aj> vorlon: i was thinking binfmt_misc, which shouldn't need a particularly new kernel; and you only need it for multiarch, not to run native binaries
  87 02:06 <aj> (or something like it)

There was a brief digression about handling libs in /opt, and what level of special treatment that required.

   1 02:07 <aj> okay, so
   2 02:07 <aj> /opt/foo/lib/sparc-solaris/
   3 02:07 <aj> how is that meant to work?
   4 02:09 <aj> surely that would be /opt/foo/bin/foo -> i386-ld.so, libc.so, /opt/foo/lib/libfoo.so ?
   5 02:09 <aj> with libc.so being pulled from /lib/i386-linux-gnu/libc.so instead of /lib/libc.so?
   6 02:09 <aj> and /opt/foo/lib/libfoo.so not being renamed at all?
   7 02:10 <aj> (assuming you're on an ia64 system, say)
   8 02:12 <aj> otoh, separate directory names would be needed for multiarch development
   9 02:14 <taggart> aj: sorry stepped out for a sec
  10 02:15 <taggart> aj: I guess the /opt case only needs to be renamed for providers who provide libraries that they want to be multiarch aware
  11 02:15 <taggart> aj: just like the non-opt case I guess
  12 02:16 <aj> taggart: the common case for /opt is for people who want to run single-arch binaries on a different-arch presumably though
  13 02:16 <aj> taggart: (like proprietary stuff, or openoffice.org)
  14 02:17 <aj> taggart: or are there lots of multiarch isvs that i don't know about?
  15 02:17 <taggart> aj: no :)
  16 02:17 <taggart> aj: so what's the problem?
  17 02:17 <aj> there's no problem, i'm just trying to understand
  18 02:17 <taggart> ok

Interspersed with that was discussion of whether biarch support (which implies only a small number of packages need be built for a particular arch variant) could be done by providing a small architecture in Debian -- that is, having a separate arcitecture that doesn't build the majority of packages -- and using multiarch. It seemed feasible (given multiarch actually working), and as RyanMurray points out, removes the drawback biarch introduces that, eg, 32bit powerpc machines can't be used to build the libfoo64 variants biarch requires, so buildds are forced to be 64bit.

   1 02:13 <aj> elmo: what would happen if we had a native ppc64 dist that only built a small subset of debian? (like "interesting" libs, but not apps)
   2 02:13 <elmo> aj: doesn't need to be a separate arch
   3 02:14 <aj> elmo: if it were a separate arch, 64 bit libs could be built without requiring changes to the packages
   4 02:14 <aj> or, at least, with less special casing int he packages
   5 02:14 <neuro> and the binaries don't FTBFS on the 32 bit arch
   6 02:14 <neuro> err, the source, rather
   7 02:14 <neuro> testsuites don't get skipped attempting to work around that, either.
   8 02:15 <elmo> aj: *shrug* sure, but still requires packaging toolchain changes/hacks to allow them to be installed
   9 02:15 <elmo> it's fine from an archive POV tho
  10 02:15 <elmo> as long as it is minimal
  11 02:15 <elmo> the full on ppc64 thing is happening over my dead or non-contributing body

Multiple copies of the same library, built for different ABIs is naturally a necessary part of multiarch, but are multiple copies of the same binary also necessary? Probably not.

   1 02:18 <aj> so under what circumstances would you want multiarch binaries?
   2 02:18 <taggart> you question does point out that I need to make the /opt section more clear
   3 02:18 <aj> ie, an i386-linux-gnu bin/foo as well as a sparc-solaris-solaris bin/foo?
   4 02:18 <taggart> aj: the main one's I've thought of are plug-ins and having a single disk be able to boot on two archs
   5 02:19 <taggart> s/one's/ones/
   6 02:19 <vorlon> aj: ok, could be done with binfmt_misc, yes.
   7 02:19 <taggart> the plug-in case has been discussed using apache as the example
   8 02:20 <aj> how's that work?
   9 02:20 <taggart> say you didn't have mod_perl on an arch or something
  10 02:20 <aj> okay, so mod_skype say
  11 02:20 <taggart> or mutually exclusive plugins on two arches
  12 02:20 <taggart> so you'd want two apaches
  13 02:21 <taggart> but they couldn't run on the same port of course
  14 02:21 <taggart> but maybe you run them on separate? kind of a corner case
  15 02:21 <aj> yeah
  16 02:21 <taggart> browser plugin or gimp/photoshop plugin is maybe a better example
  17 02:22 <taggart> so this proposal only claims to solve the lib problem
  18 02:22 <taggart> *but*
  19 02:22 <aj> well, a nuisance if you have to run two separate gimp instances just coz the plugins are incompatible
  20 02:22 <taggart> if you wanted to solve the binary problem you could use the same method
  21 02:22 <taggart> so the other thing is that plug-ins probably aren't going to be multi-arch capable themselves
  22 02:23 <aj> sure, but that's not solvable without recoding
  23 02:23 <taggart> can you imagine running an i386 flash plugin on a ppc version of mozilla?
  24 02:23 <taggart> yeah
  25 02:23 <aj> like if a plugin uses host-native endianness or whatever
  26 02:23 <aj> bam, you need a new plugin API for multiarch
  27 02:24 <taggart> ok so the other case, booting the same disk on two machines. I can see wanting to do that with amd64/i386 or ia64/i386
  28 02:24 <taggart> but even if you solve the binary issue you have the bootloader and other issues to deal with
  29 02:24 <aj> with the same /etc?
  30 02:24 <taggart> yeah, you see the problem
  31 02:25 <taggart> so what about wanting to use the same chroot on two archs?
  32 02:25 <aj> if you have an i386 chroot in /srv/my-i386-chroot on an amd64 machine
  33 02:25 <taggart> jinx
  34 02:25 <aj> can you symlink /usr/lib/amd64-linux-gnu to /srv/my-i386-chroot/usr/lib ?
  35 02:25 <taggart> yeah I think so
  36 02:26 <aj> i think that's what you'd want to do for multibooting
  37 02:26 <taggart> or more importantly, you can nfs mount other machines in the right place to use their libs
  38 02:26 <aj> two separate installs, but crosslinked so you can run whatever okay
  39 02:26 <aj> maybe share /opt and /usr/local *shrug*
  40 02:26 <taggart> yeah that would be cool
  41 02:27 <neuro> i don't see dual boot as being all that interesting if multiarch "just works"
  42 02:27 <aj> for full on chroots you can just chroot into them by saying "enter-i386-emulation-mode-on $CHROOT" instead of "chroot $CHROOT", so there's no great win to be had there
  43 02:27 <taggart> unless there's something that doesn't run well under the "newer" kernel
  44 02:28 <taggart> but I can't see people switching back and forth a lot

Based on MattTaggart's last hypothetical, AnthonyTowns thus made a proposal for multiarch support that doesn't require changes to any of the tools.

   1 02:29 <aj> okay
   2 02:29 <aj> so why can't we do multiarch like that /right now/?
   3 02:29 <aj> (a) debootstrap a chroot in /srv
   4 02:29 <taggart> so one critique I get about this is "why don't you just use tricks in the loader and filesystem to make all that stuff work? you know about plan9?"
   5 02:29 <aj> (b) symlink /usr/lib/whatever to it, symlink the native ld.so to it
   6 02:30 <aj> (c) setup the ld.so to look in the /usr/lib/otherarch search paths
   7 02:30 <aj> (d) use apt in the chroot for dependency handling
   8 02:31 <taggart> so my response to the above is "great we can do that too! but this gives us a starting place and is less confusing for developers"
   9 02:31 <aj> sorry, i'm confusing we's
  10 02:31 <taggart> aj: that should work for the bi-arch's where the ld name is different and we can create that symlink, and has the new path
  11 02:31 <aj> okay, so the LSB proposal is great
  12 02:32 <aj> it provides some fine paths, and seems to make sense
  13 02:32 <aj> as far as debian's concerned, why mess with the packaging format and whatever when we can just do (a)-(d)?
  14 02:33 <taggart> aj: have you watched the dpkg2.0 talk yet? we don't even need to mess with the package format or do a-d
  15 02:33 <aj> packaging tools, way packages are built, archive, whatever
  16 02:33 <aj> "a-d"?
  17 02:33 <taggart> (a)-(d)
  18 02:33 <aj> oh, duh
  19 02:33 <aj> a-d can be done without dpkg2.0
  20 02:33 <taggart> yes
  21 02:33 <aj> or dak2.0, or britney2.0, or apt2.0
  22 02:34 <aj> or changing the way any packages are bilt
  23 02:34 <aj> built
  24 02:34 <taggart> so using apt in the chroot requires the system fully support running in the non-native mode I guess
  25 02:34 <aj> yeah
  26 02:35 <aj> but i'm pretty sure i could hack something together to let you have limited apt features for a kinda-chroot in another arch
  27 02:35 <taggart> (a)-(d) do require the binutils/gcc/glibc changes, but I think those are already done?
  28 02:35 <neuro> aj: all that's needed is config files
  29 02:35 <taggart> (a)-(d) nicely solves the /usr/share/doc collisons
  30 02:36 <aj> taggart: right, the LSB changes are needed no matter what, i'm only interested in the packaging side of things
  31 02:36 <neuro> but you still have to have the docs on the disc twice.
  32 02:36 <aj> right, and for i386, you should be able to use qemu to generate the (i386) chroot
  33 02:37 <aj> not unpacking stuff is a SMOP
  34 02:37 <taggart> so we could use aj's solution immediately, Keybuk's solution after that, and then also migrate all lib packages to multi-arch capable in the future (as upstream adopts it)
  35 02:37 <aj> and don't most libraries either not have docs or have them in a different package?
  36 02:37 <taggart> aj: well there's always the changelog
  37 02:38 <aj> and copyright
  38 02:38 <taggart> yeah

This trailed off into a discussion of dpkg...

   1 02:38 <aj> dpkg supports some --exclude thing now doesn't it?
   2 02:38 <neuro> which should be a symlink to a -common arch-all
   3 02:38 <taggart> which is where Keybuk's solution comes in
   4 02:38 <Keybuk> no, not currently
   5 02:39 <aj> i thought doogie implemented that?
   6 02:39 <Keybuk> hahahahahaha
   7 02:39 <Keybuk> he may have done, in which case I'm sure it sat on his disk until the devasting laptop fire which was only extinguished by the flood
   8 02:39 <aj> hrm, weird
   9 02:40 <aj> i hope i'm not dreaming about doogie now
  10 02:40 <neuro> aj: no, I remember the same thing.
  11 02:40 <Keybuk> what were you hoping to exclude?
  12 02:41 <aj> dpkg -i foo.deb --ignore /usr/share/doc/**
  13 02:41 <Keybuk> certainly nothing like that ... what would happen if you upgraded it?
  14 02:41 <aj> nothing special?
  15 02:41 <neuro> in a config file, so it's there for next time?
  16 02:41 <Keybuk> would it remember the ignore or not
  17 02:41 <Keybuk> anyway, no, there's nothing like that in dpkg
  18 02:41 <aj> if you put it in a config file; otherwise it'd just be the same as if the original .deb hadn't included anything under /usr/share/doc
  19 02:42 <aj> weird
  20 02:42 <Keybuk> that's not to say there won't be, it's in the plan with a bit more fun, but it's not there now
  21 02:44 <aj> And you guys keep telling me that
  22 02:44 <aj> dpkg will get directory excludes in 2.0. 
  23 02:44 <aj>  - Bug#189762
  24 02:44 <aj> weird, i thought that was done
  25 02:44 <neuro> aj: it was.  and died in said fire/flood.
  26 02:45 <aj> i defined "done" as "uploaded", not "on some hard drive, really, truly" :)
  27 02:45  * vorlon raises an eyebrow at the reinforcement of the fire story that suggests it's not a joke
  28 02:45 <Keybuk> I came to believe that "<doogie> the code is on my harddrive" was actually some kind of metaphor for life, and the continuing struggle of the masses to obtain enlightenment
  29 02:45 <Keybuk> and that only though denial could we truly set off down that path
  30 02:45 <Keybuk> or something

But eventually returned to multiarch support.

   1 02:46 <aj> taggart: so if you've got /lib/ld.so.1 -> /lib/ia64-linux-gnu/ld.so.1 and /lib/ld-linux.so.1 -> /lib/i386-linux-gnu/ld.so.1
   2 02:46 <aj> taggart: how do you tell each of them where to look for the appropriate libs?
   3 02:47 <aj> sigh, googling for "dpkg2 talk" takes me to an lwn article from '99
   4 02:47 <taggart> aj: (the first one is /lib/ld-linux-ia64.so.2 0> /lib/ia64-linux-gnu/ld.so.1 )
   5 02:48 <taggart> aj: http://dc5video.debian.net/mpeg/2005-07-15/02-Freezing_HEL_Over-Scott_James_Remnant.mpeg 
   6 02:49 <taggart> aj: so they would each be built with a ld search path that searched their multiarch lib dirs first, then the old style stuff
   7 02:50 <taggart> aj: sound ok?
   8 02:50 <taggart> (btw gcc would also do the same thing at compile time)
   9 02:50 <aj> taggart: you have to build them differently, there isn't some /etc/ld-i386-linux-gnu.{conf,cache} type thing you can use?
  10 02:59 <taggart> aj: hmm, maybe. ldconfig assumes /lib and /usr/lib ahead of anything in /etc/ld.so.conf, you'd want it looking in the new dirs first
  11 02:59 <neuro> the kernel's elf binfmt is what execs the loader, tho?
  12 02:59 <taggart> hmm, would you need an /etc/ld-TRIPLE.{conf,cache}?
  13 03:00 <taggart> Mithandir was saying that ldconfig is smart enough to ignore non-native stuff
  14 03:00 <taggart> but I guess you'd still need separate caches
  15 03:00 <taggart> so having separate confs might be a nice thing too
  16 03:01 <aj> well, separate confs you could do right now, without hardcoding any yet-to-be-decided standards
  17 03:01 <taggart> maybe /etc/ld.so.{cache,conf} should be in /usr anyway
  18 03:02 <aj> the .conf makes sense in /etc doesn't it?
  19 03:02 <taggart> yeah, so do we change the name or add directories?
  20 03:03 <taggart> or change the format of the file?
  21 03:03 <taggart> you could have generic search paths, then triple specific
  22 03:06 -!- moya [~moya@julia.sld.cu] has joined #debian-tech
  23 03:10 <aj> changing the format wouldn't be very backwards compatible
  24 03:11 <aj> maybe /etc/ld.so.conf.d/TRIPLE would be a conveninent haxck
  25 03:11 <aj> look for that file first, then /etc/ld.so.conf if it doesn't exist
  26 03:12 <aj> does ldd work with multiarch?
  27 03:13 <aj> hrm, i guess not, but maybe objdump can
  28 03:16 <taggart> so I'm still a little worried about what the binutils/gcc/glibc camps will think
  29 03:17 <taggart> can you think of any other holes they'd point out?
  30 03:28 <aj> on the "run other-arch code" side, or on the "build other-arch code" side?
  31 03:28 <aj> i think running sould be fine
  32 03:28 <aj> building i'm not so confident on
  33 03:33 <aj> taggart: what's the "dpkg2" summary?
  34 03:35 <taggart> aj: separating the unpack and put on the disk phases of install lets you do cool stuff like relocate libraries and not install /usr/share/doc, by writing cool plugin rules
  35 03:35 <taggart> Keybuk: ok summary?
  36 03:35 <Keybuk> roughly :)
  37 03:36 <Keybuk> another way of putting it is that the dpkg2 divert system allows wildcard diverts :p
  38 03:36 <Keybuk> or even divert-by-script
  39 03:36 <aj> yeah, that still has problems with having two packages of the same name installed at the same time though
  40 03:36  * taggart also liked the don't-install-documentation-on-an-embedded-system idea for a script
  41 03:39 <neuro> don't install locale files you'll never use.
  42 03:39 <neuro> another favorite
  43 03:42 -!- Keybuk [~scott@syndicate.netsplit.com] has quit [Quit: Leaving]