Differences between revisions 18 and 19
Revision 18 as of 2016-02-09 17:14:38
Size: 16403
Editor: ?RainerDorsch
Revision 19 as of 2017-06-29 15:11:21
Size: 16457
Editor: TheAnarcat
Comment: link to CrossGrading
Deletions are marked like this. Additions are marked like this.
Line 7: Line 7:
The guide in this page needs to be updated. Several of the following steps should now be unneeded as dpkg natively supports cross-grading packages (apt does not though). See the end of the following [[https://lists.debian.org/debian-devel-announce/2012/03/msg00005.html|announcement]]. The guide in this page needs to be updated. Several of the following steps should now be unneeded as dpkg natively supports cross-grading packages (apt does not though). See the end of the following [[https://lists.debian.org/debian-devel-announce/2012/03/msg00005.html|announcement]] and the more up to date documentation in CrossGrading.

Translation(s): Русский

The guide in this page needs to be updated. Several of the following steps should now be unneeded as dpkg natively supports cross-grading packages (apt does not though). See the end of the following announcement and the more up to date documentation in CrossGrading.

These instructions do not work on current Debian stable with Multiarch For step-by-step instructions for migrating Debian Wheezy from 32 to 64 bit, see http://www.ewan.cc/?q=node/90 /!\ note that the guide on that link also performs dangerous and unnecessary steps, like unpacking raw .deb packages because it removes essential packages, instead of simply letting dpkg cross-grade them.

Migrate a 32-bit debian installation to 64-bit one (i386 to x86_64)

After several attempts to find a solution which does not need a full reinstallation of the system I began to try the approach suits myself. I have to apologize right off because it is *UGLY*, but it has the benefit that you don't have to start an installation which might do something hazardous (e.g. repartition your system without even waving for your lost sensitive data). The principle is simple:

  • install a 64-bit kernel,
  • bootstrap a mini 64-bit chroot system
  • overwrite your 32-bit system from the mini system
  • reinstall all packages

If you don't like the way, you are warned to stop reading me now! Or if you are not familiar with apt-get, dpkg and hacking small script, then this guide is not for you.

Check wether your Intel/AMD processor supports the 64-bit "long mode"

grep -q '^flags.* lm ' < /proc/cpuinfo && echo yes

Install a 64-bit kernel

You can either install the appropriate distro package (eg. linux-image-2.6.32-trunk-amd64) and boot that, or you can roll you own as I do it since the linux version 0.99 :). The important thing is, you have to enable the 32-bit mode emulation (CONFIG_IA32_EMULATION). It is enabled in distro kenels by default. If you miss it (like me at the very first attempt) then the kernel boots, tries to start init, but it's 32-bit, so it tries to find an appropriate executable loader module by starting modprobe. But of course modprobe is still a 32bit executable so, kernel tries to find an executable handler module again, ... finally it ends up with a message something like "endless loop detected, cannot start init".

If you compile you own 64-bit kernel on the still 32-bit system you just have to install the package gcc-multilib and to start make with

ARCH=x86_64 make

At the end your 64-bit kernel will smile in arch/x86_64/boot/.

Setup your bootloader to start it with and reboot your system.

Now all the kernel code runs as 64-bit machine code. It performs already better than your 32-bit kernel, because it uses more CPU registers and accesses the memory easier (especially if you used more than 4GB RAM - CONFIG_HIGHMEM64G). But the more difficult part comes now, the applications should be also changed to the 64-bit variants for better user-space performance.

Create a 64-bit mini system

I'll use the path /chroot64, but of course you can create it anywhere you like. So, if you run your 64-bit kernel, you can already start 64-bit applications. But applications rarely linked statically, they need shared libraries which will be loaded (mapped) into the process' memory space by the dynamic loader. An executable code text can be either 32-bit or 64-bit. If an application is 64-bit then it can only use 64-bit shared libraries. That is the hard nut to crack before the multi-arch support would not work in debian. We now create a mini system which works as a whole in 64 bit. It has 64-bit binaries and the 64-bit libraries they depend on.

mkdir /chroot64
apt-get install debootstrap
debootstrap --arch amd64 sid /chroot64 http://ftp.debian.org/debian

Note: I used the unstable branch (sid) in all my migration cases because I have never had a pure stable installation. At least some packages were always installed from sid, so I got used to live dangerously... :-). If you have a stable 32-bit installation you can of course migrate it to the stable 64-bit, but then you should bootstrap the mini system to stable too.

"Login" into your mini system and add some more packages:

chroot /chroot64
apt-get install libc6-i386 file bzip2 openssh-client lftp 

(libc6-i386 is required such that your existing 32-bit binaries will operate. bzip2 will be required if you wish to run "apt-get update" successfully, as the files downloaded use bzip compression. The other packages are mere suggestions.)

Create a list of the 64-bit binaries of the mini system

The mini system is what you want as your main system apart from the facts that a)it comprise a lot less packages and b) it's not you main system just a sandbox now. We will disperse this sand onto the main, but only the executables, all other files (e.g. your adored config files in /etc remain intact), so collect them:

    cd /chroot64
    find . -depth | while read f
       if file -L "$f" | grep -q ELF
          echo "$f" >> /tmp/bins.txt
    echo "./etc/ld.so.cache" >> /tmp/bins.txt

Note1: you could run the commands in the chroot too, but then you have to go to its root 'cd /' at first and your 'bins.txt' come off in /chroot64/tmp. Note2: the last command, adding the ld.so.cache is crucial! It's not an executable, but needed by the dynamic linker (ld.so).

Upgrade your 32-bit system closest to the mini system

In order to have your 32-bit system as much as compatible (from the configuration files POV) to your mini 64-bit system, upgrade all your packages you have installed in the 64-bit mini root.

chroot /chroot64
dpkg --get-selections|awk '{print $1}' > /tmp/minipackages.txt
apt-get update
apt-get install `cat /chroot64/tmp/minipackages.txt`

Boot your rescue disk

One step must be done from an external system (Live Linux CD), because you cannot change the very base binaries beneath youself. Or at least it would be quiet risky. So, get a system rescue linux CD or USB stick based distro. One of my favourite is [http://blag.chaox.net/|Chaox]. It has a very recent kernel, it's tiny, it has all the important system tools and even an X interface with nvidia proprietary driver and works both from CD and from USB out of the box.

After you booted it mount your main system eg. under /main:

mkdir /main
mount /dev/sda1 /main
cd /main
ls # check if it's really the root of your main system
cat etc/fstab # see all your other filesystems, eg: /usr
mount /dev/sda8 /main/usr
mount /dev/sda11 /main/tmp

Note: of course the partition numbers depends on your own configuration. Explore your disks with eg. cfdisk, which is which.

It is important that you mount all your system partitions into the /main as it is on the running system. If you for example have different partitions for /bin, /usr, /usr/bin, etc. then mount them all!

Overwrite your 32-bit base system

Overwrite the binaries of your main system from the 64-bit mini root:

cd /main/chroot64
cpio -pVdu /main < /main/tmp/bins.txt

Note: the file bins.txt is what you created before. If you copied to somewhere else (eg. in your /) then you don't even have to mount /main/tmp, because there are no binaries there to overwrite.

Check your /lib* directories. Probably the mini system will have only 64-bit libraries in /lib64, but your original system did not even have such a directory. The 64-bit system will need 64-bit libraries in /lib.

Unmount all your mounted partitions:

umount /main/tmp
umount /main/usr
umount /main

Reboot to your 64-bit system

Now since the all the minimalistic, most important part of your system (binaries and libraries) are 64-bit, you can boot it! Of course the 99% of your system is still 32-bit. They will most probably not work, because the rudimentary libraries are already 64-bit and the dynamic loader will complain it about. You will have a terrible boot :-), but you will able to login and use the base system.

Once in this point I got the following friendly message:

run-init: /sbin/init: No such file or directory
Kernel panic - not syncing: Attempted to kill init!

I booted from USB again and saw that the /sbin/init is there and it's a good 64-bit exec. Kernel lies! Or don't? Just a bit. The init is there but init wanted to load the libc6.so. Of course the 64-bit version which resided in /lib64. On the mini root system there were a symlink: /lib -> /lib64. So I created the symlink (sill using the rescue image) and the next boot succeeded, sigh.

At least in jessie, /lib64 is not a symlink, so I need to make /lib64 copy from the mini root. Try "chroot chroot64 /bin/sh" to check the system.

Reinstall all the packages

Do you think this migration method is ugly as hell? The horror just comes here :-). But you don't have to feel frightened, all your system is intact, remains safe. The problems you will see here is just about the installation of system files. They are cheap, they can be downloaded any times whenever you want, don't they?

According to the apt database your system is now in a brain damaged state. Apt sees that you already on 64-bit (uname -m), but for all the packages Architecture=i386. Anything you wants to do with apt, it will say that you have to fix the inconsistency by issuing "apt-get install -f". Obey its words:

apt-get install -f

It will download tons of packages and starts to reinstall them. Fine. But you will see some warnings (e.g. some postinstall procedure do not work because they want to use 32-bit binaries, which no more have their appropriate libraries).

You will need your own creativity, because the issues you will have are always differ. I could not do two migration with the same problems. I can therefore tell you about some common case you will probable experince too. After each fix I'll write here below about you can repeat your previous installation step and it will goes further.

  • libc will not be upgraded automatically

Internal Error, Could not perform immediate configuration (2) on libc6
E: Could not perform immediate configuration on 'libc6'.Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)

Apt downloaded libc6-amd64, but don't dare to exchange it, because it's another architecture. (It does not know, that it is already changed, so it would do nothing just update its own database.) So, you have to help it:

cd /var/cache/apt/archives
dpkg -i libc6*deb

Then you can continue the 'apt-get install -f'.

  • clashing libs

The dynamic loader (at the time of writing) does not support to have the 32-bit and the 64-bit version of the same library (soname) in the same directory. If you see warnings like:

ldconfig: libraries libz.so. and libz.so. in directory /usr/lib have same soname but different type.

you should go there and check, which is the 32-bit version. It won't be needed any more, so you can simply delete it:

# cd /usr/lib
# ls -l libz.so*
lrwxrwxrwx 1 root root    15 Dec 30 21:25 libz.so -> libz.so.
lrwxrwxrwx 1 root root    15 Dec 30 21:25 libz.so.1 -> libz.so.
-rw-r--r-- 1 root root 93936 Dec 28 20:10 libz.so.
-rw-r--r-- 1 root root 93936 Jun 18 11:14 libz.so.
# file libz.so.1.2.*
libz.so. ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
libz.so. ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped
# rm libz.so.
# ldconfig   # now you won't get complains about libz

Such warnings are frequent if your old system lags more from the current unstable one, because then the new version of the same libs could have another version, so they won't be simply overwritten.

  • missing "base" packages

Many poinstall scripts want to start 'menu' and 'pyhon'. But apt-get not enough clever to start the package reinstallation with them, you can help again. If you see messages 'update-menus failed', that's about the package 'menu', and 'update-menus' is called from a dpkg hook.

cd /var/cache/apt/archives
dpkg -i menu*
dpkg -i python*
  • debsums does not work

In case you get errors because not working perl module MD5, which is actually a binary perl module, you can install it manually too. Sometimes a package reinstallation complains, because in the older packages some files belonged to another package and then dpkg don't want to overwrite them, you have to force it:

cd /var/cache/apt/archives
dpkg -i --force-overwrite perl*
  • reinstall of doc-base failed

Doc base uses ndbm database files (files.db and status.db) which are seemingly architecture specific. The update procedure of the 64-bit doc-base cannot read the remained 32-bit ones. Don't care, they will be regenerated if you delete them:

cd /var/lib/doc-base/info
rm files.db status.db

Resinstall all remained 32-bit packages

Sometimes after 'apt-get install -f' succeeds everything is done, your system is migrated. But sometimes a dozen of packages remains 32-bit. They either work (if they find their 32-bit libs) or not. But in any case it's better to reinstall them too. To list what packages are they do the following query:

dpkg-query -W -f '${Package} ${Architecture}\n' | awk '/i386$/ {print $1}'

I like this migration method also because at this point a lot of package could crop up which are not more exists (installed 5 years ago, not needed any more and you haven't even know what they are. You can purge them now gloatingly.

If you want to reinstall all i386 packages:

apt-get install `dpkg-query -W -f '${Package} ${Architecture}\n' | awk '/i386$/ {print $1}'`

But of course it will stop if there are no more existing packages. Then you can "apt-get purge" the really not needed ones, and try the reinstallation command again.

If you system do have tons of such not upgradable packages then you could find this procedure tiring, then you can decide you will care for them later, just the let upgradeable be upgraded, you can perform the command for each package separately:

for p in `dpkg-query -W -f '${Package} ${Architecture}\n' | awk '/i386$/ {print $1}'`
  apt-get -y install $p

it is slower (reads the apt db for every single package), but it skips the not upgradable ones.

And when you have time and are in the humor to entertain yourself with the remained i386 "what the hell are they" packages, you can list them with their description and judge on their life or death:

# dpkg-query -W -f '${Architecture} ${Package} ${Description}\n' | awk '/^i386/ {print; getline; for(;$0 ~ /^ /;getline) print; print "\n";}'
i386 pbzip2 parallel bzip2 implementation
 pbzip2 is a parallel implementation of the bzip2 block-sorting file
 compressor that uses pthreads and achieves near-linear speedup on SMP
 machines. The output of this version is fully compatible with bzip2
 v1.0.2 (ie: anything compressed with pbzip2 can be decompressed with

i386 pcscd Middleware to access a smart card using PC/SC (daemon side)
 The purpose of PC/SC Lite is to provide a Windows(R) SCard interface
 in a very small form factor for communicating to smart cards and
 smart cards readers.
 The PC/SC daemon is used to dynamically allocate/deallocate reader
 drivers at runtime and manage connections to the readers.

I've done this procedure many times beleive it or not, it's safe, real geeks could even enjoy it. And if you failed (but you would not!) then you could still install the system in the way you would done without this guide.