Size: 4506
Comment:
|
← Revision 21 as of 2020-12-01 03:10:59 ⇥
Size: 5557
Comment: qemu-user emulation is not virtualisation
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
#language en | |
Line 3: | Line 4: |
This page describes how to setup and use QEMU user emulation in a "transparent" fashion, allowing to run non-native target executables just like native ones ({{{./program}}}). | This page describes how to setup and use QEMU user emulation in a "transparent" fashion, allowing execution of non-native target executables just like native ones (i.e. {{{./program}}}). |
Line 7: | Line 8: |
'''Note:''' this text depends on features that are not available in official Debian qemu package as of 2006-02-26. See [http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=306637 this bug report] for details. | '''Note''' this setup is incompatible with [[http://www.scratchbox.org|Scratchbox]] (both use the binfmt_misc module to register the same formats), so it's recommended to remove it (or stop its init script) before continuing. |
Line 9: | Line 10: |
'''Note''' this setup is incompatible with Scratchbox (both use the binfmt_misc module to register the same binaries), so it's recommended to remove [http://www.scratchbox.org Scratchbox] (or stop its init script) before continuing. | === Installing packages === |
Line 11: | Line 12: |
=== Installing binfmt-support === | The binfmt-support package contains a helper script to easily register/unregister binary formats with the kernel using the [[WikiPedia:binfmt_misc]] module. |
Line 13: | Line 14: |
The binfmt-support package contains a helper script to easily register/unregister binary formats with the kernel using the [http://www.tat.physik.uni-tuebingen.de/~rguenth/linux/binfmt_misc.html binfmt_misc] module. | 1. Install qemu, binfmt-support, and qemu-user-static: |
Line 15: | Line 16: |
1. Install the binfmt-support package (used by qemu to register the supported binary formats): {{{ # apt-get install binfmt-support |
{{{ # apt install qemu binfmt-support qemu-user-static |
Line 19: | Line 20: |
1. Check whether the binfmt entries were successfully registered: {{{ | 1. Check whether the binfmt entries were successfully registered: {{{ |
Line 23: | Line 26: |
The last command should print entries for each supported target user emulator, except for the host system. | This command should print entries for each supported target user emulator, except for the host system. |
Line 25: | Line 28: |
=== Installing QEMU === | === Adjusting the system === |
Line 27: | Line 30: |
Finally, install the qemu package: {{{ # apt-get install qemu }}} There is also a qemu-user-static package that contains statically linked versions of the user mode emulators. Install it if you want to chroot into target filesystems (see Appendix below). |
Depending on the your kernel settings, you may need to set 'vm.mmap_min_addr=0' sysctl option to allow a program being run under a regular user, not root. |
Line 35: | Line 34: |
With the instructions above, you should be able to run statically linked target executables. To be able to run dynamically linked binaries, QEMU needs to have access to the target ELF interpreter. For convenience, the default search path for the ELF interpreter was setup to the same directory used by dpkg-cross for installing converted target packages for cross-compilation. | With the instructions above, you should be able to run statically linked target executables. To be able to run dynamically linked binaries, QEMU needs to have access to the target ELF interpreter. The libc6 package for the target architecture contains the target's ELF interpreter used by QEMU. Installing this can be done with multiarch from wheezy onwards, or with dpkg-cross on earlier (pre-multiarch) releases. === Installing the target C libraries with multiarch === For example purposes, let's assume the target system is "armhf". {{{ sudo dpkg --add-architecture armhf sudo apt update sudo apt install libc6:armhf }}} That's it. |
Line 39: | Line 52: |
The libc6 package contains the target's ELF interpreter used by QEMU. The target Debian package cannot be installed directly on the host, so we need to use [http://packages.debian.org/unstable/utils/dpkg-cross dpkg-cross] to "cross-install" the package. | If the target Debian package cannot be installed directly on the host, we need to use DebianPkg:dpkg-cross to "cross-install" the package. |
Line 41: | Line 54: |
For example purposes, let's assume the target system is "arm". | For example purposes, let's assume the target system is "armel". |
Line 43: | Line 56: |
1. Install the dpkg-cross package: {{{ # apt-get install dpkg-cross |
1. Install the dpkg-cross package: {{{ # apt install dpkg-cross |
Line 47: | Line 62: |
1. Now download the target libc6 package from one of the Debian mirrors and install it using dpkg-cross: {{{ # wget http://http.us.debian.org/debian/pool/main/g/glibc/libc6_2.3.5-6_arm.deb # dpkg-cross -i -a arm libc6_2.3.5-6_arm.deb |
1. Now download the target libc6 package from one of the Debian mirrors and install it using dpkg-cross: {{{ # dpkg-cross -i -a arm libc6_<version>_armel.deb |
Line 51: | Line 67: |
=== Point QEMU to the target linux loader === Under multiarch the target arch loader is in the usual place (/lib/<triplet>) so nothing special is needed. If using dpkg-cross it's installed in a non-standard path so you need to tell QEMU about that. for example, for the armel architecture: add the line EXTRA_OPTS="-L /usr/arm-linux-gnueabi" to the /etc/qemu-binfmt.conf. for armhf add: EXTRA_OPTS="-L /usr/arm-linux-gnueabihf" |
|
Line 56: | Line 85: |
1. Download the hello package (e.g. from http://http.us.debian.org/debian/pool/main/h/hello/hello_2.1.1-4_arm.deb). |
1. Download the hello package (e.g. from http://deb.debian.org/debian/pool/main/h/hello/hello_version_armel.deb) |
Line 59: | Line 87: |
1. Unpack it with the command: {{{ $ dpkg -x hello_2.1.1-4_arm.deb /tmp/hello_arm |
1. Unpack it with the command: {{{ $ dpkg -x hello_version_armel.deb /tmp/hello_armel |
Line 64: | Line 94: |
$ /tmp/hello_arm/usr/bin/hello | $ /tmp/hello_armel/usr/bin/hello |
Line 71: | Line 101: |
=== Appendix: chrooting into target filesystems === | === Appendix: chrooting into target file systems === |
Line 73: | Line 103: |
To be able to chroot into a target filesystem, the qemu emulator for the target CPU needs to be accessible from inside the chroot jail. For this to work, you need first to install the qemu-user-static package: {{{ # apt-get install qemu-user-static |
To be able to chroot into a target file system, the qemu emulator for the target CPU needs to be available. You need first to install the qemu-user-static package: {{{ # apt install qemu-user-static |
Line 77: | Line 109: |
You cannot use the dynamically linked qemu because the host libraries will not be accessible from inside the chroot. | If you are using stretch or earlier, it also needs to be accessible from inside the chroot jail. From Debian buster and later, this isn't necessary because Linux will use the qemu binary from the host system instead of from the chroot. This means that you can use the dynamically linked qemu on buster or later but not stretch or earlier because the host libraries will not be accessible from inside the chroot. |
Line 79: | Line 112: |
Next, copy the the relevant emulator to the ''same path as registered by binfmt-support''. For example, for an ARM target filesystem, you need to do the following:{{{ # cp /usr/bin/qemu-arm-static /target_fs/usr/bin/qemu-arm |
If you are using stretch or earlier, make the emulator available for the target architecture inside the chroot at the path registered by binfmt-support. You can either bind-mount the binary into the chroot, or just copy it. Copying it means you won't have to copy it again, but it also means it won't get updates, while bind-mounting means you have to bind-mount For example, for an ARM target file system, you need to do one of the following: {{{ # mount --bind /usr/bin/qemu-arm-static /target_fs/usr/bin # cp /usr/bin/qemu-arm-static /target_fs/usr/bin |
Line 83: | Line 119: |
Now you are able to chroot into the filesystem: {{{ | If you are using systemd, then you can use systemd-nspawn to start a container: {{{ # apt install systemd-container # systemd-nspawn -D /target_fs/ }}} If you aren't using systemd-nspawn, then you may want to mount special filesystems for devices, terminals and processes from the host in the chroot: {{{ for f in dev dev/pts sys proc run ; do mount --bind /$f tmp/$f ; done }}} You should now be able to chroot into the file system: {{{ |
Line 87: | Line 138: |
=== See Also === * EmDebian/CrossDebootstrap * [[Embedded_Debian]] |
|
Line 89: | Line 146: |
* Modify debootstrap to allow "cross-bootstrapping" a Debian system. (DONE: I just need to sync the changes to the latest debootstrap version and post the .debdiff on the BTS). * Modify Scratchbox to take advantage of the new qemu packages, instead of using its own internal qemu. * Modify pbuilder to allow cross-compiling packages easily. |
* Modify Scratchbox to take advantage of the new qemu packages, instead of using its own internal qemu (possible?). ---- CategoryEmdebian CategoryNotNative |
QEMU User Emulation
This page describes how to setup and use QEMU user emulation in a "transparent" fashion, allowing execution of non-native target executables just like native ones (i.e. ./program).
In this text, "target" means the system being emulated, and "host" means the system where QEMU is running.
Note this setup is incompatible with Scratchbox (both use the binfmt_misc module to register the same formats), so it's recommended to remove it (or stop its init script) before continuing.
Installing packages
The binfmt-support package contains a helper script to easily register/unregister binary formats with the kernel using the binfmt_misc module.
- Install qemu, binfmt-support, and qemu-user-static:
# apt install qemu binfmt-support qemu-user-static
- Check whether the binfmt entries were successfully registered:
# update-binfmts --display
- This command should print entries for each supported target user emulator, except for the host system.
Adjusting the system
Depending on the your kernel settings, you may need to set 'vm.mmap_min_addr=0' sysctl option to allow a program being run under a regular user, not root.
Running dynamically linked executables
With the instructions above, you should be able to run statically linked target executables. To be able to run dynamically linked binaries, QEMU needs to have access to the target ELF interpreter. The libc6 package for the target architecture contains the target's ELF interpreter used by QEMU.
Installing this can be done with multiarch from wheezy onwards, or with dpkg-cross on earlier (pre-multiarch) releases.
Installing the target C libraries with multiarch
For example purposes, let's assume the target system is "armhf".
sudo dpkg --add-architecture armhf sudo apt update sudo apt install libc6:armhf
That's it.
Installing the target C libraries with dpkg-cross
If the target Debian package cannot be installed directly on the host, we need to use dpkg-cross to "cross-install" the package.
For example purposes, let's assume the target system is "armel".
- Install the dpkg-cross package:
# apt install dpkg-cross
- Now download the target libc6 package from one of the Debian mirrors and install it using dpkg-cross:
# dpkg-cross -i -a arm libc6_<version>_armel.deb
Point QEMU to the target linux loader
Under multiarch the target arch loader is in the usual place (/lib/<triplet>) so nothing special is needed. If using dpkg-cross it's installed in a non-standard path so you need to tell QEMU about that.
for example, for the armel architecture: add the line
EXTRA_OPTS="-L /usr/arm-linux-gnueabi"
to the /etc/qemu-binfmt.conf.
for armhf add: EXTRA_OPTS="-L /usr/arm-linux-gnueabihf"
Testing the emulation environment
We will use the "hello" ARM Debian package to test the new environment.
Download the hello package (e.g. from http://deb.debian.org/debian/pool/main/h/hello/hello_version_armel.deb)
- Unpack it with the command:
$ dpkg -x hello_version_armel.deb /tmp/hello_armel
Finally, run the hello executable with:
$ /tmp/hello_armel/usr/bin/hello
It should print "Hello, world!".
That's it! You can now run non-native executables transparently, as long as QEMU supports the system calls used by it.
Appendix: chrooting into target file systems
To be able to chroot into a target file system, the qemu emulator for the target CPU needs to be available. You need first to install the qemu-user-static package:
# apt install qemu-user-static
If you are using stretch or earlier, it also needs to be accessible from inside the chroot jail. From Debian buster and later, this isn't necessary because Linux will use the qemu binary from the host system instead of from the chroot. This means that you can use the dynamically linked qemu on buster or later but not stretch or earlier because the host libraries will not be accessible from inside the chroot.
If you are using stretch or earlier, make the emulator available for the target architecture inside the chroot at the path registered by binfmt-support. You can either bind-mount the binary into the chroot, or just copy it. Copying it means you won't have to copy it again, but it also means it won't get updates, while bind-mounting means you have to bind-mount For example, for an ARM target file system, you need to do one of the following:
# mount --bind /usr/bin/qemu-arm-static /target_fs/usr/bin # cp /usr/bin/qemu-arm-static /target_fs/usr/bin
If you are using systemd, then you can use systemd-nspawn to start a container:
# apt install systemd-container # systemd-nspawn -D /target_fs/
If you aren't using systemd-nspawn, then you may want to mount special filesystems for devices, terminals and processes from the host in the chroot:
for f in dev dev/pts sys proc run ; do mount --bind /$f tmp/$f ; done
You should now be able to chroot into the file system:
# chroot /target_fs/
See Also
TODO
- Modify Scratchbox to take advantage of the new qemu packages, instead of using its own internal qemu (possible?).