15063
Comment:
|
15176
|
Deletions are marked like this. | Additions are marked like this. |
Line 24: | Line 24: |
== Progress == | == Progress == |
Line 115: | Line 115: |
On this machine the file '''vmlinuz''' differs, around address ffffffff8141c167, that is in section ffffffff8141c000 <__start_notes> download file [[attachment:vmlinuz-ovh-difference-notes.txt]] |
On this machine the file '''vmlinuz''' differs, around address ffffffff8141c167, that is in section `ffffffff8141c000 <__start_notes>` Used `objdump -d vmlinuz` on the builded kernel file from run.sh. See this file: [[attachment:vmlinuz-ovh-difference-notes.txt]] difference between 2 compilations is eg: |
This allows to build Kernel in verificable way to know if .deb indeed matches the source code of kernel without added backdoors/viruses.
To use it just execute the 2 simple commands described in section "Install" below.
To get all needed files, on Debian use command: git clone https://github.com/mempo/deterministic-kernel.git
This is work in progress, ?Mempo team will help you in using/testing this software, contact us on #mempo and #debian-kernel on irc.oftc.net (see http://www.oftc.net/) or use this chat-webpage (for anon. use I2P irc #mempo) or see ?Mempo-Contact
As of now (2013-12) this work is:
- about Linux kernel only, testing on amd64
- work in progress - describing the research of this topic
We are at Step3 and we test dpkg 1.17.5 https://wiki.debian.org/ReproducibleBuilds#A.7Bdata.2Ccontrol.7D.tar..7Bgz.2Cxz.2Cbz2.7D_does_not_have_timestamps To build this version dpkg we use command debuild -us -uc -B -j2 || die "Failed to build".
We already can prove that built binary == given sources, but:
- by hand (unpack the .deb, unpack .gz inside them, then compare - identical)
- under build conditions: matching compiler, hostname(probably not?), username(probably not?), build dir path.
using: faketime; tiny kernel patch to kill TIME (and date) in sources; avoid grsecurity option that causes random .ko;
Progress
Steps of this project:
- Step1 configure kernel build to create identical intermittent files: .o (solved)
- Step2 configure kernel build to create identical final files: .ko (solved) and vmlinuz image
- Step3b identical .gz
- Step3 configure kernel build to create identical .deb
- Step4 have less constrains - any real date (done), hostname(?), user(?), buildpath
See general instructions: ReproducibleBuilds how ever they are mainly for step 2,3.
We are trying to create a script that does this deterministic build in a bit more automated way with added:
- verification of downloaded sources (check against hardcoded in script list of expected checksums of sources; also check PGP signature with hardcoded public key of kernel developers)
apply security patches for the ?Mempo subproject
grsecurity patch - http://grsecurity.net/
- misc patches if needed (e.g. quick fixes regarding security)
This page should be usable for everyone in Debian, and script we're writing will be later easy to run in pure-Debian mode too.
Trust chain
For ?Mempo-Kernel: obtain the .deb from mempo repository, then check checksum of it with trustworthy people who did build .deb from source and seen it produces same binary as in repository.
- YOU: check checksum of .deb and look for trusted 3rd party signed message that such .deb is fine.
- Volunteers: will spend the 2-3 hours needed to build and verify our .deb and then confirm you can trust the .deb with given checksum.
So the full chain trust is:
- you should install the .deb of Kernel
- if this is the official Debian Kernel, after Debian will use the script/technique described here, then this .deb is signed by apt-get GPG key so you trust it implicitly - all done. And you are able to repeat the build process to verify if the apt-get GPG key was not compromised.
if this is for the ?Mempo-Kernel, then until this flavour is hopefully included in Official Debian as another kernel option, it is not signed by GPG key of Debian official apt-get.
in this case, verify if the .deb file you downloaded is signed by security@mempo.org
but how do you verify is security@mempo.org is not compromised nor malicious?
- for this verification, you can build the kernel yourself as described here
- compare the content of provided and builded .deb with eachother
- if matching, you can publish that "Kernel .deb file with checksum sha512:........ was unpacked and verified to match results of compilation from source by me" so others can do it easier.
In time, we will upgrade the build script to have the final .deb file identical, then procedure is even faster (just run run.sh and publish sha512 of your .deb).
?Mempo project might release the .deb in own repository for easy installation for people that want this.
Ultimately, Debian.org might one day officially include Mempo-kernel in Debian repository (though, even then checksum based verification will be more secure, in case of ftp master key being compromised).
Install
You can help yourself and this project easily!
As root install dependencies:
apt-get install faketime time git build-essential libncurses5-dev libncursesw5-dev kernel-package md5deep gcc-4.7-plugin-dev g++ make time -y
Create unix user kernelbuild preferably (or, on any user) and in home directory run:
rm -rf deterministic-kernel/ ; git clone https://github.com/mempo/deterministic-kernel.git && cd deterministic-kernel/ && bash run.sh
press ENTER to confirm (e.g. the download) and then kernel should build
Resources: https://github.com/mempo/deterministic-kernel https://github.com/Happuri/deterministic-kernel
If important, then we builded on settings: computer hostname was "workbench", user name was (since 2013-12-09) "kernelbuild", working directory/path was "/home/kernelbuild/deterministic-kernel/" for most tests, if you use same and below instructions then you should get identical result to amount described here.
Tests
Here we paste research data.
v0.1.21-rc1 buildA
Build version: https://github.com/mempo/deterministic-kernel/tree/v0.1.21-rc1
Authors: members of Mempo project Kernel version: 3.2.54 Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ Kernel extra patches: grsecurity patch Kernel was built: 2 times. Computer: built on same computer each time. Directory: built in same directory path each time. Fakedate: yes, using faketime from env.sh in our script Dpkg: not fixed (regular version from Debian 7) System: build on Debian 7 amd64, gcc version 4.7.2-5 linux kernel 3.10.9-xxxx-grs-ipv6-64 (OVH server); Build date: 2014-01-07 Machine name: (m1) Settings: HOST=? USER="kernelbuild", PATH="/home/kernelbuild/" Results: vmlinuz - different other things not tested, should be as in last tests but .gz fixed
On this machine the file vmlinuz differs, around address ffffffff8141c167, that is in section ffffffff8141c000 <__start_notes> Used objdump -d vmlinuz on the builded kernel file from run.sh.
See this file: vmlinuz-ovh-difference-notes.txt difference between 2 compilations is eg:
ffffffff8141c162: 00 00 add %al,(%rax) ffffffff8141c164: 47 rex.RXB ffffffff8141c165: 4e 55 rex.WRX push %rbp -ffffffff8141c167: 00 64 ce fe add %ah,-0x2(%rsi,%rcx,8) -ffffffff8141c16b: 0d 9c ac 80 f6 or $0xf680ac9c,%eax -ffffffff8141c170: 44 93 rex.R xchg %eax,%ebx -ffffffff8141c172: 50 push %rax -ffffffff8141c173: 09 78 a3 or %edi,-0x5d(%rax) -ffffffff8141c176: 72 2c jb ffffffff8141c1a4 <__start___ex_table+0x24> -ffffffff8141c178: 02 46 d5 add -0x2b(%rsi),%al -ffffffff8141c17b: 06 (bad) +ffffffff8141c167: 00 6c 94 92 add %ch,-0x6e(%rsp,%rdx,4) +ffffffff8141c16b: 49 c4 rex.WB (bad) +ffffffff8141c16d: db 48 1e fisttpl 0x1e(%rax) +ffffffff8141c170: ff (bad) +ffffffff8141c171: f9 stc +ffffffff8141c172: ee out %al,(%dx) +ffffffff8141c173: 63 7e 7a movslq 0x7a(%rsi),%edi +ffffffff8141c176: b0 fa mov $0xfa,%al +ffffffff8141c178: 7a c7 jp ffffffff8141c141 <__start_notes+0x141> +ffffffff8141c17a: 38 2a cmp %ch,(%rdx)
Test20131212
Authors: members of Mempo project Build version: d51deed258 (https://github.com/mempo/deterministic-kernel/blob/master/run.sh) Kernel version: 3.2.53 Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ Kernel extra patches: grsecurity patch Kernel was built: 2 times. Computer: built on same computer each time. Directory: built in same directory path each time. Fakedate: yes, using faketime 2013-11-24 20:32:00 ? Dpkg: not fixed (regular version from Debian 7) System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46; Build date: 2013-12-12 Machine name: (t/wb) Settings: HOST="paleale" USER="kernelbuild", PATH="/home/kernelbuild/" Results: .o - all identical .ko - all inentical vmlinuz - identical .deb - different .gz - different (wasn't unpacked) Symlinks - ommited List of different files: linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/Buildinfo.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/Changes.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/buildinfo.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/changelog.Debian.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/conf.vars.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/debian.README.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/ChangeLog.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_postinst.d/link.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_postrm.d/link.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_prerm.d/link.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/force-build-link.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/grub_conf.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/initramfs.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/symlink_hook.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/yaird.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/force-build-link.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/grub_rm.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/initramfs.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/yaird.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/sample.kernel-img.conf.gz linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/sample.module.control.gz Disabled grsecurity option CONFIG_PAX_LANTENT_ENTROPY makes *.ko files deterministic - these files compiled twice have the same checksums. This option generates some entropy during boot. Now we working with *.gz files - now their checksums are different, but contain files with the same checksums.
Test20131127
Authors: members of Mempo project Kernel version: 3.2.52 Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ Kernel extra patches: grsecurity patch Kernel was built: 2 times. Build tool: using mempo script . Computer: built on same computer each time. Directory: built in same directory path each time. Fakedate: yes, using faketime 2013-11-24 20:32:00 Dpkg: not fixed (regular version from Debian 7) System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46; Build date: 2013-11-27 Machine name: (t/wb) Settings: HOST="workbench" USER="gb-kernel-builder", PATH="/home/gb-kernel-builder/kernel-build/" Results: .o - all identical .ko - all inentical vmlinuz - identical .deb - different .gz - different (but content seems identical when unpacked. watch out for external symlinks) Disabled grsecurity option CONFIG_PAX_LANTENT_ENTROPY makes *.ko files deterministic - these files compiled twice have the same checksums. This option generates some entropy during boot. Now we working with *.gz files - now their checksums are different, but contain files with the same checksums.
Test20131029
Authors: members of Mempo project Kernel version: 3.2.51 Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ Kernel extra patches: grsecurity patch Kernel was built: 2 times. Build tool: using mempo script. Computer: built on same computer each time. Directory: built in same directory path each time. Fakedate: yes, using faketime 2013-10-19 12:58:00 Dpkg: not fixed (regular version from Debian 7) System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46; Build date: 2013-10-29 Machine name: (t/wb) Settings: HOST="workbench" USER="gb-kernel-builder", PATH="/home/gb-kernel-builder/kernel-build/" (afair) Results: .o - all identical .ko - some match, not all (22909 files are the same and 2539 are different) (SOLVED) vmlinuz - different .deb - different
List of checksums:
different files: 3.2.51-2013.10.29-compilation-difffiles.txt
same files: 3.2.51-2013.10.29-compilation-samefiles.txt
Expamle different .ko files:
Example with differences dissassembled .ko files:
We use to disassemble "objdump -d" command.