This allows to build Kernel in verificable way to know if .deb indeed matches the source code of kernel without added backdoors/viruses.
To use it just execute the 2 simple commands described in section "Install" below.
To get all needed files, on Debian use command: git clone https://github.com/mempo/deterministic-kernel.git
This is work in progress, ?Mempo team will help you in using/testing this software, contact us on #mempo and #debian-kernel on irc.oftc.net (see http://www.oftc.net/) or use this chat-webpage (for anon. use I2P irc #mempo) or see ?Mempo-Contact
As of now (2013-12) this work is:
- about Linux kernel only, testing on amd64
- work in progress - describing the research of this topic
We are at Step3 and we test dpkg 1.17.5 https://wiki.debian.org/ReproducibleBuilds#A.7Bdata.2Ccontrol.7D.tar..7Bgz.2Cxz.2Cbz2.7D_does_not_have_timestamps To build this version dpkg we use command debuild -us -uc -B -j2 || echo "Failed to build".
We already can prove that built binary == given sources, but:
- by hand (unpack the .deb, unpack .gz inside them, then compare - identical)
- under build conditions: matching compiler, hostname(probably not?), username(probably not?), build dir path.
using: faketime; tiny kernel patch to kill TIME (and date) in sources; avoid grsecurity option that causes random .ko;
Progress
Steps of this project:
- Step1 configure kernel build to create identical intermittent files: .o (solved)
- Step2 configure kernel build to create identical final files: .ko (solved) and vmlinuz image
- Step3b identical .gz
- Step3 configure kernel build to create identical .deb
- Step4 have less constrains - any real date (done), hostname(?), user(?), buildpath
See general instructions: ReproducibleBuilds how ever they are mainly for step 2,3.
We are trying to create a script that does this deterministic build in a bit more automated way with added:
- verification of downloaded sources (check against hardcoded in script list of expected checksums of sources; also check PGP signature with hardcoded public key of kernel developers)
apply security patches for the ?Mempo subproject
grsecurity patch - http://grsecurity.net/
- misc patches if needed (e.g. quick fixes regarding security)
This page should be usable for everyone in Debian, and script we're writing will be later easy to run in pure-Debian mode too.
Trust chain
For ?Mempo-Kernel: obtain the .deb from mempo repository, then check checksum of it with trustworthy people who did build .deb from source and seen it produces same binary as in repository.
- YOU: check checksum of .deb and look for trusted 3rd party signed message that such .deb is fine.
- Volunteers: will spend the 2-3 hours needed to build and verify our .deb and then confirm you can trust the .deb with given checksum.
So the full chain trust is:
- you should install the .deb of Kernel
- if this is the official Debian Kernel, after Debian will use the script/technique described here, then this .deb is signed by apt-get GPG key so you trust it implicitly - all done. And you are able to repeat the build process to verify if the apt-get GPG key was not compromised.
if this is for the ?Mempo-Kernel, then until this flavour is hopefully included in Official Debian as another kernel option, it is not signed by GPG key of Debian official apt-get.
in this case, verify if the .deb file you downloaded is signed by security@mempo.org
but how do you verify is security@mempo.org is not compromised nor malicious?
- for this verification, you can build the kernel yourself as described here
- compare the content of provided and builded .deb with eachother
- if matching, you can publish that "Kernel .deb file with checksum sha512:........ was unpacked and verified to match results of compilation from source by me" so others can do it easier.
In time, we will upgrade the build script to have the final .deb file identical, then procedure is even faster (just run run.sh and publish sha512 of your .deb).
?Mempo project might release the .deb in own repository for easy installation for people that want this.
Ultimately, Debian.org might one day officially include Mempo-kernel in Debian repository (though, even then checksum based verification will be more secure, in case of ftp master key being compromised).
Install
You can help yourself and this project easily!
As root install dependencies:
apt-get install faketime time git build-essential libncurses5-dev libncursesw5-dev kernel-package md5deep gcc-4.7-plugin-dev g++ make time -y
Also build dpkg version from jessie (assuming you build on wheezy):
Disable (uninstall) unattended-upgrades if you use it !!! otherwise you risk heaving system demaged perhaps (half-updated to jessie)
In /etc/apt/sources.list add jessie lines:
deb http://ftp.pl.debian.org/debian/ jessie main deb-src http://ftp.pl.debian.org/debian/ jessie main deb http://security.ftp.pl.debian.org/debian/ jessie main
do aptitude update.
login to some user, download sources apt-get sources -t jessie dpkg
- enter dir cd dpkg-1* that was created
build dpkg: run debuild -us -uc -B -j2
when builded, switch back to root and install dpkg -i *.deb the created packages like dpkg_... and dselect...
edit back /etc/apt/sources.list to normal version
do aptitude update. reinstall/enable unattended-upgrades is you disabled it
do dpkg -l dpkg to see what version you have, we need dpkg >= 1.17.5 though according to other documentation, >= 1.17.1 perhaps can work.
On other vesions od dpkg the script still will work mostly, all the files inside of .deb should be the same, how ever the .deb file itself can have other checksum
Create unix user kernelbuild preferably (or, on any user) and in home directory run: press ENTER to confirm (e.g. the download) and then kernel should build Resources: https://github.com/mempo/deterministic-kernel https://github.com/Happuri/deterministic-kernel If important, then we builded on settings: computer hostname was "workbench", user name was (since 2013-12-09) "kernelbuild", working directory/path was "/home/kernelbuild/deterministic-kernel/" for most tests, if you use same and below instructions then you should get identical result to amount described here.
Here we paste research data.
Build version: https://github.com/mempo/deterministic-kernel/tree/v0.1.21-rc1 On this machine the file See this file: vmlinuz-ovh-difference-notes.txt difference between 2 compilations is eg:
List of checksums: different files: 3.2.51-2013.10.29-compilation-difffiles.txt same files: 3.2.51-2013.10.29-compilation-samefiles.txt Expamle different .ko files: Example with differences dissassembled .ko files: We use to disassemble "objdump -d" command. rm -rf deterministic-kernel/ ; git clone https://github.com/mempo/deterministic-kernel.git && cd deterministic-kernel/ && bash run.sh
Tests
v0.1.21-rc1 buildA
Authors: members of Mempo project
Kernel version: 3.2.54
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime from env.sh in our script
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7 amd64, gcc version 4.7.2-5 linux kernel 3.10.9-xxxx-grs-ipv6-64 (OVH server);
Build date: 2014-01-07
Machine name: (m1)
Settings: HOST=? USER="kernelbuild", PATH="/home/kernelbuild/"
Results:
vmlinuz - different
other things not tested, should be as in last tests but .gz fixed
ffffffff8141c162: 00 00 add %al,(%rax)
ffffffff8141c164: 47 rex.RXB
ffffffff8141c165: 4e 55 rex.WRX push %rbp
-ffffffff8141c167: 00 64 ce fe add %ah,-0x2(%rsi,%rcx,8)
-ffffffff8141c16b: 0d 9c ac 80 f6 or $0xf680ac9c,%eax
-ffffffff8141c170: 44 93 rex.R xchg %eax,%ebx
-ffffffff8141c172: 50 push %rax
-ffffffff8141c173: 09 78 a3 or %edi,-0x5d(%rax)
-ffffffff8141c176: 72 2c jb ffffffff8141c1a4 <__start___ex_table+0x24>
-ffffffff8141c178: 02 46 d5 add -0x2b(%rsi),%al
-ffffffff8141c17b: 06 (bad)
+ffffffff8141c167: 00 6c 94 92 add %ch,-0x6e(%rsp,%rdx,4)
+ffffffff8141c16b: 49 c4 rex.WB (bad)
+ffffffff8141c16d: db 48 1e fisttpl 0x1e(%rax)
+ffffffff8141c170: ff (bad)
+ffffffff8141c171: f9 stc
+ffffffff8141c172: ee out %al,(%dx)
+ffffffff8141c173: 63 7e 7a movslq 0x7a(%rsi),%edi
+ffffffff8141c176: b0 fa mov $0xfa,%al
+ffffffff8141c178: 7a c7 jp ffffffff8141c141 <__start_notes+0x141>
+ffffffff8141c17a: 38 2a cmp %ch,(%rdx)
Test20131212
Authors: members of Mempo project
Build version: d51deed258 (https://github.com/mempo/deterministic-kernel/blob/master/run.sh)
Kernel version: 3.2.53
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime 2013-11-24 20:32:00 ?
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46;
Build date: 2013-12-12
Machine name: (t/wb)
Settings: HOST="paleale" USER="kernelbuild", PATH="/home/kernelbuild/"
Results:
.o - all identical
.ko - all inentical
vmlinuz - identical
.deb - different
.gz - different (wasn't unpacked)
Symlinks - ommited
List of different files:
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/Buildinfo.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/Changes.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/buildinfo.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/changelog.Debian.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/conf.vars.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/debian.README.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/ChangeLog.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_postinst.d/link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_postrm.d/link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_prerm.d/link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/force-build-link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/grub_conf.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/initramfs.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/symlink_hook.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/yaird.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/force-build-link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/grub_rm.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/initramfs.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/yaird.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/sample.kernel-img.conf.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/sample.module.control.gz
Disabled grsecurity option CONFIG_PAX_LANTENT_ENTROPY makes *.ko files deterministic - these files compiled twice have the same checksums. This option generates some entropy during boot.
Now we working with *.gz files - now their checksums are different, but contain files with the same checksums.
Test20131127
Authors: members of Mempo project
Kernel version: 3.2.52
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Build tool: using mempo script .
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime 2013-11-24 20:32:00
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46;
Build date: 2013-11-27
Machine name: (t/wb)
Settings: HOST="workbench" USER="gb-kernel-builder", PATH="/home/gb-kernel-builder/kernel-build/"
Results:
.o - all identical
.ko - all inentical
vmlinuz - identical
.deb - different
.gz - different (but content seems identical when unpacked. watch out for external symlinks)
Disabled grsecurity option CONFIG_PAX_LANTENT_ENTROPY makes *.ko files deterministic - these files compiled twice have the same checksums. This option generates some entropy during boot.
Now we working with *.gz files - now their checksums are different, but contain files with the same checksums.
Test20131029
Authors: members of Mempo project
Kernel version: 3.2.51
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Build tool: using mempo script.
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime 2013-10-19 12:58:00
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46;
Build date: 2013-10-29
Machine name: (t/wb)
Settings: HOST="workbench" USER="gb-kernel-builder", PATH="/home/gb-kernel-builder/kernel-build/" (afair)
Results:
.o - all identical
.ko - some match, not all (22909 files are the same and 2539 are different) (SOLVED)
vmlinuz - different
.deb - different