Differences between revisions 53 and 54
Revision 53 as of 2014-01-08 07:59:59
Size: 15063
Editor: ?Mempo
Comment:
Revision 54 as of 2014-01-08 08:02:12
Size: 15176
Editor: ?Mempo
Comment:
Deletions are marked like this. Additions are marked like this.
Line 24: Line 24:
== Progress ==  == Progress ==
Line 115: Line 115:
On this machine the file '''vmlinuz''' differs, around address ffffffff8141c167, that is in section ffffffff8141c000 <__start_notes>
download file [[attachment:vmlinuz-ovh-difference-notes.txt]]
On this machine the file '''vmlinuz''' differs, around address ffffffff8141c167, that is in section `ffffffff8141c000 <__start_notes>`
Used `objdump -d vmlinuz` on the builded kernel file from run.sh.

See this file:
[[attachment:vmlinuz-ovh-difference-notes.txt]]
difference between 2 compilations is eg:

This allows to build Kernel in verificable way to know if .deb indeed matches the source code of kernel without added backdoors/viruses.

To use it just execute the 2 simple commands described in section "Install" below.

To get all needed files, on Debian use command: git clone https://github.com/mempo/deterministic-kernel.git

This is work in progress, ?Mempo team will help you in using/testing this software, contact us on #mempo and #debian-kernel on irc.oftc.net (see http://www.oftc.net/) or use this chat-webpage (for anon. use I2P irc #mempo) or see ?Mempo-Contact

As of now (2013-12) this work is:

We already can prove that built binary == given sources, but:

  • by hand (unpack the .deb, unpack .gz inside them, then compare - identical)
  • under build conditions: matching compiler, hostname(probably not?), username(probably not?), build dir path.
  • using: faketime; tiny kernel patch to kill TIME (and date) in sources; avoid grsecurity option that causes random .ko;

Progress

Steps of this project:

  • Step1 configure kernel build to create identical intermittent files: .o (solved)
  • Step2 configure kernel build to create identical final files: .ko (solved) and vmlinuz image
  • Step3b identical .gz
  • Step3 configure kernel build to create identical .deb
  • Step4 have less constrains - any real date (done), hostname(?), user(?), buildpath

See general instructions: ReproducibleBuilds how ever they are mainly for step 2,3.

We are trying to create a script that does this deterministic build in a bit more automated way with added:

  • verification of downloaded sources (check against hardcoded in script list of expected checksums of sources; also check PGP signature with hardcoded public key of kernel developers)
  • apply security patches for the ?Mempo subproject

  • grsecurity patch - http://grsecurity.net/

  • misc patches if needed (e.g. quick fixes regarding security)

This page should be usable for everyone in Debian, and script we're writing will be later easy to run in pure-Debian mode too.

Trust chain

For ?Mempo-Kernel: obtain the .deb from mempo repository, then check checksum of it with trustworthy people who did build .deb from source and seen it produces same binary as in repository.

  • YOU: check checksum of .deb and look for trusted 3rd party signed message that such .deb is fine.
  • Volunteers: will spend the 2-3 hours needed to build and verify our .deb and then confirm you can trust the .deb with given checksum.

So the full chain trust is:

  • you should install the .deb of Kernel
  • if this is the official Debian Kernel, after Debian will use the script/technique described here, then this .deb is signed by apt-get GPG key so you trust it implicitly - all done. And you are able to repeat the build process to verify if the apt-get GPG key was not compromised.
  • if this is for the ?Mempo-Kernel, then until this flavour is hopefully included in Official Debian as another kernel option, it is not signed by GPG key of Debian official apt-get.

  • in this case, verify if the .deb file you downloaded is signed by security@mempo.org

  • but how do you verify is security@mempo.org is not compromised nor malicious?

  • for this verification, you can build the kernel yourself as described here
  • compare the content of provided and builded .deb with eachother
  • if matching, you can publish that "Kernel .deb file with checksum sha512:........ was unpacked and verified to match results of compilation from source by me" so others can do it easier.

In time, we will upgrade the build script to have the final .deb file identical, then procedure is even faster (just run run.sh and publish sha512 of your .deb).

?Mempo project might release the .deb in own repository for easy installation for people that want this.

Ultimately, Debian.org might one day officially include Mempo-kernel in Debian repository (though, even then checksum based verification will be more secure, in case of ftp master key being compromised).

Install

You can help yourself and this project easily!

As root install dependencies:

apt-get install faketime time git build-essential libncurses5-dev libncursesw5-dev kernel-package md5deep gcc-4.7-plugin-dev g++ make time -y  

Create unix user kernelbuild preferably (or, on any user) and in home directory run:

rm -rf deterministic-kernel/ ; git clone https://github.com/mempo/deterministic-kernel.git && cd deterministic-kernel/  && bash run.sh

press ENTER to confirm (e.g. the download) and then kernel should build :)

Resources: https://github.com/mempo/deterministic-kernel https://github.com/Happuri/deterministic-kernel

If important, then we builded on settings: computer hostname was "workbench", user name was (since 2013-12-09) "kernelbuild", working directory/path was "/home/kernelbuild/deterministic-kernel/" for most tests, if you use same and below instructions then you should get identical result to amount described here.


Tests

Here we paste research data.

v0.1.21-rc1 buildA

Build version: https://github.com/mempo/deterministic-kernel/tree/v0.1.21-rc1

Authors: members of Mempo project
Kernel version: 3.2.54
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ 
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime from env.sh in our script
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7 amd64, gcc version 4.7.2-5 linux kernel 3.10.9-xxxx-grs-ipv6-64 (OVH server); 
Build date: 2014-01-07
Machine name: (m1)
Settings: HOST=? USER="kernelbuild", PATH="/home/kernelbuild/" 

Results:
vmlinuz - different
other things not tested, should be as in last tests but .gz fixed

On this machine the file vmlinuz differs, around address ffffffff8141c167, that is in section ffffffff8141c000 <__start_notes> Used objdump -d vmlinuz on the builded kernel file from run.sh.

See this file: vmlinuz-ovh-difference-notes.txt difference between 2 compilations is eg:

 ffffffff8141c162:      00 00                   add    %al,(%rax)
 ffffffff8141c164:      47                      rex.RXB
 ffffffff8141c165:      4e 55                   rex.WRX push %rbp
-ffffffff8141c167:      00 64 ce fe             add    %ah,-0x2(%rsi,%rcx,8)
-ffffffff8141c16b:      0d 9c ac 80 f6          or     $0xf680ac9c,%eax
-ffffffff8141c170:      44 93                   rex.R xchg %eax,%ebx
-ffffffff8141c172:      50                      push   %rax
-ffffffff8141c173:      09 78 a3                or     %edi,-0x5d(%rax)
-ffffffff8141c176:      72 2c                   jb     ffffffff8141c1a4 <__start___ex_table+0x24>
-ffffffff8141c178:      02 46 d5                add    -0x2b(%rsi),%al
-ffffffff8141c17b:      06                      (bad)  
+ffffffff8141c167:      00 6c 94 92             add    %ch,-0x6e(%rsp,%rdx,4)
+ffffffff8141c16b:      49 c4                   rex.WB (bad) 
+ffffffff8141c16d:      db 48 1e                fisttpl 0x1e(%rax)
+ffffffff8141c170:      ff                      (bad)  
+ffffffff8141c171:      f9                      stc    
+ffffffff8141c172:      ee                      out    %al,(%dx)
+ffffffff8141c173:      63 7e 7a                movslq 0x7a(%rsi),%edi
+ffffffff8141c176:      b0 fa                   mov    $0xfa,%al
+ffffffff8141c178:      7a c7                   jp     ffffffff8141c141 <__start_notes+0x141>
+ffffffff8141c17a:      38 2a                   cmp    %ch,(%rdx)

Test20131212

Authors: members of Mempo project
Build version: d51deed258 (https://github.com/mempo/deterministic-kernel/blob/master/run.sh)
Kernel version: 3.2.53
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ 
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime 2013-11-24 20:32:00 ?
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46; 
Build date: 2013-12-12
Machine name: (t/wb)
Settings: HOST="paleale" USER="kernelbuild", PATH="/home/kernelbuild/" 

Results:
.o - all identical
.ko - all inentical
vmlinuz - identical
.deb - different
.gz - different (wasn't unpacked)
Symlinks - ommited

List of different files:

linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/Buildinfo.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/Changes.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/buildinfo.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/changelog.Debian.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/conf.vars.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/debian.README.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/ChangeLog.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_postinst.d/link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_postrm.d/link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_prerm.d/link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/force-build-link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/grub_conf.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/initramfs.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/symlink_hook.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/yaird.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/force-build-link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/grub_rm.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/initramfs.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/yaird.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/sample.kernel-img.conf.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/sample.module.control.gz

Disabled grsecurity option CONFIG_PAX_LANTENT_ENTROPY makes *.ko files deterministic - these files compiled twice have the same checksums. This option generates some entropy during boot.
Now we working with *.gz files - now their checksums are different, but contain files with the same checksums.

Test20131127

Authors: members of Mempo project
Kernel version: 3.2.52 
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ 
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Build tool: using mempo script .
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime 2013-11-24 20:32:00
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46; 
Build date: 2013-11-27
Machine name: (t/wb)
Settings: HOST="workbench" USER="gb-kernel-builder", PATH="/home/gb-kernel-builder/kernel-build/" 

Results:
.o - all identical
.ko - all inentical 
vmlinuz - identical
.deb - different
.gz - different (but content seems identical when unpacked. watch out for external symlinks)

Disabled grsecurity option CONFIG_PAX_LANTENT_ENTROPY makes *.ko files deterministic - these files compiled twice have the same checksums. This option generates some entropy during boot.
Now we working with *.gz files - now their checksums are different, but contain files with the same checksums.

Test20131029

Authors: members of Mempo project
Kernel version: 3.2.51 
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ 
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Build tool: using mempo script.
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime 2013-10-19 12:58:00
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46; 
Build date: 2013-10-29
Machine name: (t/wb)
Settings: HOST="workbench" USER="gb-kernel-builder", PATH="/home/gb-kernel-builder/kernel-build/" (afair)

Results:
.o - all identical
.ko - some match, not all (22909 files are the same and 2539 are different) (SOLVED)
vmlinuz - different
.deb - different

List of checksums:

Expamle different .ko files:

Example with differences dissassembled .ko files:

We use to disassemble "objdump -d" command.

CategoryKernel