Testing the SameKernel builds. Please help if you can, write to us on #mempo (?Mempo)

Tests

Here we paste research data.

v0.1.22-rc1 buildA

Build version: https://github.com/mempo/deterministic-kernel/tree/v0.1.22-rc1

test in progress, using new dpkg and --build-id=none , maybe this time it will all work?
Authors: rfree
Kernel version: 3.2.54

v0.1.21-rc1 buildA

Build version: https://github.com/mempo/deterministic-kernel/tree/v0.1.21-rc1

Authors: members of Mempo project
Kernel version: 3.2.54
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ 
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime from env.sh in our script
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7 amd64, gcc version 4.7.2-5 linux kernel 3.10.9-xxxx-grs-ipv6-64 (OVH server); 
Build date: 2014-01-07
Machine name: (m1)
Settings: HOST=? USER="kernelbuild", PATH="/home/kernelbuild/" 

Results:
vmlinuz - different
other things not tested, should be as in last tests but .gz fixed

On this machine the file vmlinux (we check the unpacked file right after built, not vmlinuz) differs, around address ffffffff8141c167, that is in section ffffffff8141c000 <__start_notes> Used objdump -d vmlinux on the builded kernel file from run.sh.

See this file: vmlinuz-ovh-difference-notes.txt difference between 2 compilations is eg:

 ffffffff8141c162:      00 00                   add    %al,(%rax)
 ffffffff8141c164:      47                      rex.RXB
 ffffffff8141c165:      4e 55                   rex.WRX push %rbp
-ffffffff8141c167:      00 64 ce fe             add    %ah,-0x2(%rsi,%rcx,8)
-ffffffff8141c16b:      0d 9c ac 80 f6          or     $0xf680ac9c,%eax
-ffffffff8141c170:      44 93                   rex.R xchg %eax,%ebx
-ffffffff8141c172:      50                      push   %rax
-ffffffff8141c173:      09 78 a3                or     %edi,-0x5d(%rax)
-ffffffff8141c176:      72 2c                   jb     ffffffff8141c1a4 <__start___ex_table+0x24>
-ffffffff8141c178:      02 46 d5                add    -0x2b(%rsi),%al
-ffffffff8141c17b:      06                      (bad)  
+ffffffff8141c167:      00 6c 94 92             add    %ch,-0x6e(%rsp,%rdx,4)
+ffffffff8141c16b:      49 c4                   rex.WB (bad) 
+ffffffff8141c16d:      db 48 1e                fisttpl 0x1e(%rax)
+ffffffff8141c170:      ff                      (bad)  
+ffffffff8141c171:      f9                      stc    
+ffffffff8141c172:      ee                      out    %al,(%dx)
+ffffffff8141c173:      63 7e 7a                movslq 0x7a(%rsi),%edi
+ffffffff8141c176:      b0 fa                   mov    $0xfa,%al
+ffffffff8141c178:      7a c7                   jp     ffffffff8141c141 <__start_notes+0x141>
+ffffffff8141c17a:      38 2a                   cmp    %ch,(%rdx)

Test20131212

Authors: members of Mempo project
Build version: d51deed258 (https://github.com/mempo/deterministic-kernel/blob/master/run.sh)
Kernel version: 3.2.53
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ 
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime 2013-11-24 20:32:00 ?
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46; 
Build date: 2013-12-12
Machine name: (t/wb)
Settings: HOST="paleale" USER="kernelbuild", PATH="/home/kernelbuild/" 

Results:
.o - all identical
.ko - all inentical
vmlinuz - identical
.deb - different
.gz - different (wasn't unpacked)
Symlinks - ommited

List of different files:

linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/Buildinfo.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/Changes.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/buildinfo.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/changelog.Debian.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/conf.vars.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/debian.README.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/ChangeLog.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_postinst.d/link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_postrm.d/link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/header_prerm.d/link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/force-build-link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/grub_conf.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/initramfs.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/symlink_hook.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postinst.d/yaird.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/force-build-link.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/grub_rm.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/initramfs.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/kernel/postrm.d/yaird.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/etc/sample.kernel-img.conf.gz
linux-image-3.2.53-grsec-mempo.good.0.1.18_01_amd64/usr/share/doc/linux-image-3.2.53-grsec-mempo.good.0.1.18/examples/sample.module.control.gz

Disabled grsecurity option CONFIG_PAX_LANTENT_ENTROPY makes *.ko files deterministic - these files compiled twice have the same checksums. This option generates some entropy during boot.
Now we working with *.gz files - now their checksums are different, but contain files with the same checksums.

Test20131127

Authors: members of Mempo project
Kernel version: 3.2.52 
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ 
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Build tool: using mempo script .
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime 2013-11-24 20:32:00
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46; 
Build date: 2013-11-27
Machine name: (t/wb)
Settings: HOST="workbench" USER="gb-kernel-builder", PATH="/home/gb-kernel-builder/kernel-build/" 

Results:
.o - all identical
.ko - all inentical 
vmlinuz - identical
.deb - different
.gz - different (but content seems identical when unpacked. watch out for external symlinks)

Disabled grsecurity option CONFIG_PAX_LANTENT_ENTROPY makes *.ko files deterministic - these files compiled twice have the same checksums. This option generates some entropy during boot.
Now we working with *.gz files - now their checksums are different, but contain files with the same checksums.

Test20131029

Authors: members of Mempo project
Kernel version: 3.2.51 
Kernel deterministic patches: custom patch to remove __TIME__ and __DATE__ 
Kernel extra patches: grsecurity patch
Kernel was built: 2 times.
Build tool: using mempo script.
Computer: built on same computer each time.
Directory: built in same directory path each time.
Fakedate: yes, using faketime 2013-10-19 12:58:00
Dpkg: not fixed (regular version from Debian 7)
System: build on Debian 7.1 amd64, gcc version 4.7.2-1, linux kernel 3.2.46; 
Build date: 2013-10-29
Machine name: (t/wb)
Settings: HOST="workbench" USER="gb-kernel-builder", PATH="/home/gb-kernel-builder/kernel-build/" (afair)

Results:
.o - all identical
.ko - some match, not all (22909 files are the same and 2539 are different) (SOLVED)
vmlinuz - different
.deb - different

List of checksums:

Expamle different .ko files:

Example with differences dissassembled .ko files:

We use to disassemble "objdump -d" command.

CategoryKernel