|Deletions are marked like this.||Additions are marked like this.|
|Line 29:||Line 29:|
|To disable it, just remove '''all''' lines containing the word grsecurity from script's [[https://github.com/mempo/deterministic-kernel/blob/master/kernel-build/linux-mempo/sources.list|sources.list]]) and then run the [[#build]] again.
(In [[Mempo/future|future]] we would like to also provide here the .deb for non-grsecurity kernels if this is requested).
|To disable it, just remove '''all''' lines containing the word grsecurity from script's [[https://github.com/mempo/deterministic-kernel/blob/master/kernel-build/linux-mempo/sources.list|sources.list]] and then run the [[#build]] again.
(In [[Mempo#future|future]] we would like to also provide here the .deb for non-grsecurity kernels if this is requested).
SameKernel (Deterministic Kernel) is software created in Mempo project for Debian that allows to build Kernel in Verifiable (reproducible, deterministic) way. The resulting .deb are identical each time, so you can be sure no backdoor/trojan was addedd - because your friends confirm they also are getting same checksum from this sources.
Most kernels built here (for now) are in the hardened variant, but you can change that (read below).
- Current status
- Optional Grsecurity
- DOWNLOAD and Releases
- Install kernel from SameKernel
- How this works
- Verification process
- Trust chain
beta-testing as of 2014-02-10:
- Linux kernel only, testing on amd64 (and i386?)
Includes by default grsecurity, but that can be easily removed
Users should be able to rebuild identical kernel that matches the distributed .deb files, to know that they do not contain a virus added on top of the source code.
In our SameKernel script, grsecurity is enabled by default;
To disable it, just remove all lines containing the word grsecurity from script's sources.list and then run the #build again. (In future we would like to also provide here the .deb for non-grsecurity kernels if this is requested).
To fully use Grsecurity kernel you must follow steps in #install!
(Note to editors: please leave this section/anchor even if this topic would be moved)
DOWNLOAD and Releases
wheezy, with grsecurity/mempo, variant "?good"
linux-image-3.2.55-grsec-mempo.good.0.1.28_02_amd64.deb = sha256:19a34375ff5ed5f8ef4577b8679fd9ac278b6fd13876e0d2f5990d14d89d74d9 mirror1 or mirror2 (?)
linux-headers-3.2.55-grsec-mempo.good.0.1.28_02_amd64.deb = sha256:9ab993f142a1fb37d762c0969d73f18e4f6a813b0332d4ba2c18e745fa855f6b mirror1 or mirror2 (?)
wheezy, with grsecurity/mempo, variant "?good"
- wheezy, without grsecurity
- not tested yet (TODO)
Install kernel from SameKernel
Read about known bugs
Install them with dpkg -i thefiles.deb
For grsecurity version you must run the setfattr script for grsecurity (and other instructions there - enable user_xattr) or otherwise JIT-using programs like java, python, firefox, icedove etc., will be prohibited from running! (tip: if you do not have internet or browser is not working already, remember the script to fix firefox is located also in your local copy of SameKernel/deterministic-kernel, search for that filename e.g. in directory apps/.)
if you are interested in security for more kernel-related and other tools see: Mempo#install
Steps of this project:
Step 1 configure kernel build to create identical intermittent files: .o
Step 2 configure kernel build to create identical final files: .ko
Step 3 identical .gz files (docs?)
Step 4 have identical all files and vmlinux including vmlinux .notes
Step 5 have identical .deb files on the same machine
Step 6 have identical .deb files on different machines (this works, provided username kernelbuild and directory /home/kernelbuild/deterministic-kernel/)
Step 6b fix varying xz header/tail between some machines ( /Test#v0.1.23-rc1_buildA )
- Step 7a remove requirement of having same directory
- Step 7b remove requirement of having same username
- Step 8 test if hostname, and timezone are ignored
- Step 9 any other differences between alpha-testers doing the verification
See general instructions for all programs: ReproducibleBuilds .
We created a script that does this deterministic build in a bit more automated way with added:
- verification of downloaded sources (check against hardcoded in script list of expected checksums of sources; also check PGP signature with hardcoded public key of kernel developers)
apply security patches for the Mempo subproject
grsecurity patch - http://grsecurity.net/
- misc patches if needed (e.g. quick fixes regarding security)
This page should be usable for everyone in Debian, and script we're writing will be later easy to run in pure-Debian mode too.
Build from sources easily - get the software and publish (PGP-signed) cheksums confirmation so that other users can trust the binary *.deb we publish - please consider doing: #create_verification.
Extra tools that are not-standard-wheezy and are installed globally:
GNU gettext >= 0.18.2 (install them from official debian-wheezy backports)
Extra tools that do not need any non-standard global changes:
User local installation of dpkg pu/reproducible_builds - instructions below
As root from normal repository install faketime (in wheezy) and other common programs
Prepare dependencies for Build
As root :
apt-get install faketime time git build-essential libncurses5-dev libncursesw5-dev kernel-package md5deep gcc-4.7-plugin-dev g++ make time automake pkg-config flex
Actual and authorized depencies are in README.md
Create user for kernel build
At this moment, to get the same *.deb kernel files on different machines, build must be ran on the same user!
Create unix user kernelbuild.
Get and build dpkg
We are using Lunar's branch of dpkg: pu/reproducible_builds. We need GNU gettext >= 0.18.2, therefore on wheezy please add (skip this step if you're building .deb files on Jessie!!!):
deb http://''YOURMIRROR''.debian.org/debian wheezy-backports main
to /etc/apt/sources.list and run:
# aptitude update # aptitude install -t wheezy-backports gettext autopoint
Please use Mempo's script: https://github.com/mempo/mempo-deb/tree/master/pack/dpkg that will fetch, build and install reproducible-dpkg locally (no need to upgrade your system wide dpkg) (it will delete our old directory in home without asking)
cd ~ rm -rf mempo-deb/ git clone https://github.com/mempo/mempo-deb cd mempo-deb/pack/dpkg
Verify version of git repository with the command (check if this is ?Mempo/#pgp signed correctly)
LANG=C git tag -v `git describe --tags`
Build and install reproducible-dpkg.
Run kernel compilation
In home directory run (it will delete our old directory in home without asking)
cd ~ rm -rf deterministic-kernel/ git clone https://github.com/mempo/deterministic-kernel.git cd deterministic-kernel/
Verify version of git repository with the command (check if this is ?Mempo/#pgp signed correctly):
LANG=C git tag -v `git describe --tags`
press ENTER to confirm (e.g. the download) and then kernel should build
(Or better execute this by hand and check sha1sum of git version)
How this works
Following fixes are applied:
- (TODO update this doc)
Set anv options like USER, HOST
Replace TIME with given time in kernel sources (maybe not needed since faketime, however it's cleaner solution to the problem)
Overlay fix-remove-debuglink.patch (probably not needed because of pu/reproducible_builds debhelper)
Overlay fix-kpkg-gz.patch (probably not needed because of pu/reproducible_builds dpkg)
Change to build-id=none for vmlinux to fix it quickly
Check exact version of libc6, gcc, and other tools that embed their name in elf binaries in headers package
- Turn back on buildid for vmlinux, if the exact versions are forced then it should be the same each time? (TODO)
Tar --mtime and sorting files are managed by reproducible dpkg
Deterministic ar is set by reproducible dpkg
Force always same XZ options: CRC and compression in our dpkg from Mempo
First your friend, trusted person, or even you yourself should #build the kernel from sources and publish resulting checksums of deb files.
Then everyone can download the .deb files, verify if the checksum is as published and only then install them (also verify auxiliary files like the fsattr script here).
Then you can be sure that the .deb file at least is matching the source code, without hidden surprises that would be not visible from reviewing the sources.
We (Mempo) are looking for various people, known to community as freedom or FOSS supporters, to help this project in multi-verification of the code.
This are multi-signatures - the idea to have multiple parties providing independent signatures for given .deb file(s), so that user could choose to wait for few separate confirmations from various people before installing a package in order to avoid binary malicious or backdoored version even when the 1 person doing compilation&signature gets compromised (it would then require to compromise many or even all verification developers).
It will probably work better with signing individual .deb files, then when signing entire repository. Therefore it was suggested (<pabs>) to possibly check out dpkg-sig (e.g see this article).
One idea is to provide files like:
linux-image-3.2.55-grsec-mempo.good.0.1.28_02_amd64.deb.verified.by-bob.gpg linux-image-3.2.55-grsec-mempo.good.0.1.28_02_amd64.deb.verified.by-alice.gpg linux-image-3.2.55-grsec-mempo.good.0.1.28_02_amd64.deb.verified.by-rfree.gpg
This files would be gpg signed text files with expected checksums and some other information, including checksum of the source code that was built.
Doing this is possible today, thought there is no tool yet to download/verify this data automatically.
Help in verification
The work is quite trivial - please do the following to help us and all other users:
execute the #build commands including installing our dpkg (locally on kernelbuild user)
- look at the resulting .deb files checksums
fill in the form below, sign it with gnupg and paste bin somewhere and notify us on IRC Mempo#Contact also ask us if anything below is not clear.
software=mempo/deterministic-kernel sources-url=https://github.com/mempo/deterministic-kernel.git sources-sha1=.......... sources-tag=.......... sources-sign-by-fpr=.......... my-name-or-pseudonym=.......... my-code-review=.......... (see below) my-grsecurity-sigchecked=.......... (see below) my-checksum=.......... .......... my-checksum=.......... .......... my-computer-system=.......... my-computer-kernel=.......... my-computer-arch=.......... my-computer-type=.......... my-computer-trust=.......... (see below) my-tools-sha1-mempo-deb-dpkg=.......... [details] <human> --Following "cannary" lines scan be deleted, but it's preferred to have it. Also DELETE THIS comment line please.-- <cannary>I hereby freely say that I am not, and I never was, and I do not suspect to be in foreseeable future, under ANY form of NDA or gag order (nor ANY influence) that could in any way affect this verification testimony. (I am in law jurisdiction: <j>.........</j>).</cannary> I, ...... (known from/related to project: ......) can confirm, that I downloaded the source code with git sha1 sum as in line sources-sha1= above, and I built it on my system as described above, and that these sources did in fact produced the files with checksums as listed above in my-checksum= lines, and that the said build was done under following conditions: * value of line my-computer-trust= means security of computer used by me for this: 0-50 is regular, 50-100 is hardened, 100-200 is +always airgap, 200-300 is +openhardware computer, best self-build * value of my-code-review= means how I reviewed the primary source code (the sources that have checksum sources-sha1=) as follows: 0=no check; 2=glanced over the code; 4=reviewed by comparing to previously reviewed versions; 8=reviewed from scratch. 10=did deep review. (for all this values, the review applies to files as downloaded from git, with EXCLUSION of the grsecurity*patch file). WARNING: all extra code (e.g. downloaded by running this script) should be checksummed on download by primary source code, otherwise the compilation outcome is not guaranteed to be meaningful. * my-tools-sha1... list sha1sums (e.g. git name) of other sources that I build in order to build this code (e.g. the mempo-deb dpkg). * as in my-grsecurity-sigchecked I did[/not] verified grsecurity*path files as correctly signed by the Spender's pubkey from http://grsecurity.net </human>
The information to fill in about sources is result of command LANG=C git tag -v git describe --tags and also of lsb_release -a
In the line my-tools-sha1-mempo-deb-dpkg= you should paste the git version of the dpkg that you installed as described in #build (use git log in the mempo-deb repository, where you build our version of dpkg).
Please fill it exactly as in following example, because we plan to auto-process this files in future (and paste the <human>...</human> and read before signing)
software=mempo/deterministic-kernel sources-url=https://github.com/mempo/deterministic-kernel.git sources-sha1=bdfae2b8144c4cf03722fccdf14e8b571320d7a5 sources-tag=v0.1.30-03-rc sources-sign-by-fpr=21A5 9D31 7421 F02E C3C3 81F3 4623 E8F7 4595 3F23 my-name-or-pseudonym=John Doe my-code-review=4 (see below) my-checksum=linux-image-3.2.55-grsec-mempo.good.0.1.28_02_amd64.deb 19a34375ff5ed5f8ef4577b8679fd9ac278b6fd13876e0d2f5990d14d89d74d9 my-checksum=linux-headers-3.2.55-grsec-mempo.good.0.1.28_02_amd64.deb 9ab993f142a1fb37d762c0969d73f18e4f6a813b0332d4ba2c18e745fa855f6b my-computer-system=Debian 7.4 my-computer-kernel=3.2.55-grsec-mempo.good.0.1.28 my-computer-arch=amd64 my-computer-type=intel my-computer-trust=100 my-tools-dpkg-from-sources=3b754a0b57fb235d32648abef8ec176f996d3920 [details] <human> <cannary>I can say that during writing, signing, publishing this testimony I am NOT under ANY form of gag order (nor ANY pressure nor ANY influence) that could in any way affect this code verification testimony. (I am in law jurisdiction: <j>Poland, dependency of EU</j>).</cannary> I, John Doe (e.g. related to John Doe Inc.) can confirm, [... snip ...] </human>
This text should be saved in file named like: thecreated.deb.verified.by-bob and then gpg signed to create thecreated.deb.verified.by-bob.gpg
For Mempo-Kernel: obtain the *.deb from mempo repository, then check checksum of it with trustworthy people who did build *.deb from source and check if it produces same binary as in repository.
- You: check checksum of *.deb and look for trusted 3rd party signed message that such *.deb is fine.
- Volunteers: will spend the 2-3 hours needed to build and verify our *.deb and then confirm you can trust the *.deb with given checksum.
So the full chain trust is:
- You should install the *.deb of Kernel
- If this is the official Debian Kernel, after Debian will use the script/technique described here, then this *.deb is signed by apt-get GPG key so you trust it implicitly - all done and you are able to repeat the build process to verify if the apt-get GPG key was not compromised.
If this is for the Mempo-Kernel, then until this flavour is hopefully included in Official Debian as another kernel option, it is not signed by GPG key of Debian official apt-get.
In this case, verify if the *.deb file you downloaded is signed by firstname.lastname@example.org
But how do you verify is email@example.com is not compromised nor malicious?
- For this verification, you can build the kernel yourself as described here
- Compare the content of provided and built *.deb with each other
- If matching, you can publish that "Kernel *.deb file with checksum sha512:........ was unpacked and verified to match results of compilation from source by me" so others can do it easier.
In time, we will upgrade the build script to have the final *.deb file identical, then procedure is even faster (just run run.sh and publish sha512 of your *.deb).
Mempo project might release the *.deb in own repository for easy installation for people that want this.
Ultimately, Debian.org might one day officially include Mempo-kernel in Debian repository (though, even then checksum based verification will be more secure, in case of ftp master key being compromised).
FAQ posts solutions to common questions and problems:
Bugs - read also the known #bugs below first!
- How can I use the kernel
- What system or computer is this compatible with
Debian Wheezy amd64(meaning 64bit both AMD and Intel). Should also work on i386. And should work on Debian Jessie. We consider building thoes versions too, when we have volunteers/resources
- Any downside - will things run slower?
CPU calculations seemed to be not affected, other things could be a bit slower, see ?grsecurity/#performance
- Any downside - will something not work?
please read grsecurity#compatibility
- Using kernel: Firefox and other programs do not run
- Using kernel: Still some things do not run
some drivers, special programs, some hardware drivers, opengl, opencl could be blocked on this security settings. Other settings of grsecurity in future could be more permissive. Ask us on mempo as well as at #grsecurity. See grsecurity#compatibility
- Using kernel: Mounting remote file systems with sshfs don't work
Solution is included in grsecurity#compatibility section.
- Using kernel: Programs run slower
Grsecurity at high security levels is auditing many system calls and internal operations to make sure even new unknown exploits would be usually blocked. This takes time. CPU operations should work as fast. See grsecurity#performance
- Build: Wrong version of something
if libc differs a bit then it should work but the binary files can differ similar to what happened in #bug2 (that is not our bug - just you need to use correct version of libs). If dpkg, gettext etc - then you must install them as described here.
- Build: Not enough free space
It will probably run out of disk space. If you would try to fix that and create space on another partition then move (or symlink) entire home or the top directory, in the end the working directory must be as expected (at least as of 2014-02-14) script will warn you if the work dir is incorrect.
- Build: Wrong checksum on file (on linux kernel)
probably file was corrupted in the cached download on your hard drive, in ~/Downloads/linux... or in kernel-sources/ where you build SameKernel. Delete this partially download file, and script will re-download. -or- in rare cases it could mean network download error, or actual attack on you (DNS spoof/network takeover - and sending malicious file instead), in such case back up the file and report people you trust / security researchers
- Build: Wrong checksum on file (other file, included in SameKernel)
- files corrupted on disk, or mistake in our script/sources listing. Contact us
- Build: Wrong dpkg version
- do as the error message says.
- Can not run as root
- do as the error message says.
- Build: Wrong username
- do as the error message says.
- Build: Wrong directory
- do as the error message says.
bug1 [low] [fixed?] gzip problem while building from source
#bug1 Error unknown option '-' to gzip:: error
Thanks to benbrown we managed to locate problem. Output of strace:
execve("/bin/gzip", ["gzip", "-n", "-c-1"], [/* 15 vars */]) = 0
This bug is caused by passing -c%d instead -c9 to gzip compress in dpkg, since we fixed it only for xz compression. Problem appears on machines without installed zlib1g-dev when compiling dpkg.
Fix: We are patched dpkg, forcing the same level of compression on all compressors.
Workaround: Install zlib1g-dev and rebuild dpkg.
bug2 [very-low] [fixed] Headers checksum not matching since fixdep has other gcc name
#bug2 (fixed before 2014-02-29) not matching checksum of the deb with headers, just the gcc name differs
Solution: you must have identical version of tools that have their version embed in binaries during compilation - especially gcc and libc6. Now script will partially try to detect/warn you. Simply run system update before building. If you try to build older kernel it might be harder, you would need correct versions of libs - or instead compare by hand as explained in next paragraphs.
Impact: just makes it harder to produce-verification (but despite that you can just trust that other people your trust said source==kernel)
Info: the headers deb is sometimes different, between systems; It turn out that some of binary programs there like fixdep have embbed other exact name/version of gcc compiler.
News: seems to be result of differences in libc library package, will add check for it in run.sh
Workaround: update fully your system before building (and in retrospection - use identical version that developers used). Workaround: unpack your deb and the orginal deb and inspect diff, only the binaries should differ, and with command you can check the problem:
<vyrly@oftc> Can check GCC version easly using command: readelf -p .comment fixdep <tefnoot@oftc> String dump of section '.comment': <tefnoot@oftc> [ 0] GCC: (Debian 4.7.2-5) 4.7.2 <tefnoot@oftc> [ 1c] GCC: (Debian 4.4.7-2) 4.4.7
also you can hexdump the both binaries and diff them, it should be like:
<rfree> -000022f0 20 28 44 65 62 69 61 6e 20 34 2e 34 2e 37 2d 33 | (Debian 4.4.7-3| <rfree> +000022f0 20 28 44 65 62 69 61 6e 20 34 2e 34 2e 37 2d 32 | (Debian 4.4.7-2| but also this line, it's probably same info about version but in binary (int?) -00000240 14 00 00 00 03 00 00 00 47 4e 55 00 9c 99 41 b2 |........GNU...A.| -00000250 36 53 75 44 d1 54 57 84 cd 01 f0 2b 84 17 a2 27 |6SuD.TW....+...'| +00000240 14 00 00 00 03 00 00 00 47 4e 55 00 ad 80 0e 61 |........GNU....a| +00000250 2c ee 29 52 92 10 8f f0 09 90 e3 e3 42 2d 4b 36 |,.)R........B-K6|
Though above is not 100% guaranteed the difference is not a trojan. Perhaps further assembler analysis of that section can prove it (readelf)
Research: this happens even if dpkg says gcc, gcc-4.7 are same. Some other packages are involved.