Differences between revisions 42 and 43
Revision 42 as of 2017-10-12 00:09:21
Size: 17420
Editor: ?HelenKoike
Revision 43 as of 2017-10-12 00:50:32
Size: 17420
Editor: ?HelenKoike
Deletions are marked like this. Additions are marked like this.
Line 109: Line 109:
 1. Cons about the byhand aproach:  1. Cons about the byhand approach:
Line 117: Line 117:
  * 2.1. The exact same -signed.deb package is not reproducible, but the signature can be easily striped away and we can compare if the generated package is the same. While the byhand approach we would have two source packages, and the source for the -signed package would be shipped with the dettached signatures.   * 2.1. The exact same -signed.deb package is not reproducible, but the signature can be easily striped away and we can compare if the generated package is the same. While the byhand approach we would have two source packages, and the source for the -signed package would be shipped with the detached signatures.

/!\ Work is now being co-ordinated via #820036 so this page may be out of date.

Implementation choices

Open questions

  • Can the signature on shim be detached from the binary and combined with it during the build? This would allow reproducible builds.
    • - Yes, sbattach in sbsigntool supports this
  • How to verify that Microsoft hasn't tampered with the shim binary before signing?
    • - Applying 'sbattach --remove' to signed shim should yield the unsigned version we sent
  • How many people and who should hold the private keys for the certificate(s)? DSA? Secure Boot team? How should we backup the private keys?
  • Do we want to use key sharding or a HSM?

  • What is the verification process for the EV cert, which identity is being verified?
  • Sign shim by just MS key or one package for the Debian key, one for the MS key?
  • Do we want a canary like the canary for Fedora/RHEL/shim upstream.

Proposed signing architecture

First option: by-hand script in dak

signing architecture with dak

The main idea is to have a signing-box with access to the signing usb keys.

Basically, the steps can be described as:

  • The maintainer of the package (grub/linux) uploads it to Dak
  • A by-hand script inside dak is called, which will send the binaries to be signed by the signing-box
  • The signing-box (which runs as user 'codesign' in the diagram above) uses the ?YubiKey to sign the binaries inside the tarball

  • The signing-box sends a tarball back to the by-hand script in dak with all the detached signatures
  • The by-hand script sends the detached signatures to a machine accessible by the DDs
  • Another (or the same) DD retrieves the detached signatures and makes a signed version of the package
  • The maintainer uploads the signed package to dak

Ideally dak would upload the -signed version of the package automatically, but this can be done later. Right now, we currently have the ?YubiKeys plugged into the fasolo machine, so the signing-box would also run in fasolo for now. For security and better key management, the signing-box would run as another user that we called codesign in the diagram above.

One of the suggestions is to use a machine called coccia to host the detached signatures for the DDs to retrieve them.

what we have

signing-box code:

dak patch:

what we need

  • To patch dak's code, adding the by-hand script
  • Install the signing-box scripts on fasolo

Second option: use buildd + debhelper instead of dak

The idea is that instead of changing dak, add the signing logic to a helper script and use that in build process on the buildd.

Signing architecture with buildd

Basically, the steps can be described as:

  1. The maintainer uploads the source package to dak (without the binary packages)
  2. dak/wanna-build sends the package to the buildd as usual; the buildd works as normal and start the build process of the package in an isolated environment (chroot)
  3. During the package build, it uses the dh_signcli tool (which doesn't exist yet) to request signatures on various files.
  4. dh_signcli communicates with dh_signd (which also doesn't exist yet) outside of the chroot environment to request signatures. This communication can be done via some method (maybe d-bus, maybe unix socket?), so that dh_signcli can work even if the chroot environment doesn't have network enabled. dh_signd checks if the package is allowed to request signatures.
  5. dh_signd sends a tarball of binaries to the signing-box through ssh
  6. The signing-box uses the ?YubiKey to sign the binaries

  7. The signing-box sends a tarball back to dh_signd with the detached signatures
  8. dh_signd send the detached signatures back to the dh_signcli
  9. The package build continues, using the output of dh_signcli to assemble packages that include signatures.
  10. buildd uploads the build output into the archive as normal.

Handling errors

  • What to do if dh_signd is not running or if signing-box fail or can't be reached? dh_signcli will fail and we fail the whole build?


1. When the package enters in the NEW queue, the binary package sent by the maintainer is not discarded, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798000 Possible solutions:

  • Upload source+all to the NEW queue, but it won't work with other packages as grub which doesn't provide a build for "all", we could add a grub-doc (sounds like a hack)
  • Modify the NEW queue policy and accept source only uploads
  • Modify the NEW queue policy for only some specific packages/maintainers
  • Upload the binary packages without the -signed version of it, but make buildd to rebuild it again and discard the binaries uploaded by the maintainer

2. Builds are going to stall waiting on the signing service to do stuff. That could take some time for Linux, for example, with many many signs needed for individual modules.

what we have

signing-box code:

what we need

  • Install/configure the signing-box scripts on fasolo
  • Develop the dh_sign scripts
  • Install/configure x86_64 buildd machine with the scripts
  • Solve the NEW queue issue described above
  • Adapt grub/kernel to this new process

Wrap-up of the discussions so far

  1. Cons about the byhand approach:
    • 1.1. byhand files for security-master break the security-master -> ftp-master sync.

    • 1.2. The autobyhand scripts don't work for uploads that go to NEW
    • 1.3. byhand files are not that nice; I would rather not add more of them
    • 1.4. byhand files aren't publish in the public archive, so harder to see what was actually signed. Nothing enforces the binaries in the *.deb and the *.tar.gz are related after all. Or that a *.deb is present.
    • 1.5. Not clear how to publish signed binaries for the manual step in preparing uploads for the security archive.
  2. About the Buildd approach:
    • 2.1. The exact same -signed.deb package is not reproducible, but the signature can be easily striped away and we can compare if the generated package is the same. While the byhand approach we would have two source packages, and the source for the -signed package would be shipped with the detached signatures.
    • 2.2. Signing things automatically is dangerous as an attacker can attack the infrastructure and get things signed (even if we have a whitelist to limit our signing process to some packages as grub and the kernel), so we will need to revoke signatures or the key. We could develop a mechanism to easily revoke things.
    • 2.3. Ftp masters don't want verify each package by to say it is safe to sign
    • 2.4. As buildd will access an external signing service, it makes it hard to isolate the builds and restrict the potentially evil things only to this build

Secure Boot General Information

The idea is to use the latest shim signed by Microsoft that will enable us to bootstrap into a later boot loader that can be signed by Debian. grub itself will need to be signed by a FTP master, so we will need to sort out key signing. Same for the kernel.

Roughly matches existing deployment in Ubuntu, although hopefully with some further improvements on top.


The shim is an EFI executable that is in a format that is acceptable for signing by microsoft. UEFI has a database of keys that can be used for signing, the shim is to be used by a 3rd party.

It supports its own user-modifiable key database (?MokManager) so an end user can install their own key (requires the user to be physically present). Grub will call into the shim to verify the kernel, so you get a fully signed root of trust. Could theoretically verify initramfs and root filesystem. You can also disable sig validation so shim will launch anything you give it (again requiring physical presence)... from then on it will boot any copy of grub/kernel, without disabling secure boot entirely... allows for kernel and grub development without having to jump through a lot of hoops.

The aim is the least worst that still respects user's freedoms. Local key management was implemented by SUSE, the rest by redhat.

Potential improvements: fall-back bootloader so if a system loses all boot entries, the shim will re-enroll and register them and boot normally.


Grub has many patches from redhat/mjg causing it to operate in a secure boot way. When grub core is signed it refuses to load any modules that are unsigned. Secure boot core images are larger as a result. That code is in debian, some of it is configured off by default, but that is trivial to change. There is code to build custom uploads. For more information, please see: http://thread.gmane.org/gmane.linux.debian.devel.boot/143954

Involved and related software/packages: shim grub efilinux linux openssl mokutil pesign sbsigntool secureboot-db efitools vboot-kernel-utils

MSFT key requirements

MSFT has a short list of requirements, it boils down to these critical points:

  • EV cert: this requires identify verification
  • code must not be subject to GPLv3, "or any license that purports to give someone the right to demand authorization keys to be able to install modified forms of the code on a device. Code that is subject to such a license that has already been signed might have that signature revoked. For example, GRUB 2 is licensed under GPLv3 and won’t be signed."
  • because of the above, we use a shim (which hands off executation to another bootloader)...this is a model where the vendor cert is embedded in a new place in the executable. Even if you receive a binary from debian, you can still verify the executables are identical. This will allow MSFT to verify the binary is the same, with only the certificate changed.
  • code that hasn't been SecureBoot "enlightened" won't be signed

  • code signing keys must be backed up, stored, and recovered only by personnel in trusted roles, using at least dual-factor authorization in a physically secured environment.
  • The private key must be protected with a hardware cryptography module. This includes but is not limited to HSMs, smart cards, smart card–like USB tokens, and TPMs. The operating environment must achieve a level of security at least equal to FIPS 140-2 Level 2.
  • submitter must design and implement a strong revocation mechanism for everything the shim loads, directly and subsequently.
  • some shims are known to present weaknesses into the SecureBoot system. For a faster signing turnaround, we recommend that you use source code of 0.8 or higher from shim - GitHub branch.

TODO: Ubuntu does a key sharding process, required some kind of approval from Microsoft, find out what this procedure is

Pre-submission testing

Task list

Package the software

sbsigntool is in the archive.

shim: 820052

secureboot-db does not have an ITP.

Prove the setup

Prove the setup works using qemu, the non-free OVMF firmware and a test dak install, see the Ubuntu wiki for details.

Generate a key

We need a RSA 2048 bit key for debian to get into ?MokManager. Debian must generate the key and the self-signed certificate of the correct form, which is embedded in the shim package that is then submitted to Microsoft. The signing request requires obtaining an EV code-signing cert, and then this has to be uploaded via Windows to Microsoft.

Here is the procedure for how it is done in Ubuntu.

TODO 1. tollef: DSA generates key and obtains EV cert for submission to MS 2. tollef: DSA generates key and self-signed cert for shim

Prepare shim

Vorlon said he would include the public key and cert from step 2 in the shim package, upload this to debian and gets through binary NEW. This will embed our own set of public keys (corresponding to those used by dak) and can load any other EFI executable signed by one of them. Later, there will be a shim-signed package containing the same executable with a Microsoft signature. (This costs money and takes several days, but shim should require only very infrequent changes.)

Then Tollef (or whoever has control of the EV cert) extracts the shim binary, puts it into a cab, signs that with the EV cert

Need ?MokManager support in the shim (in order to allow user to enable/disable kernel signature verification). Ubuntu has packaging that has mokmanager, though it has not yet passed through Microsoft review and is therefore not currently live in any Ubuntu release. TODO: vorlon will maintain this in debian, but is going to wait on this until we have a key

Sacrifice goats

Some poor soul sacrifices their remaining dignity and runs windows to upload this

dak changes

Ben Hutchings and Julien Cristau working on this in 821051.

For reference: Launchpad code, tests


Colin will update the GRUB package to build a to-be-signed monolithic EFI executable separate from the package. Then he will add a grub-signed package (820050) that includes the Debian-signed executable from the archive. This executable would be suitable for use on both removable media and the installed system.

Note: this cannot be done until the dak changes are in place and configured with an appropriate signing key. However, once they are, it's a trivial switch in grub2/debian/rules to enable it, since the necessary code is already there, just conditionalised for Ubuntu.


The kernel team will need to upload kernel images for signing and add linux-image-signed packages (820006) with the Debian-signed kernel images.

Locking down things in the kernel: you need to be able to distinguish between root and the kernel in a secure boot environment. In the past root could modify kernel... secure boot prevents that, so there is an incentive to lock down the kernel so root cannot do that in various ways. There would be an external patch set that would do this that debian would have to carry until upstream Linux will do their reimplementation. We are going to turn on this patchset by default when secureboot is enabled (820008).