Translation(s): English - Español

In Debian Security Advisory 1571, also known as CVE-2008-0166 (New openssl packages fix predictable random number generator), the Debian Security Team disclosed a vulnerability in the openssl package that makes many cryptographic keys that are used for authentication (e.g. through SSH) or signing (e.g. web server certificates) potentially vulnerable.

End User Summary

The scope of the problem includes:

The following cryptographic tools are unaffected:

Identifying Weak Keys

Characteristics of potentially vulnerable keys:

In addition, any DSA key must be considered compromised if it has been used on a machine with a 'bad' OpenSSL. Simply using a 'strong' DSA key (i.e., generated with a 'good' OpenSSL) to make a connection from such a machine may have compromised it. This is due to an 'attack' on DSA that allows the secret key to be found if the nonce used in the signature is known or reused.

Blacklists of vulnerable keys available in unstable:

There is a web-based check available at https://secure.comodo.net/utilities/decodeCSR.html which will identify a CSR with a weak key. This page uses the data from openssl-blacklist.

Many lists of 'weak' keys have been generated by the metasploit project: http://metasploit.com/users/hdm/tools/debian-openssl/

Applications/protocols known to use these keys:

To fix this, first aptitude update && aptitude upgrade to install the new version of the openssl and libssl0.9.8 packages (the vulnerability is fixed in version 0.9.8c-4etch3 for etch and version 0.9.8g-9 for lenny/sid). You probably want to also pick up the new openssh packages that include the blacklist of known weak keys, but you will need to aptitude dist-upgrade for that in order to install the new openssh-blacklist package.

If you choose not to use the above aptitude command, note that all of the following packages must be upgraded (they all come from the same source package):

Then, regenerate and distribute any potentially vulnerable keys. Instructions for how to regenerate the keys for these applications are below. You can also test to see if keys are vulnerable using the dowkd.pl utility as described below.

How weak?

The broken version of OpenSSL was being seeded only by process ID. Due to differences between endianness and sizeof(long), the output was architecture-specific: little-endian 32bit (e.g. i386), little-endian 64bit (e.g. amd64, ia64), big-endian 32bit (e.g. powerpc, sparc). PID 0 is the kernel and PID_MAX (32768) is not reached when wrapping, so there were 32767 possible random number streams per architecture. This is (2^15-1)*3 or 98301.

Non-broken OpenSSL seeds from PID and /dev/urandom.

Application Details

Asterisk

Asterisk uses RSA keys as an optional authentication method for IAX2 and for DUNDI. Keys are SSL public/private key pairs. The Asterisk package does not generate keys automatically and most users don't seem to use them. You should probably know if you use such a key.

BIND9

To regenerate your rndc key, do the following. (This is what the postinst script does as well) 

rndc-confgen -r /dev/urandom -a

> I don't know if this is neccessary or not though... ronalde: According to the changelog for bind9 in Debian rndc-confgen in Debian uses /dev/urandom since March 2002 (before then /dev/random was used); I guess rndc-keys aren't affected.

Keys for DNSSEC or DynamicDNS are probably weak too and should also be recreated through the use of dnssec-keygen(1). Exactly what parameters to use depends on how you are using the keys. See Secure DDNS Howto for some examples for DDNS.

boxbackup

See Official Key-Rollover page.

Cfengine

For each cfengine host, remove the old keys and generate new keys: 

rm /var/lib/cfengine2/ppkeys/localhost.priv
rm /var/lib/cfengine2/ppkeys/localhost.pub
/usr/sbin/cfkey
Then restart cfservd:
/etc/init.d/cfengine2 restart

Once the keys are regenerated, exchange keys between hosts as necessary to reestablish 2-way trusts.

courier imap/pop3

Follow the "Generic PEM Generation" instructions and add a openssl gendh >> mysite.pem.

Or 

rm /etc/courier/imapd.pem
dpkg-reconfigure courier-imap-ssl

and let dpkg generate back an imapd.pem file.

uw-imapd

Let dpkg generate back an imapd.pem file. 

cd /etc/ssl/certs
rm `openssl x509 -noout -hash < imapd.pem`.0
rm imapd.pem
dpkg-reconfigure uw-imapd

cryptsetup

See Official Key-Rollover page.

csync2

As described in /usr/share/doc/csync2/README.Debian 

/etc/csync2_ssl*
openssl genrsa -out /etc/csync2_ssl_key.pem 1024
openssl req -new -key /etc/csync2_ssl_key.pem -out /etc/csync2_ssl_cert.csr
openssl x509 -req -days 600 -in /etc/csync2_ssl_cert.csr \
        -signkey /etc/csync2_ssl_key.pem -out /etc/csync2_ssl_cert.pem

Then the keys must be distributed out to each host prior to running csync2 again.

cyrus imapd

To find out which certificates/keys are in use, see the directives whose names contain "key_file" or "cert_file" in /etc/imapd.conf.

Generate new private keys and certificates as shown in "Generic PEM Generation" above, then restart the service: 

invoke-rc.d cyrus21 restart

if you are using version 2.1 (if you are using another version, please examine /etc/init.d for the correct service name)

dovecot

Generate a new PEM as shown in "Generic PEM Generation" above (making sure that it is placed in the correct place according to your dovecot configuration, then restart dovecot:  invoke-rc.d dovecot restart .

Or: 

rm /etc/ssl/certs/dovecot.pem
rm /etc/ssl/private/dovecot.pem
dpkg-reconfigure dovecot-common

dropbear

See Official Key-Rollover page.

exim4

If TLS is in use, generate a new PEM using /usr/share/doc/exim4-base/examples/exim-gencert --force to get a self-signed certificate. Per default it has a three years expiration duration.

ftpd-ssl

See Official Key-Rollover page.

Generic PEM Generation

This is just a reminder for those who generate PEM encoded certificates. Your site probably has other policies in place about how to manage keys which you should follow. Additionally, you may need to get the certificates signed again by a 3rd party Certificate Authority rather than by using a self-signed certificate as shown below: 

cd /etc/ssl/private
openssl genrsa 1024 > mysite.pem
cd /etc/ssl/certs
openssl req -new -key ../private/mysite.pem -x509 -days 9999 -out mysite.pem

(The last command (openssl req.....) is all on one line, ending with .pem)

gitosis

sudo -H -u gitosis gitosis-init < new_SSH_KEY.pub

check all keys under /var/cache/gitosis/repositories/gitosis-admin.git/gitosis-export/keydir/*.pub

(older versions of gitosis using /var/cache/git)

OpenSSH (Server)

See also Official Key-Rollover page.

Updated packages for openssh that have a blacklist of known weak keys are now available; see DSA 1576 for more information. Installing these packages on hosts with weak keys will cause the ssh server to regenerate its keys. Weak user keys being used for a connection will also be rejected where possible.

/!\ Note that you will have to use aptitude dist-upgrade (or apt-get dist-upgrade) to install these packages rather than just upgrade because this update will cause the new package openssh-blacklist to be installed.

You can also update your openssh-server keys manually. 

rm /etc/ssh/ssh_host_*
dpkg-reconfigure openssh-server

/!\ Note that in either case, your users will see a "IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!" warning when they next log on to your ssh server because the key has changed. They will need to edit $HOME/.ssh/known_hosts to remove the offending line before continuing; checking that the key fingerprint is correct, of course (the fingerprint of your new key can be found with ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key). You can remove the key from known_hosts by running "ssh-keygen -R hostname"

Also, note that your existing ssh connections shouldn't be interrupted.

OpenSSH (Client)

See also Official Key-Rollover page.

You will need to have a list of the openssh keys that you currently have and where they have been copied to. For each key that is vulnerable: 

cd ~/.ssh
ssh-keygen -t rsa -f filename
ssh-copy-id -i filename hostname

Replacing rsa by dsa if you prefer dsa keys and replacing filename and hostname with appropriate values.

Also remember to remove compromised keys from your .ssh/authorized_keys file!

OpenSWAN

Openswan's raw RSA key generation is not vulnerable, as it does not use the openssl library. This means all connections using authby=rsasigkey are not vulnerable.

X.509 based keys generated by administrators for use with IPsec could be vulnerable if created on a Debian system using the openssl command (as per documentation)

See Official Key-Rollover page.

StrongSWAN

See Official Key-Rollover page.

OpenVPN

See Official Key-Rollover page.

If you're using x509 certificates, you need to create a whole new CA if you generated the CA key with a broken OpenSSL. Even if you CA key isn't compromised, some of the keys of the OpenVPN clients might be. In that case you need to revoke all the certificates for those keys and add the CRL to your OpenVPN configuration. See the OpenVPN HOWTO for more information about revocation.

postfix

Generate a new PEM as shown in "Generic PEM Generation" above (making sure that it is placed in the correct place according to your postfix configuration, then restart postfix:  invoke-rc.d postfix restart .

puppet

See Official Key-Rollover page.

ssl-cert

See Official Key-Rollover page.

telnetd-ssl

See Official Key-Rollover page.

tinc

See Official Key-Rollover page.

Tor Onion Router / Hidden Service Keys

See Official Key-Rollover page.

encfs

Analysis of the impact is still ongoing. Details known so far: encfs uses the RNG from libssl to create an internal encryption key with some post-processing applied. Checking of that key against a pre-calculated blacklist like with dowkd.pl might be possible when it's ready.

In the meantime, encrypted filesystems that might be created with a broken version of libssl shall be considered vulnerable against offline attacks. Any copy of encrypted data located in unthrusted environment might become readable, sooner or later. If the date of creation is unknown, checking of the originating version in output of the encfsctl command against the changelog of encfs package may provide some hints but is not absolutely reliable either because there was no strong dependency on certain openssl version.

To regain security, try following:

xrdp

See Official Key-Rollover page.

Kerberos (MIT and Heimdal)

If you were using MIT Kerberos, you would be fine so far as I can tell since MIT Kerberos has its own crypto layer and its own randomness functions. Heimdal uses OpenSSL as its crypto layer. Given that, it may well be possible to brute-force the session key of any captured GSSAPI-encrypted traffic and decrypt it retroactively.

If you're using Heimdal, you should also change all long-term random keys (such as any key in generated keytab files) that were generated using the vulnerable version of OpenSSL.

This can be done using cpw -r <principal> within kadmin. Take into account that this principals have been randomed-key assigned and should be regenerated as well. 

kadmin/admin
kadmin/hprop
kadmin/changepw
changepw/kerberos
krbtgt/YOUR.REALM
host/SOMEHOST@YOUR.REALM
host/SOMEOTHERHOST@YOUR.REALM

Keys based on user passwords should be fine.

pwsafe

First issue: the random seed of the DBs is now the same. More exactly, the random, the salt and the iv. The random is fixed at creation of the DB. Apparently the seed & iv are renewed at every edition of the DB and depend only on a fresh random. Data encryption depend only on hash and seed/iv so even with different random, if the fresh random for seed is the same, the encrypted data is the same. A practical attack against a pwsafe DB is e.g. to construct a generic rainbow with the possible random seeds and your favorite character class. Once done it could attack any pwsafe DB generated with a broken OpenSSL or even edited with a broken OpenSSL (to be confirmed).

Second issue: all nice passwords proposed by pwsafe when creating a new entry are now broken! So are all the accounts you created around based on those passwords. No matter the DB/user/group/... you are using. Of course no matter how decently you seeded the RANDFILE ~/.rnd (cf man). To rekey your pwsafe, create a new one, import the old one then delete the old one: 

$ pwsafe -f newdb --create
Enter passphrase for newdb:
Reenter passphrase for newdb:
$ pwsafe -f newdb --mergedb=/home/myself/.pwsafe.dat
Enter passphrase for newdb:
Enter passphrase for /home/myself/.pwsafe.dat:
Merged 247 entries; skipped 0; 0 duplicates.
$ wipe /home/myself/.pwsafe.dat
$ mv newdb /home/myself/.pwsafe.dat

And if you generated your account passwords with pwsafe, you should renew all those passwords...

(pwsafe information from http://wiki.yobi.be/wiki/Debian_OpenSSL#pwsafe )

slurm-llnl

Remove the vulnerable SLURM key files: 

rm -f /etc/slurm-llnl/slurm.key /etc/slurm.cert

Generate new key pair with the commands: 

openssl genrsa -out /etc/slurm-llnl/slurm.key 1024
openssl rsa -in /etc/slurm-llnl/slurm.key -pubout -out \
        /etc/slurm-llnl/slurm.cert

Copy the key files on the controller and the backup controller and the cert file on all the nodes of your cluster then restart SLURM with the command: 

invoke-rc.d slurm-llnl restart

SSL Certificate Reissuance

If you paid good money to have a vulnerable key signed by a Certificate Authority (CA), chances are your CA can re-issue a certificate for free, provided all information in the CSR is identical to the original CSR. Create a new key with a non-vulnerable OpenSSL installation, re-create the CSR with the same information as your original (vulnerable) key's CSR, and submit it to your CA according to their reissuance policy

Testing keys using ssh-vulnkey

In some openssh-client packages (starting from 4.7p1-9 up to 6.5p1-1), ssh-vulnkey is included along with a blacklist of known weak keys. This tool appears to be similar to dowkd.pl (below) although it is restricted to checking ssh keys (not that dowkd.pl can do much more than that anyway).

Run ssh-vulnkey as a user to check your host and your own keys.

Run ssh-vulnkey -a as root to check all users keys (as well as the host key).

Sample output: 

ssh-vulnkey -a
Not blacklisted: 2048 fa:2e:1d:a6:84:64:a1:80:c4:31:68:5a:b0:1a:cb:fe /etc/ssh/ssh_host_rsa_key.pub
Not blacklisted: 1024 f4:34:04:85:58:a0:6b:0a:a1:b9:2d:3b:e6:19:5a:76 /etc/ssh/ssh_host_dsa_key.pub
COMPROMISED: 2048 5c:10:8a:c0:55:8c:1f:d9:4b:05:f0:35:0a:0d:2f:5c /home/someuser/.ssh/authorized_keys
Not blacklisted: 2048 a7:b4:3e:41:18:cb:f7:68:5e:4f:ae:30:14:d2:17:fd /home/someuser/.ssh/authorized_keys

/!\ Note: about output is one line per key not linewrapped as shown here in the wiki

/!\ Note: 4096bit and 1024bit RSA fingerprints are only included since openssh-blacklist-0.3.

Testing keys using dowkd.pl

The security team has prepared a utility for you to test your keys to see if they have a fingerprint that matches a list of known-weak finger prints. Download dowkd.pl.gz (OpenPGP signature) /!\ Attention: This version is not the same than the original tool released by Florian Weimer. So the patches below will not apply!

Changes to the original version:

As I (Klaus Ethgen) can see the tool do what it should (Version sha1:027adef3b86991661394b6ab6159d75dde3c3087 0.9). But please check yourself. Also if someone make update of the script please tell it on this page to not confuse users. 

cd /tmp
wget http://security.debian.org/project/extra/dowkd/dowkd.pl.gz
wget http://security.debian.org/project/extra/dowkd/dowkd.pl.gz.asc
gpg --keyserver subkeys.pgp.net --recv-keys 02D524BE
gpg --verify dowkd.pl.gz.asc
gunzip dowkd.pl.gz

/!\ Note that dowkd.pl may produce false negatives. As of 2008-05-19, the current version of dowkd.pl prints warnings if a false negative is likely, except if the key in question has been generated on a big-endian architecture. If unsure, regenerate your key material.

Use dowkd.pl in the following ways:

SSH Server Host Key

Key is OK: 

perl dowkd.pl host localhost
# localhost SSH-2.0-OpenSSH_4.3p2 Debian-9
# localhost SSH-2.0-OpenSSH_4.3p2 Debian-9

Key is weak: 

perl dowkd.pl host localhost
# localhost SSH-2.0-OpenSSH_4.3p2 Debian-9
# localhost SSH-2.0-OpenSSH_4.3p2 Debian-9
localhost: weak key
localhost: weak key

A remote check based on the keys generated by HD Moore (http://metasploit.com/users/hdm/tools/debian-openssl/ ) is available at http://itsecurity.net. debian_ssh_scan_v3 now includes fingerprints of all weak DSA 1024, RSA 2048 and RSA 4096 bit keys. 

root@box:~/debian_ssh_scan_v2# ./debian_ssh_scan_v2.py 10.128.15.110
65536 fingerprints loaded.
10.128.15.110:22 sshd fingerprint 9cf71acb1b0dff0dceef4f755f721e9d VULNERABLE (RSA 2048 bit key, pid 5252)

SSH User Key

To check one user (someuser): 

perl dowkd.pl user someuser
/home/someuser/.ssh/authorized_keys:1: warning: unparsable line
/home/someuser/.ssh/authorized_keys:3: warning: unparsable line
/home/someuser/.ssh/authorized_keys:4: weak key
/home/someuser/.ssh/authorized_keys:5: weak key

Notes:

To check all users: 

perl dowkd.pl user

perl dowkd.pl user > /tmp/weak 2>/tmp/weak.err
./gethk.sh /tmp/weak
==== FILE: /root/.ssh/known_hosts:6:
10.0.0.4 ssh-rsa AAAAB3NzaC1[....]

==== FILE: /home/joe/.ssh/known_hosts:13:
10.0.0.1 ssh-rsa AAAAB3NzaC1[....]

PEM keys (SSL certificates)

The SSL server (and client) certificates, aka PEM keys, may be used in various tools (Apache + mod SSL, etc.).

The Ubuntu team has created a package which will verify if PEM files are vulnerable. I used it against PEM keys created on a known vulnerable system and the utility highlighted them as vulnerable. The package is called "openssl-blacklist". The package is available here. It can be installed on etch using dpkg -i --force-all.

openssl-blacklist repackaged for Debian: openssl-blacklist_0.1-0~debian-1_all.deb . Built for etch/stable, but should install cleanly on any later debian versions. I'm not uploading this to Debian proper at present, because I haven't checked whether anyone else is already doing that. For the (signed) sources, see http://xillion.org/openssl-blacklist/ . -- PaulCannon

There is a way to convert PEM files to SSH-style public keys, which the tool can parse. I've not verified this method completely, it comes from information found at: http://webjob.sourceforge.net/Files/Recipes/openssl-convert-ssl-key-to-ssh-keypair.txt 

cp /etc/apache/ssl.key/your.keyfile.key ~/.ssh/id_ssl
ssh-keygen -y -f ~/.ssh/id_ssl  > ~/.ssh/id_ssl.pub
./dowkd.pl file ~/.ssh/id_ssl.pub

I have as yet not determined how accurate this method is, but it seems to work reliably. Someone more familiar with the workings of ssh-keygen can confirm the reliability of this method. Scripting this test should be easy enough.

someone who actually understands this tool and its output, please insert details here!

I created a patch to parse PEM files. fw told me to look at the Modulus of the RSA key, so I am doing that. Even though I tested more than 20 certificates generated on an affected machine, I did not find a single weak key. This could mean that

As I manually verified the results against, I do not think there is an error in my patch. It is worth noting that none of the certs showed any hits when I converted them into SSH format. Please test this and report back to me!

To test this for yourself, do this 

for i in $(seq 1 100)
do
    openssl genrsa 1024 > server$i.key 2>/dev/null
    echo -e "\n\n\n\n\n\n\n\n\n\n" | \
            openssl req -new -key server$i.key -days 365 -out server$i.req 2>/dev/null
    openssl x509 -req -days 365 -in server$i.req -signkey server$i.key -out server$i.crt 2>/dev/null
    chmod 600 server$i*
    ssh-keygen -y -f server$i.key > server$i.pub 2>/dev/null
    perl dowkd.pl file server$i.pub
    perl dowkd.pl pem server$i.crt
done

and please post your results.

The Patch for authorized_keys2 and known_hosts (dowkd.pl.ke.patch.gz) is not needed anymore. -- Klaus Ethgen

I have hacked a little shell script using the tables from openssl-blacklist from ubuntu which allows the user to check the ssl key of a webhost from remote by just calling "chksslkey hostname". You can find it here: chksslkey.tar.bz2 -- Michael Holzt

I threw together a little script that combines everyone elses and lets you test a cert, ssl rsa key, pem file, or remote https against the blacklist's. You can find it at http://www.gokickrocks.us/wp-content/uploads/2008/05/audit-ssl.tar.gz. Super easy to use, just take a peek at the README. Basic usage is just "chkssl testtype fileorurl". -- Florian Hines

Scope of the blacklists

The current official blacklists (in "openssh-blacklist") cover RSA-2048 and DSA-1024 keys as generated on 32-bit little-endian, 64-bit little-endian and 32-bit big-endian systems. That's sufficient to cover users who used the default key length, but not if some of your users decided they wanted a longer keylength. For non-default keys, see "openssh-blacklist-extra" which contains RSA-1024, RSA-4096, and (an empty) DSA-2048 for all three architectures.

Technical Summary

(If you want to add more technical details that an end-user doesn't need to know or isn't likely to understand, please add them here rather than making the above summary impossible for the average user to understand.)

Causes

It is important to understand that this problem was caused by trying to remove valgrind warnings related to the use of uninitialised memory within the openssl libraries. This was done to try to make it easier to debug C applications that use the openssl libraries which is a good thing to do.

A discussion of why this change was made can be found at #363516 and also on the openssl-dev list. Judging from the discussion there, the main culprit seems to be a misunderstanding about which is the right list to ask this question on, followed by misleading answers from the list.

A bit more detail

In an effort to clear up confusion about this bug, here's a bit more technical description. This was caused by an overzealous, well-intentioned elimination of code that was believed to have no impact on security. (Please do note from the above links that this was discussed with the !OpenSSL team and that no objections were raised at the time.)

So here's the problem: the Debian maintainer wanted very much to get rid of valgrind errors while using OpenSSL; certainly a noble cause, right? As you can see here, there are two identical lines,  MD_Update(&m, buf, j);  in SSL's md_rand.c file that were commented out way back in 2006: http://svn.debian.org/viewsvn/pkg-openssl/openssl/trunk/rand/md_rand.c?rev=141&view=diff&r1=141&r2=140&p1=openssl/trunk/rand/md_rand.c&p2=/openssl/trunk/rand/md_rand.c

The second of these is in  ssleay_rand_bytes  where  buf  is used as an output buffer. It had already been marked as a bad idea when -DPURIFY was in effect, because Purify (and valgrind, naturally) dislike this use of an output buffer as input. This use of  MD_Update  is dubious but shouldn't hurt as long as the mixing function of the PRNG is "sufficiently good". The removal of this call to MD_Update should not meaningfully alter the entropy available in the pool -- that is, OpenSSL does not depend upon uninitialized memory for its correct operation.

The first call, however, is in  ssleay_rand_add  where  buf  is used as an INPUT buffer, to add entropy to the pool. Failing to call  MD_Update  there means that the pool will never actually get the entropy intended for it.

Acknowledgements

This document was put together by the folks on #debian to help with user support. It would have been nice to have had the chance to prepare this (and the key rollover page) in the 5 days between the commit being made to pkg-openssl and the DSA being released. Particular credit to themill, dondelelcaro, stew, Maulklin, iobound for reading, writing and suggesting bits for this page. No thanks to the people who keep using the wiki's GUI editor and keep generating 15 to 20 KB diffs from when it randomly reformats the entire page when you only change one line.

See Also

http://www.debian.org/security/key-rollover/

CategorySystemSecurity | CategoryPermalink