With the introduction of openssl version 1.1.1 in Debian several packages have become buggy. This is due to changes in openssl that may come up during run time.

Debian

In Debian the defaults are set to more secure values by default. This is done in the /etc/ssl/openssl.cnf config file. At the end of the file there is:

[system_default_sect]
MinProtocol = TLSv1.2
CipherString = DEFAULT@SECLEVEL=2

This can results in errors such as:

dh key too small
ee key too small
ca md too weak

This is caused by the SECLEVEL 2 setting the security level to 112 bit. This means that RSA and DHE keys need to be at least 2048 bit long. SHA-1 is no longer supported for signatures in certificates and you need at least SHA-256. Note that CAs have stopped issuing certificates that didn't meet those requirements in January 2015, and since January 2017 all valid CA certificates should meet those requirements. However there are certificates generated by private CAs or that are in a test suite that do not meet those requirements.

SECLEVEL 1 was the default in previous versions and is at the 80 bit security level, requiring a 1024 bit RSA key.

You can also get errors such as:

version too low
ssl3_read_bytes:sslv3 alert handshake failure:../ssl/record/rec_layer_s3.c:1528:SSL alert number 40

We now require as minimum the TLS 1.2 version instead TLS 1.0. If the other side does not support TLS 1.2 or higher you will get some connection errors.

TLS 1.3

TLS 1.3 has lots of changes that might cause issues. See https://wiki.openssl.org/index.php/TLS1.3 for more information.

One of those changes may cause time out:

SSL_MODE_AUTO_RETRY is enabled by default. Applications that use blocking
I/O in combination with something like select() or poll() will hang. This
can be turned off again using SSL_CTX_clear_mode().
Many applications do not properly handle non-application data records, and
TLS 1.3 sends more of such records. Setting SSL_MODE_AUTO_RETRY works
around the problems in those applications, but can also break some.
It's recommended to read the manpages about SSL_read(), SSL_write(),
SSL_get_error(), SSL_shutdown(), SSL_CTX_set_mode() and
SSL_CTX_set_read_ahead() again.

Please check bugs blocking https://bugs.debian.org/907015 for related issues.