pyopenssl: Setting socket timeouts breaks handshake

@moospit @webratz Here’s what works for me:

def print_chain(context, hostname):
    print('Connecting to {0}'.format(hostname))
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock = SSL.Connection(context=context, socket=sock)
    sock.settimeout(5)
    sock.connect((hostname, 443))
    sock.setblocking(1)
    sock.do_handshake()
    for cert in sock.get_peer_cert_chain():
        print('   s:{0}'.format(cert.get_subject()))
        print('   i:{0}'.format(cert.get_issuer()))
    sock.shutdown()
    sock.close()

You can put a timeout on the connect and it will work as desired, you just have to put the socket back into blocking mode before calling into OpenSSL. Of course this just gets you a timeout on the TCP connection; if things stall during the SSL handshake you’re still going to be left hanging but it’s better than nothing.

Thanks.

From this, I see that the problem is the mismatch between expectations for do_handshake vs the native Python socket APIs.

Python’s socket.socket.connect method respects the Python-level socket timeout. This is easy for it because they’re both native Python socket features (timeouts and connection setup).

OpenSSL.SSL.Connection.do_handshake is not a native Python socket operation though. It is a call into OpenSSL’s SSL_do_handshake API which operates on the actual (platform-level) socket directly. Python’s timeout support puts that socket into non-blocking mode. OpenSSL’s SSL_do_handshake encounters this and does the standard OpenSSL-level thing - translate the “EWOULDBLOCK” read error into an OpenSSL WantReadError (that’s pyOpenSSL’s spelling of the error but that’s easier to talk about here). pyOpenSSL raises this exception up to the caller of do_handshake.

The only idea I have for fixing this is to teach pyOpenSSL about Python’s socket timeout feature: at every point in the API where there is an OpenSSL operation that operates directly on a platform-level socket, introduce the same kind of wait-and-retry logic that Python’s own socket library has (which implements the timeout feature).

This will introduce a lot of complexity into pyOpenSSL. That’s not a sufficient reason to say it would be a bad thing to add this functionality to pyOpenSSL but it does hint that it might not be a good idea. The contrary argument might be that any application that wants to use timeout-enabled sockets (or more generally, non-blocking sockets) with pyOpenSSL will need to implement this logic (and indeed they have, that’s why pyOpenSSL exposes these error conditions as exceptions in the first place).

The general shape of this solution is probably to make all methods that invoke an OpenSSL API that interacts with a socket (as opposed to methods that interact with the Python-level socket, eg OpenSSL.SSL.Connection.connect) have some code kind of like this:

    timeout = self._socket.gettimeout()
    if timeout is not None:
        start = time()
    while True:
        try:
            return <some OpenSSL API>
        except (WantReadError, WantWriteError):
            if timeout is None or start + timeout > time():
                raise
            select([self._socket], [self._socket], [], timeout - (time() - start))
            if <select timed out>:
                raise <something - the original exception?  a specific timeout exception?>

This could probably be encapsulated into a helper function. It would probably also be beneficial to inspect the implementation of timeouts in Python’s sockets to see if there are any other behaviors worth emulating or non-obvious implementation concerns worth dealing with here (to provide the least surprising behavior).

I found this code here which worked great for me: https://gemfury.com/hemamaps/python:urllib3/-/content/contrib/pyopenssl.py

cnx = OpenSSL.SSL.Connection(ctx, sock)
if isinstance(server_hostname, six.text_type):  # Platform-specific: Python 3
    server_hostname = server_hostname.encode('utf-8')
cnx.set_tlsext_host_name(server_hostname)
cnx.set_connect_state()
while True:
    try:
        cnx.do_handshake()
    except OpenSSL.SSL.WantReadError:
        rd, _, _ = select.select([sock], [], [], sock.gettimeout())
        if not rd:
            raise timeout('select timed out')
        continue
    except OpenSSL.SSL.Error as e:
        raise ssl.SSLError('bad handshake: %r' % e)
    break

yeah in my case i sometimes connect to weird servers that happily open the tcp connection, but then do not properly respond on the SSL layer. My workaround was using the timeout-decorator from pypi