openssl: malformed HTTPS chunked output in 1.1.1c, but working in 1.0.2s in Linux
I am getting malformed output from the openssl when using 1.1.1c (probably with other 1.1.x too), but working fine with 1.0.2s (probably ok as well with 1.0.y). The use case is for HTTPS chunked, I am getting mal formed chunks, and like neverending stream of data (it looks like same data over and over).
Very important note: there is some randomness in the failure, sometimes it works OK even with 1.1.1c, but most times not. So if it worked for you in 1.1.1c, you get lucky, if you retry again after minutes / hours you may get a failure. I am not sure if this randomness is caused by server or client … But it makes things harder to test and debug.
I build two docker images in the same host, one with 1.0.2s and one with 1.1.1c (latest stable versions to date).
openssl s_client -connect www.telecreditobcp.com:443 -servername www.telecreditobcp.com
and enter the following:
GET /tlcnp/index.do HTTP/1.1
Host: www.telecreditobcp.com
Another way to reproduce it is with the following python code:
import requests
requests.get('https://www.telecreditobcp.com/tlcnp/index.do')
You will get the following exception when it fails:
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(17 bytes read)', IncompleteRead(17 bytes read))
Another way to reproduce it is:
curl -X GET https://www.telecreditobcp.com/tlcnp/index.do
Some additional insights:
-
it seems to fail only for Linux. Someone told me it works on Windows, but not sure what version was tested, maybe it just happen to be 1.0.x version.
-
I’ve been told sometimes using some kind of VPN makes things work.
Let me know if there is anything else I can provide to help with the debugging of this.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 5
- Comments: 34 (19 by maintainers)
Commits related to this issue
- Add FAQ entry for "server sends HTTP headers in a loop" Older versions of a few commercial HTTPS servers don't handle extended-master-secret and/or encrypt-then-mac very well, but we end up getting a... — committed to kaduk/web by kaduk 4 years ago
- Add FAQ entry for "server sends HTTP headers in a loop" Older versions of a few commercial HTTPS servers don't handle extended-master-secret and/or encrypt-then-mac very well, but we end up getting a... — committed to openssl/web by kaduk 4 years ago
I am not very enthusiastic about adding an environment variable to control this behavior, for several reasons: first, it’s not a “fix”, it’s a workaround for broken servers; second, we are generally reluctant to add new environment variables at all; and third, for this particular case, adding the environment variable would add an easy way to inadvertently (and in a hard-to-diagnose manner) weaken the security properties afforded to TLS clients by openssl. BTW, the
-EncryptThenMacdirective in the openssl configuration file has the same effect, if your application level allows a way to specify a configuration file to use. (There is the system global config, which is not a great knob, and IIRC theOPENSSL_CONFenvironment variable is also consulted.)Sure, something like that. As well as the concrete advice to set the “no etm” option.
The world is nasty, and folks have to solve problems. OpenSSL saying “not my fault” doesn’t help a great deal, especially when we can advise on the work-around.
What would such an FAQ entry say? “Some deployed TLS servers are fatally buggy and do not implement extensibility in a standards-complaint manner; these servers may exhibit strange behavior such as repeating the HTTP headers in a loop after receiving a ClientHello that includes relatively new TLS extensions such as encrypt-then-mac and extended-master secret. While these new TLS extensions provide significant security benefits to clients and are rightly enabled by default in openssl, if bringing the server into compliance is not possible, the extension(s) in question can be disabled on a per-connection basis when talking to the buggy server, by using
SSL_set_options(3).”?Yes, I mean things like OpenSSL and python should not set that option by default. The current OpenSSL default is the correct default.
That corresponds to when libssl started sending the encrypt-then-mac and extended-master-secret extensions by default. And indeed, if I patch
s_clientto callSSL_set_options(con, SSL_OP_NO_ENCRYPT_THEN_MAC)right afterSSL_new(), then the reproducer from the initial report produces a (lengthy) error page instead of looping through the same output.So I think the resolution here is pretty clear: the server is buggy, and a workaround exists. (Note that encrypt-then-mac is only relevant for non-AEAD ciphers, though I do not expect the server in question to support any.) I’ll set an appropriate tag so the issue will be closed after a waiting period.