reqwest: Hangs in OSX reading certain sized responses unless specifying `Connection::close`

I’m making an authenticated GET request that succeeds, but any attempt to read_to_string or read_to_end just hangs indefinitely, but only under all the following criteria:

  • OSX (works fine on linux)
  • responses in the range of 3835-16120 bytes (~260 bytes short of the 4K/16K boundaries).
  • over SSL (works fine if I make the same request through a local http nginx proxy)

This is the debug output of an example Response object I see before it hangs during reading:

Response { inner: Response { status: Ok, headers: Headers { Date: Wed, 23 Nov 2016 03:06:35 GMT, X-Frame-Options: DENY, Strict-Transport-Security: max-age=0; includeSubDomains; preload, X-Data-Type: directory, Connection: keep-alive, Content-Length: 7452, Content-Type: application/json; charset=utf-8, }, version: Http11, url: "https://api.algorithmia.com/v1/connector/data/anowell/foo", status_raw: RawStatus(200, "OK"), message: Http11Message { is_proxied: false, method: None, stream: Wrapper { obj: Some(Reading(SizedReader(remaining=7452))) } } } }

And yet, I haven’t managed to repro this with a simple test script to just reqwest::get some public URL in that size range, so I’m still missing some contributing factor. I’ve also tried an old build of my client which used hyper+openssl, and it had no issue.

Things on my mind to try still:

  • Read in smaller chunks to see if perhaps read_to_end is waiting on the wrong number of bytes
  • Put together the minimal repro using an API key for a shareable test account
  • Try a bit harder to find a repro outside of our API

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 47 (43 by maintainers)

Commits related to this issue

Most upvoted comments

I believe I’ve found the issue. Searching the Apple’s source code for SSLRead revealed that if called with data length longer than what was buffered internally in SslContext, it would copy from the buffer, and then perform a socket read. This is contrary to how typical IO objects work, such as TCP will first return the data in its buffer, and only subsequent calls will block on the socket.

We should have a new update of security-framework soon, and then a cargo update should hopefully fix everyone.

Published security-framework 0.1.14 with this fix!

Woo! Closing as fixed. Thanks all for helping track down this issue.

@sfackler

OMG APPLE WTF

got a laugh out of that