go: net/http: don't block RoundTrip when the Transport hits MaxConcurrentStreams
The CL that disables connection pooling for HTTP2 creates a significant discontinuity in throughput when the server specifies a small number of maximum concurrent streams.
https://go-review.googlesource.com/c/net/+/53250
HTTP2 support is automatically enabled in Go under conditions not always specified by the developer. For example, configuration files often alternate between http and https endpoints. When using an http endpoint, go will use HTTP/1, whereas https endpoints use HTTP/2.
The HTTP/1 default transport will create as many connections as needed in the background. The HTTP2 default transport does not (although it used to).
As a result, HTTP1 endpoints get artificially high throughput when compared to HTTP2 endpoints that block waiting for more streams to become available instead of creating a new connection. For example, the AWS ALB limit the maximum number of streams to 128.
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html
This HTTP/2 client is blocked once it hits 128 streams and waits for more to become available. The HTTP/1 client does not. The performance of the HTTP/1 client is orders of magnitude faster as a result. This effect is annoying and creates a leaky abstraction in the net/http package.
The consequence of this is that importers of the net/http package now have to:
1.) Distinguish between HTTP and HTTPS endpoints 2.) Write a custom connection pool for the transport when HTTP2 is enabled
I think the previous pooling functionality should be restored.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 13
- Comments: 25 (17 by maintainers)
Commits related to this issue
- http2: revert Transport's strict interpretation of MAX_CONCURRENT_STREAMS And add the http2.Transport.StrictMaxConcurrentStreams bool knob to behavior being reverted. In CL 53250 for golang/go#13774... — committed to golang/net by bradfitz 6 years ago
- http2: revert Transport's strict interpretation of MAX_CONCURRENT_STREAMS And add the http2.Transport.StrictMaxConcurrentStreams bool knob to behavior being reverted. In CL 53250 for golang/go#13774... — committed to Appboy/net by bradfitz 6 years ago
- http2: revert Transport's strict interpretation of MAX_CONCURRENT_STREAMS (#1) And add the http2.Transport.StrictMaxConcurrentStreams bool knob to behavior being reverted. In CL 53250 for golang/... — committed to Appboy/net by froodian 5 years ago
I’m leaning towards reverting this behavior for Go 1.12 and making it more opt-in somehow.
To add some numbers, with unpatched HTTP/2 library we get around 1.5Gbit/s (up+down), with same settings and parts of the patch removed to allow more connections to be open we get 7Gbit/s (up+down) over several connections.
Number of concurrent streams is limited on server side to 30, this might not be the optimal number, we’ll keep testing but definitely +1 for this change to be reverted or making it configurable.
Sounds like the decision is to revert this behavior for Go 1.12. Maybe new API in Go 1.13.
I’d like to second the sentiment that using browsers as our only guidance behind this doesn’t feel like the best path, simply because of the different use cases between a user browsing a blog and a service that’s built to multiplex lots of requests to backend systems. I think the browser functionality should be considered as part of the decision, but we should also take a look at the HTTP/2 implementations in other languages too.
With gRPC using HTTP/2, and its usage in the industry growing, polyglot interoperability is becoming more prevalent. I think we should make sure Go is going to play nicely in those ecosystems, as well as be a viable option over other languages. It’d be unfortunate for it to have some sort of red mark like this that would prevent people from adopting Go.
We also need to be able to monitor in-flight requests at the connection pool level so we can anticipate the need for the opening of a new connections. Here is an old proposal on that: HTTP/2 Custom Connection Pool.
I’m not sure what browsers do is the most relevant for Go. I would assume Go is more often used to build proxies or backend API clients than user facing HTTP clients. Having the HTTP library choosing to block a request with no good way for the caller to control or avoid it, is a big no-no for any low latency projects IMHO.
We’re seeing significantly reduced throughput after this change using http://google.golang.org/api/bigquery/v2 to stream data into BigQuery.
Nothing prevents a load balancer from replying with a high max concurrent streams. The problem I see is h2 implementations using the recommended minimum
SETTINGS_MAX_CONCURRENT_STREAMSfor no reason. I would guess this is because implementors simply choose the only value mentioned. I’ve also not seen yet an implementation that re-negotiates this value upwards.Go’s current h2 bundle uses a value of 1000 for client connections by default, but only 250 for server connections. Further, the server stream comment refers to the Google Front End using a default value of 100—and I would think that Google servers can handle more than 100 concurrent requests, regardless of whether they are from one host, or, say, a proxy server.
If a user has it configured to allow an infinite number of connections per host, then I believe the H2 implementation then should create a new connection when it has reached the server-advertised maximum.
However, there is some level of overhead in creating a new connection, perhaps a deadline on the RoundTrip block to attempt to stay in the server configured maxStreams before offloading the request to a new connection?