go: x/net/http2: connecting to non-compliant HTTP2 server returns Client.Timeout exceeded; fallback to HTTP/1?

What version of Go are you using (`go version`)?

go version go1.8.3 darwin/amd64

What operating system and processor architecture are you using (`go env`)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/tkng"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.8.3/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.8.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/z1/nc5xfnrs71g2zxr_chmshx3h0000gp/T/go-build993898931=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

Following snippet code should be enough to reproduce the problem.

https://play.golang.org/p/PmZWp6NJlq

If I disabled HTTP2 by the environment variable GODEBUG=http2client=0, then I see no error. Hence I guess this issue is related to HTTP2. This is might be the same as #13959, but I’m not sure.

Also, I’m not sure this is the problem of golang’s HTTP2 client library, or the problem of server side. (In most case, HTTP2 client works nicely.)

What did you expect to see?

I can connect to the server (by http2, or falling back to http1.1). At least, I want to see a more suggestive error message.

What did you see instead?

I saw an error message like following:

2017/07/11 15:22:01 http.Get failed, error: Get https://precious.jp: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

About this issue

Original URL
State: open
Created 7 years ago
Comments: 30 (18 by maintainers)

Commits related to this issue

Force TLS negotiation to only support HTTP/1.1 Seems OKTA has a broken implementation of HTTP2? There seem to be some quirks with they way golang handles HTTP2 requests (specifically when receiving GO... — committed to mvallaly-rally/aws-okta by mvallaly-rally 4 years ago

Most upvoted comments

Hello everyone, I’m in charge of F5’s HTTP/2 code. We are aware of the problem and patched it in our upcoming release months ago. In a matter of days, this will also be available as a patch to deployed versions.

If this problem is affecting you, please contact F5 support and ask for the patch for bug number 677119.

+11

martinduke on Aug 4, 2017

@tombergan, what do you think of changing Go’s http Transport to retry Requests with HTTP/1.1 if it sees a GOAWAY with LastStreamID=0?

We could do that but it would be somewhat complex: The H2 transport needs to return errGoAwayBeforeAnyRequests if it sees GOAWAY with LastStreamID=0. Then pconn.shouldRetryRequest needs to recognize this error. So far not too bad. https://github.com/golang/go/blob/75ab6134fcea003803e25c9bad7f092be3eeb5c3/src/net/http/transport.go#L418

When that error happens (and only that error), we need to disable TLSNextProto on the next call to Transport.getConn and Transport.dialConn. Currently there’s no easy way to do this, and adding such plumbing could get ugly. IMO, I’d rather wait until we had a second case where we could use this feature before adding it.

The current issue can be avoided by having the client set http2.Transport.MaxHeaderListSize to 32768 or lower. That seems like a better short-term fix while clients wait for the next F5 release to roll out.

tombergan on Aug 9, 2017

Sorry for my wrong comment about Netty caused confusions.

If my understanding is correct, precious.jp (the site we have the problem now) runs with Big-IP. However I don’t have evidence about that, since the HTTP server hides the name of the software and version of the software.

Unfortunately, I don’t have a reply from site owner for several days, there’s little hope that reply will come.

Sending a GOAWAY frame is a totally reasonable thing to do

Is it? That sounds like a painful workaround for a broken server that will helpfully soon be fixed.

Sadly, I don’t think we can hope the server will soon be fixed. Even contacting them is hard.

Put it aside, IMO it’s basically a good idea for smooth transition to downgrade a protocol to an old one if a new protocol doesn’t work. (If there’s no security issue.)

Whether the standard library should automatically downgrade the protocol that is using is controversial, however I hope that the programmer is possible to downgrade if it’s needed.

Current implementation doesn’t allow that behavior to developers, since it doesn’t return any error message (it seems not returns infinitely, or returns with timeout error if timeout is set). I think there is room for improvement for this aspect.

I hope golang’s HTTP library to return an error message when the HTTP2 protocol was used and it received GO_AWAY frame before received anything.

tkng on Aug 21, 2017