go: net/http, x/net/http2: http server shutdown doesn't gracefully shut down HTTP2 connections

What version of Go are you using (go version)?

$ go version
go version go1.13.7 darwin/amd64

Does this issue reproduce with the latest release?

Yes, tested on go version go1.14.4 darwin/amd64

What operating system and processor architecture are you using (go env)?

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/lszaszki/Library/Caches/go-build"
GOENV="/Users/lszaszki/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/lszaszki/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/4x/d8vbcg5s6lx229_q7l5wbw0h0000gn/T/go-build581125651=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I wrote a simple unit test that sets up an HTTP2 connection between a client and the server, sends a few requests and shuts down the backend server as soon as the last request arrives at the server. Since each request waits for 60 seconds in the handler I consider these requests to be active.

I analyzed net/http package and here’s what I have found out. On calling http.server.Shutdown() the server will be closed immediately because the connection is in StateNew (not in StateActive). In general net/http package manages connection states and provides a hook (ConnState) for reporting connections state. For HTTP 1.x it will transition the connection to StateActive as soon as it reads any byte on the wire. For HTTP 2.x h2_bundle.go will transition the connection to StateActive once there is more than one active stream. But the issue here is that ConnState hook is not wired. At least I haven’t found the wiring. So the server doesn’t know the current state of the connection.

I changed net/http so that it provides a default ConnState method. It is used by the HTTP2 implementation to report back the connection state. With this modification, my test takes ~60 seconds. The server waits until all requests complete before shutting down and all requests complete with success (HTTP 200).

What did you expect to see?

According to the documentation, the Shutdown method must wait indefinitely for active connections to return to the idle state and only then shut down the server. In that case, I would expect all requests to complete with success (HTTP 200) and to test take at least 60 seconds because each request simulates work by sleeping for that duration.

What did you see instead?

My test ended after ~6 seconds, all requests failed with Get "https://localhost:52066": http2: server sent GOAWAY and closed the connection; LastStreamID=49, ErrCode=NO_ERROR, debug="".