go: bufio: nil pointer dereference

What version of Go are you using (`go version`)?

$ go version
go1.18.3

Does this issue reproduce with the latest release?

I don’t no it’s a single occurrence in a system running about 500 instances every day 24x7.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/share/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/share/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.3"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build813518630=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Run the binary in a container and this was a single occurrence after months of running different versions of the binary (we regularly update it ~weekly).

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x52160a]
container=skipper-ingress version=v0.13.220 container_id=22847bac9b2e73e49c21b85e150d93c67919bfa4e306ba2442d9d2ee48e1d6f6 pod=skipper-ingress-7d7bc75576-5nzn9 stream=stderr component=ingress application=skipper-ingress namespace=kube-system
goroutine 31240317 [running]:
bufio.(*Writer).Available(...)
/usr/local/go/src/bufio/bufio.go:645
bufio.(*Writer).WriteString(0x0, {0x1118d77?, 0x7fdacf6cf0f8?})
/usr/local/go/src/bufio/bufio.go:735 +0x6a
net/http.(*expectContinueReader).Read(0xc009fd4560, {0xc002833980, 0x5c, 0x5c})
/usr/local/go/src/net/http/server.go:908 +0xfd
io.(*LimitedReader).Read(0xc009fbee28, {0xc002833980?, 0xc0056d3580?, 0xc005368930?})
/usr/local/go/src/io/io.go:476 +0x45
io.copyBuffer({0x129a7a0, 0xc0056d3580}, {0x1298f00, 0xc009fbee28}, {0x0, 0x0, 0x0})
/usr/local/go/src/io/io.go:426 +0x1b2
io.Copy(...)
/usr/local/go/src/io/io.go:385
net.genericReadFrom({0x1299120?, 0xc009f91e00?}, {0x1298f00, 0xc009fbee28})
/usr/local/go/src/net/net.go:662 +0x6a
net.(*TCPConn).readFrom(0xc009f91e00, {0x1298f00, 0xc009fbee28})
/usr/local/go/src/net/tcpsock_posix.go:54 +0x78
net.(*TCPConn).ReadFrom(0xc009f91e00, {0x1298f00?, 0xc009fbee28?})
/usr/local/go/src/net/tcpsock.go:130 +0x36
io.copyBuffer({0x1299120, 0xc009f91e00}, {0x1298f00, 0xc009fbee28}, {0x0, 0x0, 0x0})
/usr/local/go/src/io/io.go:412 +0x14b
io.Copy(...)
/usr/local/go/src/io/io.go:385
net/http.persistConnWriter.ReadFrom({0x438f3c?}, {0x1298f00?, 0xc009fbee28?})
/usr/local/go/src/net/http/transport.go:1775 +0x5b
bufio.(*Writer).ReadFrom(0xc0051a1d40, {0x1298f00, 0xc009fbee28})
/usr/local/go/src/bufio/bufio.go:768 +0x1ab
io.copyBuffer({0x1296cc0, 0xc0051a1d40}, {0x1298f00, 0xc009fbee28}, {0x0, 0x0, 0x0})
/usr/local/go/src/io/io.go:412 +0x14b
io.Copy(...)
/usr/local/go/src/io/io.go:385
net/http.(*transferWriter).doBodyCopy(0xc000d999a0, {0x1296cc0?, 0xc0051a1d40?}, {0x1298f00?, 0xc009fbee28?})
/usr/local/go/src/net/http/transfer.go:411 +0x4d
net/http.(*transferWriter).writeBody(0xc000d999a0, {0x1296cc0, 0xc0051a1d40})
/usr/local/go/src/net/http/transfer.go:369 +0x3d5
net/http.(*Request).write(0xc009880a00, {0x1296cc0, 0xc0051a1d40}, 0x0, 0x0, 0xc009fbede0)
/usr/local/go/src/net/http/request.go:698 +0xb46
net/http.(*persistConn).writeLoop(0xc003a07e60)
/usr/local/go/src/net/http/transport.go:2395 +0x185
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1751 +0x1791

What did you expect to see?

no panic withing the stdlib

What did you see instead?

A panic

About this issue

Original URL
State: open
Created 2 years ago
Comments: 21 (11 by maintainers)

Most upvoted comments

@szuecs Ok, I’m going to assign you as owner so no one has to look at this until more information arrives.

randall77 on Jan 23, 2023

The cause looks to be body read after connection close which nullifies bufw on final flush https://github.com/golang/go/blob/4068be56ce7721a3d75606ea986d11e9ca27077a/src/net/http/server.go#L1698

The read then fails on attempt to write https://github.com/golang/go/blob/4068be56ce7721a3d75606ea986d11e9ca27077a/src/net/http/server.go#L908

Could be reproduced via:

package main

import (
	"log"
	"net/http"
	"time"
)

func main() {
	s := &http.Server{
		Addr: ":8888",
		Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			go func() {
				time.Sleep(1000 * time.Millisecond)
				r.Body.Read(make([]byte, 1))
			}()
		}),
	}
	log.Fatal(s.ListenAndServe())
}

# trigger use of expectContinueReader
curl -v -H "Expect: 100-continue" -d"test" localhost:8888

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x4fb40a]

goroutine 5 [running]:
bufio.(*Writer).Available(...)
        /usr/lib/go-1.18/src/bufio/bufio.go:645
bufio.(*Writer).WriteString(0x0, {0x666d76?, 0x0?})
        /usr/lib/go-1.18/src/bufio/bufio.go:735 +0x6a
net/http.(*expectContinueReader).Read(0xc00006a040, {0xc00012c27b, 0x1, 0x1})
        /usr/lib/go-1.18/src/net/http/server.go:908 +0xfd
main.main.func1.1()

AlexanderYastrebov on Jul 13, 2022