go: runtime: fatal error: sweep increased allocation count, go1.9.x

What version of Go are you using (`go version`)?

go version go1.9.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

Ubuntu Server 16.04.3 LTS with PostgreSQL 9.5.10 and OpenSSH_7.2p2 selected during install.

Intel(R) Core(TM) i5-6400 CPU

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/pciet/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

Load testing by acting as 40 clients automatically without delays between actions playing my chess game implemented as a Go HTTP server with a postgres database, which should partially simulate a load of hundreds of players. Crash within 30 seconds as the number of active clients ramps up. The clients are making HTTP calls from a separate 2015 MacBook Pro on the local network.

What did you expect to see?

The load test should run indefinitely.

What did you see instead?

Crash with the signature “fatal error: sweep increased allocation count” reproducible within 30 seconds. The crash does not happen with a client count of eight or less. The crash does not happen with go1.6.4, go1.7.6, or go1.8.4, but does with go1.9, go1.9.1, and go1.9.2.

A race condition is warned with go build -race set, but the memory indicated should not have been shared between goroutines.

My investigation is detailed here, with stack traces (with and without GODEBUG=gccheckmark=1) and the race warnings: https://github.com/pciet/wichess/issues/19

My email to golang-nuts: https://groups.google.com/forum/#!topic/golang-nuts/7lcongdGOMM

I’ll try to put together a case with less code and without the database.

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 40 (37 by maintainers)

Commits related to this issue

[release-branch.go1.9] cmd/compile: fix mapassign_fast* routines for pointer keys The signature of the mapassign_fast* routines need to distinguish the pointerness of their key argument. If the affe... — committed to golang/go by randall77 7 years ago
test: fast GC+concurrency+types verification This test runs independent goroutines modifying a comprehensive variety of local vars to look for garbage collector regressions. This test has been verifi... — committed to golang/go by pciet 6 years ago

Most upvoted comments

Here’s a small autocontained reproducer:

package main

type Path struct {
	P []int
}

type ZMap map[*Path]struct{}

func F() {
	zm := make(ZMap)
	for i := 0; i < 10; i++ {
		pp := Path{make([]int, 0, 8)}
		zm[&pp] = struct{}{}
	}
}

func main() {
	for i := 0; i < 16; i++ {
		go func() {
			for true {
				F()
			}
		}()
	}
	select {}
}

It crashes fairly quickly (usually within one minute) when running GOMAXPROCS=4 go run test.go on two different machines I tried (both with go1.9.2, but I also see crashes on tip).

ALTree on Nov 21, 2017

go1.9.3 has been packaged and includes:

CL 88635 [release-branch.go1.9] cmd/compile: fix mapassign_fast* routines for pointer keys

The release is posted at golang.org/dl.

— golang.org/x/build/cmd/releasebot, Jan 22 21:02:56 UTC

andybons on Jan 22, 2018