go: runtime: "fatal error: unexpected signal" 0xC0000005 on Windows for a small program with a large allocation

What version of Go are you using (go version)?

$ go version
go version go1.14 windows/amd64

Does this issue reproduce with the latest release?

The tests were run with Go 1.14 on a fully patched Windows 10 Home Version 1909 (Build 18363.657).

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Uli\AppData\Local\go-build
set GOENV=C:\Users\Uli\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\Uli\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\Uli\src\lz\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\Uli\AppData\Local\Temp\go-build893962726=/tmp/go-build -gno-record-gcc-switches
GOROOT/bin/go version: go version go1.14 windows/amd64
GOROOT/bin/go tool compile -V: compile version go1.14

What did you do?

I’m developing a package that creates Lempel-Ziv sequences for byte streams. On my usual development environment Ubuntu 18.04 I observe sometimes crashes of the test runs (fatal error: bad g->status in ready) for go1.13.8 and go1.14 on multiple kernels including 4.15.0 and 5.3.18. Both should not be affected by the AVX register corruption.

To exclude Linux as a factor I tested the package on Windows and got a fatal error on every run. I was able to reduce it to a small program. The program runs without any errors on Linux. Whether the Windows issue is related to the Linux problems I cannot tell. I’m aware that initializing a structure with a huge array this way is not a good idea, but that is what I wrote initially and what appears to trigger the stack extension that runs into an invalid address access.

I started the program with go run.

> go run main.go
package main

import "fmt"

type Sequencer struct {
        htable [1 << 17]uint32
        buf    []byte
}

func (s *Sequencer) Init(windowSize int) *Sequencer {
        if !(0 <= windowSize) {
                panic(fmt.Errorf("windowSize out of range [%d,%d]", 0, 0))
        }
        *s = Sequencer{
                buf: []byte{0xff},
        }

        return s
}

func main() {
        var s Sequencer
        s.Init(0)
}

https://play.golang.org/p/VRavJw-WPie

What did you expect to see?

No output and no fatal error.

What did you see instead?

fatal error: unexpected signal during runtime execution
[signal 0xc0000005 code=0x1 addr=0xc000134000 pc=0x4143a9]

runtime stack:
runtime.throw(0x4d5ec2, 0x2a)
        c:/go/src/runtime/panic.go:1112 +0x79
runtime.sigpanic()
        c:/go/src/runtime/signal_windows.go:240 +0x25a
runtime.runGCProg(0x4a294c, 0x0, 0xc000132000, 0x1, 0x579680)
        c:/go/src/runtime/mbitmap.go:1901 +0xd9
runtime.materializeGCProg(0x80008, 0x4a2948, 0x7bfc20)
        c:/go/src/runtime/mbitmap.go:1925 +0x93
runtime.adjustframe(0x7bfb30, 0x7bfc20, 0x579680)
        c:/go/src/runtime/stack.go:696 +0x272
runtime.gentraceback(0xffffffffffffffff, 0xffffffffffffffff, 0x0, 0xc00004a000, 0x0, 0x0, 0x7fffffff, 0x4d7720, 0x7bfc20, 0x0, ...)
        c:/go/src/runtime/traceback.go:334 +0x111c
runtime.copystack(0xc00004a000, 0x200000)
        c:/go/src/runtime/stack.go:888 +0x298
runtime.newstack()
        c:/go/src/runtime/stack.go:1043 +0x219
runtime.morestack()
        c:/go/src/runtime/asm_amd64.s:449 +0x97

goroutine 1 [copystack]:
main.(*Sequencer).Init(0xc0004dff60, 0x0, 0x0)
        C:/Users/Uli/src/lz/main.go:10 +0x1af fp=0xc0004dff48 sp=0xc0004dff40 pc=0x49f93f
main.main()
        C:/Users/Uli/src/lz/main.go:23 +0x76 fp=0xc00055ff88 sp=0xc0004dff48 pc=0x49f9c6
runtime.main()
        c:/go/src/runtime/proc.go:203 +0x212 fp=0xc00055ffe0 sp=0xc00055ff88 pc=0x434952
runtime.goexit()
        c:/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc00055ffe8 sp=0xc00055ffe0 pc=0x45cd61
exit status 2

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 25 (20 by maintainers)

Commits related to this issue

Most upvoted comments

Found it.

In this case, the GC bitmap for Sequencer will be 65536 zeros followed by 3 ones, or exactly 8 KiB of zero bytes, followed by an 0x7 byte. t.ptrdata is 524312 (the bytes of Sequencer up to an including the last pointer). The calculation for the scratch buffer size in materializeGCProg is wrong: (ptrdata/(8*sys.PtrSize)+pageSize-1)/pageSize. It needs to allocate 8193 bytes, but the ptrdata/(8*sys.PtrSize) rounds down and it allocates only 8192 bytes. That just happens to land on a page boundary, and I guess the next page happens to be unmapped, so runGCProg faults when it tries to write the last byte of the GC bitmap.

@dmitshur, technically a workaround could be to pad all types of size [N524288+1, N524288+63] for any integer N so they’re not that size any more. That’s both really awful (and can apply to code you depend on but don’t control), and you have to know that you’ve encountered this issue to even think about doing something like that, which is most likely to just show up as memory corruption, which you might not even notice.

So, practically speaking, I’d say there isn’t really a workaround.

@networkimprov, now that 1.14 has been released I would not expect any further 1.12.x releases.