go: runtime: Go randomly crashes on Windows with no stack trace, just "unknown pc" and memory/registry dump

What version of Go are you using (go version)?

$ go version
go version go1.14.3 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

OS is Windows 10 Pro 10.0.18362

$ go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\jconnell\AppData\Local\go-build
set GOENV=C:\Users\jconnell\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\jconnell\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\jconnell\go\src\github.com\jasontconnell\chashprms\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\jconnell\AppData\Local\Temp\go-build801242386=/tmp/go-build -gno-record-gcc-switches
GOROOT/bin/go version: go version go1.14.3 windows/amd64
GOROOT/bin/go tool compile -V: compile version go1.14.3

What did you do?

I don’t think the code matters. It has crashed with multiple different programs. But here’s an example. It’s more in the “running it repeatedly” that breaks it. I’ve tested it on Go 1.13 and it runs fine. The earliest version that I’ve tried that showed the error was Go 1.14.3 as displayed in the version section of this post. It is reproducible on Go 1.15 as well.

So my manager was reporting a crash, a memory dump. I wasn’t able to reproduce it locally. But then I wrote the simple program

https://play.golang.org/p/KMabbeiz1hW

And a powershell script

For ($i = 0; $i -lt 100; $i++) { Write-Host $i .\chashprms “$i is the value” }

Run go build and run.ps1.

What did you expect to see?

If I run the same code in Docker using the golang:1.15 image, it runs fine. Runs through 100 instances and always prints out the hash. I would expect to see 100 hashes printed. In Go 1.13 it works fine (the latest I tested where it works).

What did you see instead?

In Go 1.14.3 (the earliest that I tested where it crashes), I see a memory and registry dump after a few calls:

Exception 0xc0000005 0x0 0x7ffd4c110fff 0x910000 PC=0x910000

runtime: unknown pc 0x910000 stack: frame={sp:0x84f6a0, fp:0x0} stack=[0x0,0x84ff00) 000000000084f5a0: 000000000098d7c0 000000000084f610 000000000084f5b0: 0000000000000000 000000000084f6a8 000000000084f5c0: 0000000000989710 000000000084f600 000000000084f5d0: 0000000000000000 00007ffd48e60108 000000000084f5e0: 0000000000000000 0000000000963880 000000000084f5f0: 000000000084f738 00007ffd49d194d1 000000000084f600: 00007ffd49d194cb 00007ffd4bf3ba6f 000000000084f610: 00007ffd4bf200d8 0000000000975c50 000000000084f620: 0000000000000005 000000000084f670 000000000084f630: 00007ffd49d17900 00007ffd4bf51810 000000000084f640: 0000000000963880 0000000000000000 000000000084f650: 0000000000050005 00007ffd49d194cb 000000000084f660: 000000000000001b 0000000000000000 000000000084f670: 0000000000000000 00007ffd4bf200d8 000000000084f680: 0000000000000000 00007ffd49ac0000 000000000084f690: 0000000000000001 00007ffd4bf3c734 000000000084f6a0: <0000000000000001 0000000000000000 000000000084f6b0: 00007ffd0000ebb0 000000000084f7a8 000000000084f6c0: 00000000009897e0 000000000084f728 000000000084f6d0: 0000000000250024 00007ffd48e7b816 000000000084f6e0: 000000000098d690 00007ffd4c06c500 000000000084f6f0: 0000000000963880 00007ffd4c06ea5c 000000000084f700: 000000000000094c 00007ffd48deb2d8 000000000084f710: 00007ffd48de9138 00007ffd4c06c528 000000000084f720: 00007ffd4c07ec6a 00007ffd48de0000 000000000084f730: 00007ffd4c070f8c 00007ffd4bfa0d80 000000000084f740: 000000001c1dc789 000000000098d818 000000000084f750: 000000000084fab0 00000000c0000135 000000000084f760: 0000000000000000 0000000000000040 000000000084f770: 0000000000000004 00007ffd4c0852f0 000000000084f780: 0000000000000001 000000000084f900 000000000084f790: 0000000000975c50 00007ffd4bf4e2a8 runtime: unknown pc 0x910000 stack: frame={sp:0x84f6a0, fp:0x0} stack=[0x0,0x84ff00) 000000000084f5a0: 000000000098d7c0 000000000084f610 000000000084f5b0: 0000000000000000 000000000084f6a8 000000000084f5c0: 0000000000989710 000000000084f600 000000000084f5d0: 0000000000000000 00007ffd48e60108 000000000084f5e0: 0000000000000000 0000000000963880 000000000084f5f0: 000000000084f738 00007ffd49d194d1 000000000084f600: 00007ffd49d194cb 00007ffd4bf3ba6f 000000000084f610: 00007ffd4bf200d8 0000000000975c50 000000000084f620: 0000000000000005 000000000084f670 000000000084f630: 00007ffd49d17900 00007ffd4bf51810 000000000084f640: 0000000000963880 0000000000000000 000000000084f650: 0000000000050005 00007ffd49d194cb 000000000084f660: 000000000000001b 0000000000000000 000000000084f670: 0000000000000000 00007ffd4bf200d8 000000000084f680: 0000000000000000 00007ffd49ac0000 000000000084f690: 0000000000000001 00007ffd4bf3c734 000000000084f6a0: <0000000000000001 0000000000000000 000000000084f6b0: 00007ffd0000ebb0 000000000084f7a8 000000000084f6c0: 00000000009897e0 000000000084f728 000000000084f6d0: 0000000000250024 00007ffd48e7b816 000000000084f6e0: 000000000098d690 00007ffd4c06c500 000000000084f6f0: 0000000000963880 00007ffd4c06ea5c 000000000084f700: 000000000000094c 00007ffd48deb2d8 000000000084f710: 00007ffd48de9138 00007ffd4c06c528 000000000084f720: 00007ffd4c07ec6a 00007ffd48de0000 000000000084f730: 00007ffd4c070f8c 00007ffd4bfa0d80 000000000084f740: 000000001c1dc789 000000000098d818 000000000084f750: 000000000084fab0 00000000c0000135 000000000084f760: 0000000000000000 0000000000000040 000000000084f770: 0000000000000004 00007ffd4c0852f0 000000000084f780: 0000000000000001 000000000084f900 000000000084f790: 0000000000975c50 00007ffd4bf4e2a8 rax 0x7ffd48debe7c rbx 0x7ffd48debe7a rcx 0x41 rdi 0xffffffffffbadd11 rsi 0x0 rbp 0x84f900 rsp 0x84f6a0 r8 0x0 r9 0x0 r10 0x0 r11 0x94b r12 0x7ffd4bf20000 r13 0x0 r14 0x7ffd48debe7c r15 0xc000007a rip 0x910000 rflags 0x10202 cs 0x33 fs 0x53 gs 0x2b

We are a Windows shop and do .NET dev and these tools I wrote help with that, I would love to be able to keep them in Go 😄 For now I have to revert them to 1.13ish but I’ll be watching this closely. I am available and on the Gopher slack, if you need more information I’ll be happy to provide.