go: cmd/compile: fatal error: runtime: cannot map pages in arena address space

What version of Go are you using (go version)?

go version go1.9.2 windows/386

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

set GOARCH=386
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=386
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\karalabe\go
set GORACE=
set GOROOT=C:\Work\go
set GOTOOLDIR=C:\Work\go\pkg\tool\windows_386
set GCCGO=gccgo
set GO386=sse2
set CC=gcc
set GOGCCFLAGS=-m32 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\karalabe\AppData\Local\Temp\go-build701140035=/tmp/go-build -gno-record-gcc-switches
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config

What did you do?

We’ve noticed on our CI that 32bit Windows builds started crashing during compilation https://ci.appveyor.com/project/ethereum/go-ethereum/build/master.4791/job/hcsxkgy0uh018j05#L136.

After playing around with it trying to repro locally, we’ve managed to triage it to the 32bit Windows build of Go. To reproduce the issue yourself:

  • Download and install the 32bit version of Go on Windows
  • Download and install the 32bit version of MinGW64 on Windows
  • Run go get -v github.com/ethereum/go-ethereum/dashboard

The repository is quite large, so downloading it will take a while.

What did you expect to see?

A successful build.

What did you see instead?

>go get -v github.com/ethereum/go-ethereum/dashboard
github.com/ethereum/go-ethereum/vendor/github.com/go-stack/stack
github.com/ethereum/go-ethereum/vendor/github.com/aristanetworks/goarista/monotime
github.com/ethereum/go-ethereum/crypto/sha3
github.com/ethereum/go-ethereum/common/hexutil
github.com/ethereum/go-ethereum/common/math
github.com/ethereum/go-ethereum/rlp
github.com/ethereum/go-ethereum/vendor/github.com/rcrowley/go-metrics
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/httpu
github.com/ethereum/go-ethereum/common/mclock
github.com/ethereum/go-ethereum/crypto/secp256k1
github.com/ethereum/go-ethereum/log
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/scpd
github.com/ethereum/go-ethereum/common
github.com/ethereum/go-ethereum/event
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/soap
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/ssdp
github.com/ethereum/go-ethereum/vendor/golang.org/x/net/html/atom
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/internal/identifier
github.com/ethereum/go-ethereum/vendor/golang.org/x/net/html
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/transform
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/internal/utf8internal
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/internal/tag
github.com/ethereum/go-ethereum/vendor/github.com/jackpal/go-nat-pmp
github.com/ethereum/go-ethereum/p2p/netutil
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/language
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/runes
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/comparer
github.com/ethereum/go-ethereum/vendor/github.com/rcrowley/go-metrics/exp
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/cache
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/internal
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/storage
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/filter
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/charmap
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/japanese
github.com/ethereum/go-ethereum/metrics
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/korean
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/simplifiedchinese
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/traditionalchinese
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/unicode
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/errors
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/iterator
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/journal
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/opt
github.com/ethereum/go-ethereum/vendor/github.com/golang/snappy
github.com/ethereum/go-ethereum/vendor/golang.org/x/net/context
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/memdb
github.com/ethereum/go-ethereum/vendor/golang.org/x/net/websocket
github.com/ethereum/go-ethereum/vendor/gopkg.in/fatih/set.v0
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/table
github.com/ethereum/go-ethereum/vendor/gopkg.in/natefinch/npipe.v2
github.com/ethereum/go-ethereum/vendor/github.com/rs/xhandler
github.com/ethereum/go-ethereum/vendor/golang.org/x/text/encoding/htmlindex
github.com/ethereum/go-ethereum/vendor/golang.org/x/net/html/charset
github.com/ethereum/go-ethereum/vendor/github.com/rs/cors
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb
github.com/ethereum/go-ethereum/rpc
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/dcps/internetgateway1
github.com/ethereum/go-ethereum/vendor/github.com/huin/goupnp/dcps/internetgateway2
github.com/ethereum/go-ethereum/p2p/nat
github.com/ethereum/go-ethereum/crypto
github.com/ethereum/go-ethereum/crypto/ecies
github.com/ethereum/go-ethereum/p2p/discover
github.com/ethereum/go-ethereum/p2p/discv5
github.com/ethereum/go-ethereum/p2p
github.com/ethereum/go-ethereum/dashboard
# github.com/ethereum/go-ethereum/dashboard
runtime: VirtualAlloc of 1703936 bytes failed with errno=487
fatal error: runtime: cannot map pages in arena address space

runtime stack:
runtime.throw(0xa85b46, 0x30)
        c:/go/src/runtime/panic.go:605 +0x7c
runtime.sysMap(0x43f20000, 0x1a0000, 0x3377fe01, 0xdac358)
        c:/go/src/runtime/mem_windows.go:124 +0x113
runtime.(*mheap).sysAlloc(0xd97b20, 0x1a0000, 0x33648000)
        c:/go/src/runtime/malloc.go:470 +0x231
runtime.(*mheap).grow(0xd97b20, 0xcc, 0x0)
        c:/go/src/runtime/mheap.go:887 +0x4e
runtime.(*mheap).allocSpanLocked(0xd97b20, 0xcc, 0xdac368, 0x94c5b9c5)
        c:/go/src/runtime/mheap.go:800 +0x2e4
runtime.(*mheap).alloc_m(0xd97b20, 0xcc, 0x6a870101, 0x40e24826)
        c:/go/src/runtime/mheap.go:666 +0x115
runtime.(*mheap).alloc.func1()
        c:/go/src/runtime/mheap.go:733 +0x40
runtime.systemstack(0x3377ff14)
        c:/go/src/runtime/asm_386.s:409 +0x88
runtime.(*mheap).alloc(0xd97b20, 0xcc, 0x33000101, 0x41171b)
        c:/go/src/runtime/mheap.go:732 +0x6c
runtime.largeAlloc(0x196132, 0x33640100, 0x33eb980c)
        c:/go/src/runtime/malloc.go:827 +0x7f
runtime.mallocgc.func1()
        c:/go/src/runtime/malloc.go:722 +0x39
runtime.systemstack(0x13648000)
        c:/go/src/runtime/asm_386.s:393 +0x5e
runtime.mstart()
        c:/go/src/runtime/proc.go:1135

goroutine 1 [running]:
runtime.systemstack_switch()
        c:/go/src/runtime/asm_386.s:347 fp=0x14a93a0c sp=0x14a93a08 pc=0x44c3e0
runtime.mallocgc(0x196132, 0x0, 0x0, 0x0)
        c:/go/src/runtime/malloc.go:721 +0x67e fp=0x14a93a60 sp=0x14a93a0c pc=0x40e4ce
runtime.rawstring(0x196132, 0x0, 0x0, 0x0, 0x0, 0x0)
        c:/go/src/runtime/string.go:245 +0x56 fp=0x14a93a74 sp=0x14a93a60 pc=0x43c666
runtime.rawstringtmp(0x0, 0x196132, 0xc, 0x8, 0x0, 0x0, 0x0)
        c:/go/src/runtime/string.go:115 +0x51 fp=0x14a93a90 sp=0x14a93a74 pc=0x43c1b1
runtime.concatstrings(0x0, 0x14a93af8, 0x3, 0x3, 0x12272ccc, 0x40e1a8)
        c:/go/src/runtime/string.go:46 +0xc8 fp=0x14a93ad8 sp=0x14a93a90 pc=0x43be08
runtime.concatstring3(0x0, 0x43c88000, 0x19604b, 0x0, 0x0, 0x138945a1, 0xe7, 0x0, 0x13bd3310)
        c:/go/src/runtime/string.go:59 +0x39 fp=0x14a93af4 sp=0x14a93ad8 pc=0x43bfd9
strings.Join(0x142e1c30, 0x2, 0x2, 0x0, 0x0, 0x142e1c30, 0x1)
        c:/go/src/strings/strings.go:473 +0x2da fp=0x14a93b58 sp=0x14a93af4 pc=0x4a852a
cmd/compile/internal/gc.evconst(0x13834a50)
        c:/go/src/cmd/compile/internal/gc/const.go:676 +0x337 fp=0x14a93cdc sp=0x14a93b58 pc=0x8c9b87
cmd/compile/internal/gc.typecheck1(0x13834a50, 0x2, 0x69f64e)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:2215 +0x32a fp=0x14a94414 sp=0x14a93cdc pc=0x983d1a
cmd/compile/internal/gc.typecheck(0x13834a50, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a944dc sp=0x14a94414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13834af0, 0x2, 0x69f654)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a94c14 sp=0x14a944dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13834af0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a94cdc sp=0x14a94c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13834b90, 0x2, 0x69f66d)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a95414 sp=0x14a94cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13834b90, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a954dc sp=0x14a95414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13834c30, 0x2, 0x69f673)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a95c14 sp=0x14a954dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13834c30, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a95cdc sp=0x14a95c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13834cd0, 0x2, 0x69f681)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a96414 sp=0x14a95cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13834cd0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a964dc sp=0x14a96414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13834d70, 0x2, 0x69f687)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a96c14 sp=0x14a964dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13834d70, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a96cdc sp=0x14a96c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13834e10, 0x2, 0x69f6aa)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a97414 sp=0x14a96cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13834e10, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a974dc sp=0x14a97414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13834eb0, 0x2, 0x69f6b0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a97c14 sp=0x14a974dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13834eb0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a97cdc sp=0x14a97c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13834f50, 0x2, 0x69f6c2)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a98414 sp=0x14a97cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13834f50, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a984dc sp=0x14a98414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13834ff0, 0x2, 0x69f6c8)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a98c14 sp=0x14a984dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13834ff0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a98cdc sp=0x14a98c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835090, 0x2, 0x69f6e2)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a99414 sp=0x14a98cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835090, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a994dc sp=0x14a99414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835130, 0x2, 0x69f6e8)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a99c14 sp=0x14a994dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835130, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a99cdc sp=0x14a99c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x138351d0, 0x2, 0x6a044e)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9a414 sp=0x14a99cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x138351d0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9a4dc sp=0x14a9a414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835270, 0x2, 0x6a0454)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9ac14 sp=0x14a9a4dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835270, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9acdc sp=0x14a9ac14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835310, 0x2, 0x6a046d)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9b414 sp=0x14a9acdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835310, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9b4dc sp=0x14a9b414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x138353b0, 0x2, 0x6a0473)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9bc14 sp=0x14a9b4dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x138353b0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9bcdc sp=0x14a9bc14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835450, 0x2, 0x6a0481)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9c414 sp=0x14a9bcdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835450, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9c4dc sp=0x14a9c414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x138354f0, 0x2, 0x6a0487)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9cc14 sp=0x14a9c4dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x138354f0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9ccdc sp=0x14a9cc14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835590, 0x2, 0x6a04b1)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9d414 sp=0x14a9ccdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835590, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9d4dc sp=0x14a9d414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835630, 0x2, 0x6a04b7)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9dc14 sp=0x14a9d4dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835630, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9dcdc sp=0x14a9dc14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x138356d0, 0x2, 0x6a04c9)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9e414 sp=0x14a9dcdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x138356d0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9e4dc sp=0x14a9e414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835770, 0x2, 0x6a04cf)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9ec14 sp=0x14a9e4dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835770, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9ecdc sp=0x14a9ec14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835810, 0x2, 0x6a04e9)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9f414 sp=0x14a9ecdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835810, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9f4dc sp=0x14a9f414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x138358b0, 0x2, 0x6a04ef)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14a9fc14 sp=0x14a9f4dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x138358b0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14a9fcdc sp=0x14a9fc14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835950, 0x2, 0x6a04ff)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa0414 sp=0x14a9fcdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835950, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa04dc sp=0x14aa0414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x138359f0, 0x2, 0x6a04ff)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa0c14 sp=0x14aa04dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x138359f0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa0cdc sp=0x14aa0c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835a90, 0x2, 0x6a04ff)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa1414 sp=0x14aa0cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835a90, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa14dc sp=0x14aa1414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835b30, 0x2, 0x6a04ff)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa1c14 sp=0x14aa14dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835b30, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa1cdc sp=0x14aa1c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835bd0, 0x2, 0x6a0d4a)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa2414 sp=0x14aa1cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835bd0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa24dc sp=0x14aa2414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835c70, 0x2, 0x6a0d50)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa2c14 sp=0x14aa24dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835c70, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa2cdc sp=0x14aa2c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835d10, 0x2, 0x6a0d69)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa3414 sp=0x14aa2cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835d10, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa34dc sp=0x14aa3414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835db0, 0x2, 0x6a0d6f)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa3c14 sp=0x14aa34dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835db0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa3cdc sp=0x14aa3c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835e50, 0x2, 0x6a0d7e)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa4414 sp=0x14aa3cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835e50, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa44dc sp=0x14aa4414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835ef0, 0x2, 0x6a0d84)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa4c14 sp=0x14aa44dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835ef0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa4cdc sp=0x14aa4c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13835f90, 0x2, 0x6a0d9a)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa5414 sp=0x14aa4cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13835f90, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa54dc sp=0x14aa5414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13664050, 0x2, 0x6a0da0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa5c14 sp=0x14aa54dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13664050, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa5cdc sp=0x14aa5c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x136640f0, 0x2, 0x6a0db2)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa6414 sp=0x14aa5cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x136640f0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa64dc sp=0x14aa6414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13664190, 0x2, 0x6a0db8)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa6c14 sp=0x14aa64dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13664190, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa6cdc sp=0x14aa6c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13664230, 0x2, 0x6a0dd2)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa7414 sp=0x14aa6cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13664230, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa74dc sp=0x14aa7414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x136642d0, 0x2, 0x6a0dd8)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa7c14 sp=0x14aa74dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x136642d0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa7cdc sp=0x14aa7c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13664370, 0x2, 0x6a1460)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa8414 sp=0x14aa7cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13664370, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa84dc sp=0x14aa8414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13664410, 0x2, 0x6a1466)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa8c14 sp=0x14aa84dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13664410, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa8cdc sp=0x14aa8c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x136644b0, 0x2, 0x6a147f)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa9414 sp=0x14aa8cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x136644b0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa94dc sp=0x14aa9414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13664550, 0x2, 0x6a1485)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aa9c14 sp=0x14aa94dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13664550, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aa9cdc sp=0x14aa9c14 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x136645f0, 0x2, 0x6a1498)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aaa414 sp=0x14aa9cdc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x136645f0, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aaa4dc sp=0x14aaa414 pc=0x983592
cmd/compile/internal/gc.typecheck1(0x13664690, 0x2, 0x6a149e)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:569 +0x2d00 fp=0x14aaac14 sp=0x14aaa4dc pc=0x9866f0
cmd/compile/internal/gc.typecheck(0x13664690, 0x2, 0x0)
        c:/go/src/cmd/compile/internal/gc/typecheck.go:201 +0x622 fp=0x14aaacdc sp=0x14aaac14 pc=0x983592
...additional frames elided...

Issue being tracked on the go-ethereum repo too: https://github.com/ethereum/go-ethereum/issues/15738.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 15 (13 by maintainers)

Commits related to this issue

Most upvoted comments

TL;DR: For cmd/compile, this is a dup of #16394, which @mdempsky fixed in November (thanks!). For cmd/vet (really go/types), there is a new problem, for which I will file an issue and fix for Go 1.10. There is also a minor cmd/gofmt issue for Go 1.11, which I will file.

For the older releases I also sent a PR for go-bindata to emit balanced trees and also a PR to ethereum to use that go-bindata. Those are linked above and will fix the problems with older toolchains and ethereum. (Thanks @kevinburke and @karalabe.)


go get -u rsc.io/tmp/bigprog is a simple test of how our tools handle long arithmetic expressions like 1 + 1 + 1 + 1 + … + 1 or “1” + “1” + “1” + “1” + … + “1”, compared with balanced expressions like ((“1” + “1”) + (“1” + “1”)), inspired by the problem in #23222, which involved a string concatenation of about 700 strings. To run it, also do go get -u golang.org/x/tools/cmd/gotype.

The results on my 2015 Mac laptop, in both 64-bit and 32-bit mode, are below. This line:

      1000  int   0.019s  strbal   0.021s  strbigbal   0.037s  str   0.025s  strbig    0.032s

means that a tool was run on five different programs each with a single expression containing 1000 addition operations.

The int program is 1 + 1 + 1 + … + 1. The str program (later in the line) is “1” + “1” + “1” + … + “1”. The strbig program is str but each literal is actually 1000 1 characters: “111…1”. The strbal and strbigbal programs are str and strbig but add parentheses to make a balanced parse tree. Note that the tables report strbal and strbigbal before str and strbig, because strbig in particular is the worst case for many tools.

The compiler does very well, because Matthew fixed this in the parser to address #16394, in https://golang.org/cl/76450, during the Go 1.10 cycle. The reports of stack overflow in the Go 1.9.2 compiler are essentially dups of #16394, now fixed. Even before that change, the compiler needed stack to recurse over the whole expression but otherwise handled it efficiently, with a single allocation to construct the whole concatenation (I did that in 2014, mainly to speed program execution, but it had the effect of speeding compilation too (https://golang.org/cl/83090046)). With Matthew’s fix, there’s basically nothing left to do in the compiler.

The go/types package does not do as well, because it does not have the OADDSTR optimization in typecheck, nor in the parser. When go/constant builds up the actual value of the expression, it is accidentally quadratic on str: it allocates “1”, then “11”, then “111”, then “1111”, and so on. Similarly for strbig, with longer strings. On my Mac, a 32-bit gotype runs out of memory in a concatenation of 4000 1k strings (4MB total). On strbal and strbigbal it does better: the balanced tree makes it accidntally O(n log n). Luckily the go/constant API is well enough designed that the underlying representation can be changed to do the actual concatenation lazily, cutting it back to O(n) even on the unbalanced case. This is important to fix because the new automatic go vet uses go/types. I will file a new issue and send a CL for that, for Go 1.10.

Finally gofmt. I ran gofmt originally as a control, thinking that comparing to gotype would let us see how much time was spent in typechecking (gotype minus gofmt). Unfortunately, gofmt itself seems to have some accidentally quadratic behavior in making its layout decisions. (I won’t be surprised if the problem is in the code to decide whether to add spaces around operators, which would make it my fault, and if so, my apologies). I will file a Go 1.11 issue for gofmt.

amd64

go tool compile

      1000  int   0.019s  strbal   0.021s  strbigbal   0.037s  str   0.025s  strbig    0.032s
      2000  int   0.020s  strbal   0.023s  strbigbal   0.062s  str   0.032s  strbig    0.047s
      4000  int   0.022s  strbal   0.025s  strbigbal   0.099s  str   0.044s  strbig    0.078s
      8000  int   0.024s  strbal   0.032s  strbigbal   0.187s  str   0.072s  strbig    0.132s
     16000  int   0.029s  strbal   0.048s  strbigbal   0.370s  str   0.127s  strbig    0.226s
     32000  int   0.048s  strbal   0.076s  strbigbal   0.760s  str   0.238s  strbig    0.459s
     64000  int   0.059s  strbal   0.141s  strbigbal   1.551s  str   0.426s  strbig    0.861s
    128000  int   0.109s  strbal   0.275s  strbigbal   3.207s  str   0.869s  strbig    1.785s
    256000  int   0.220s  strbal   0.523s  strbigbal   6.641s  str   stack!  strbig    3.422s

gotype

      1000  int   0.023s  strbal   0.007s  strbigbal   0.022s  str   0.011s  strbig    0.254s
      2000  int   0.016s  strbal   0.008s  strbigbal   0.038s  str   0.014s  strbig    1.130s
      4000  int   0.027s  strbal   0.013s  strbigbal   0.066s  str   0.022s  strbig    4.861s
      8000  int   0.072s  strbal   0.020s  strbigbal   0.122s  str   0.042s  strbig   23.908s
     16000  int   0.199s  strbal   0.031s  strbigbal   0.262s  str   0.093s  strbig  sigkill!
     32000  int   0.530s  strbal   0.061s  strbigbal   0.536s  str   0.183s
     64000  int   1.630s  strbal   0.123s  strbigbal   1.060s  str   0.368s
    128000  int   5.338s  strbal   0.286s  strbigbal   2.243s  str   0.706s
    256000  int  22.431s  strbal   0.580s  strbigbal   4.652s  str   1.500s

go vet

      1000  int   0.217s  strbal   0.091s  strbigbal   0.128s  str   0.086s  strbig    0.323s
      2000  int   0.099s  strbal   0.092s  strbigbal   0.183s  str   0.094s  strbig    1.181s
      4000  int   0.109s  strbal   0.100s  strbigbal   0.271s  str   0.103s  strbig    4.676s
      8000  int   0.158s  strbal   0.111s  strbigbal   0.475s  str   0.130s  strbig   23.340s
     16000  int   0.386s  strbal   0.162s  strbigbal   0.895s  str   0.180s  strbig  sigkill! 
     32000  int   0.775s  strbal   0.245s  strbigbal   1.791s  str   0.292s
     64000  int   1.912s  strbal   0.430s  strbigbal   3.653s  str   0.536s
    128000  int   5.899s  strbal   0.808s  strbigbal   7.649s  str   0.978s
    256000  int  23.739s  strbal   1.617s  strbigbal  14.617s  str   stack! 

gofmt

      1000  int   0.035s  strbal   0.008s  strbigbal   0.030s  str   0.026s  strbig    0.047s
      2000  int   0.071s  strbal   0.012s  strbigbal   0.051s  str   0.079s  strbig    0.115s
      4000  int   0.291s  strbal   0.022s  strbigbal   0.092s  str   0.271s  strbig    0.362s
      8000  int   1.049s  strbal   0.032s  strbigbal   0.173s  str   1.092s  strbig    1.249s
     16000  int   4.332s  strbal   0.070s  strbigbal   0.346s  str   4.338s  strbig    4.852s
     32000  int  18.897s  strbal   0.112s  strbigbal   0.662s  str  18.212s  strbig   18.889s
     64000  int  82.845s  strbal   0.224s  strbigbal   1.292s  str  84.138s  strbig   83.718s
    128000  int 382.780s  strbal   0.425s  strbigbal   2.588s  str 381.781s  strbig  392.788s

386

go tool compile

      1000  int   0.014s  strbal   0.014s  strbigbal   0.039s  str   0.024s  strbig    0.034s
      2000  int   0.013s  strbal   0.015s  strbigbal   0.067s  str   0.022s  strbig    0.050s
      4000  int   0.016s  strbal   0.018s  strbigbal   0.118s  str   0.035s  strbig    0.097s
      8000  int   0.018s  strbal   0.026s  strbigbal   0.219s  str   0.054s  strbig    0.162s
     16000  int   0.023s  strbal   0.039s  strbigbal   0.455s  str   0.092s  strbig    0.311s
     32000  int   0.039s  strbal   0.071s  strbigbal   0.971s  str   0.195s  strbig    0.641s
     64000  int   0.061s  strbal   0.131s  strbigbal   2.024s  str   0.352s  strbig    1.247s
    128000  int   0.106s  strbal   0.253s  strbigbal   3.994s  str   stack!  strbig    2.495s
    256000  int   0.231s  strbal   0.501s  strbigbal     mem!   str  stack!  strbig    4.944s

gotype

      1000  int   0.024s  strbal   0.005s  strbigbal   0.020s  str   0.008s  strbig    0.287s
      2000  int   0.014s  strbal   0.007s  strbigbal   0.039s  str   0.014s  strbig    1.115s
      4000  int   0.033s  strbal   0.010s  strbigbal   0.072s  str   0.022s  strbig      mem! 
      8000  int   0.074s  strbal   0.016s  strbigbal   0.142s  str   0.040s
     16000  int   0.208s  strbal   0.033s  strbigbal   0.290s  str   0.091s
     32000  int   0.608s  strbal   0.060s  strbigbal   0.616s  str   0.188s
     64000  int   1.843s  strbal   0.131s  strbigbal   1.222s  str   0.374s
    128000  int  stack!   strbal   0.272s  strbigbal   2.585s  str   stack! 
    256000  int  stack!   strbal   0.640s  strbigbal     mem!   str  stack! 

go vet

      1000  int   0.098s  strbal   0.073s  strbigbal   0.135s  str   0.078s  strbig    0.396s
      2000  int   0.081s  strbal   0.073s  strbigbal   0.203s  str   0.077s  strbig    1.268s
      4000  int   0.097s  strbal   0.081s  strbigbal   0.340s  str   0.093s  strbig      mem! 
      8000  int   0.147s  strbal   0.093s  strbigbal   0.623s  str   0.134s
     16000  int   0.278s  strbal   0.130s  strbigbal   1.224s  str   0.230s
     32000  int   0.712s  strbal   0.219s  strbigbal   2.398s  str   0.452s
     64000  int   2.027s  strbal   0.389s  strbigbal   4.815s  str   0.863s
    128000  int  stack!   strbal   0.724s  strbigbal   9.902s  str   stack! 
    256000  int  stack!   strbal   1.514s  strbigbal     mem!  str   stack! 

gofmt

      1000  int   0.033s  strbal   0.006s  strbigbal   0.034s  str   0.021s  strbig    0.048s
      2000  int   0.067s  strbal   0.010s  strbigbal   0.068s  str   0.070s  strbig    0.125s
      4000  int   0.244s  strbal   0.019s  strbigbal   0.119s  str   0.223s  strbig    0.327s
      8000  int   0.854s  strbal   0.031s  strbigbal   0.238s  str   0.863s  strbig    1.101s
     16000  int   3.519s  strbal   0.062s  strbigbal   0.490s  str   3.417s  strbig    3.892s
     32000  int  13.626s  strbal   0.117s  strbigbal   0.932s  str  13.617s  strbig   14.544s
     64000  int  55.858s  strbal   0.232s  strbigbal   1.844s  str  56.563s  strbig   57.930s
    128000  int 252.079s  strbal   0.493s  strbigbal   3.724s  str 250.775s  strbig  255.842s
    256000  int  stack!   strbal   0.940s  strbigbal   7.614s  str   stack!