go: cmd/link: internal linker fails to build openshift/installer with "program too large, call relocation distance" and segfaults on linux/arm64
What version of Go are you using (go version)?
go version devel +85afa2eb19 Tue Jul 28 16:59:04 2020 +0000 linux/arm64 Also seen with go1.15rc1/1.14.6/1.13.14
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
linux/arm64
What did you do?
git clone https://github.com/openshift/installer pushd installer go build -mod=vendor -o bin/openshift-install ./cmd/openshift-install
What did you expect to see?
Build succeed.
What did you see instead?
For the record following is with the master branch go and gcc-10.1.1, gibc-2.31 and binutils-2.34 on (Fedora 32) host.
# go build -mod=vendor -o bin/openshift-install ./cmd/openshift-install
# github.com/openshift/installer/cmd/openshift-install
net(.text): program too large, call relocation distance = 146365316
net(.text): program too large, call relocation distance = 146365296
net(.text): program too large, call relocation distance = 146365196
net(.text): program too large, call relocation distance = 146365108
net(.text): program too large, call relocation distance = 146365120
net(.text): program too large, call relocation distance = 146365084
net(.text): program too large, call relocation distance = 146365056
net(.text): program too large, call relocation distance = 146364940
runtime/cgo(.text): program too large, call relocation distance = 146364852
runtime/cgo(.text): program too large, call relocation distance = 146364808
runtime/cgo(.text): program too large, call relocation distance = 146364812
runtime/cgo(.text): program too large, call relocation distance = 146364824
runtime/cgo(.text): program too large, call relocation distance = 146364804
runtime/cgo(.text): program too large, call relocation distance = 146364792
runtime/cgo(.text): program too large, call relocation distance = 146364788
runtime/cgo(.text): program too large, call relocation distance = 146364680
runtime/cgo(.text): program too large, call relocation distance = 146364712
runtime/cgo(.text): program too large, call relocation distance = 146364680
runtime/cgo(.text): program too large, call relocation distance = 146364612
runtime/cgo(.text): program too large, call relocation distance = 146364624
runtime/cgo(.text): program too large, call relocation distance = 146364564
/root/upstream/go/pkg/tool/linux_arm64/link: too many errors
unexpected fault address 0xffff03aca4ca
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0xffff03aca4ca pc=0xe10b0]
goroutine 305 [running]:
runtime.throw(0x29ff4c, 0x5)
/root/upstream/go/src/runtime/panic.go:1116 +0x54 fp=0x4000037540 sp=0x4000037510 pc=0x41594
runtime.sigpanic()
/root/upstream/go/src/runtime/signal_unix.go:727 +0x3b8 fp=0x4000037570 sp=0x4000037540 pc=0x577d8
encoding/binary.littleEndian.PutUint32(...)
/root/upstream/go/src/encoding/binary/binary.go:73
encoding/binary.(*littleEndian).PutUint32(0x489780, 0xffff03aca4ca, 0x4, 0xe6a7302, 0x126c)
<autogenerated>:1 +0x40 fp=0x40000375a0 sp=0x4000037580 pc=0xe10b0
cmd/link/internal/ld.relocsym2(0x4000071880, 0x40007c2000, 0x4000071890, 0x40000718d8, 0x401fcbc480)
/root/upstream/go/src/cmd/link/internal/ld/data2.go:481 +0x620 fp=0x4000037750 sp=0x40000375a0 pc=0x171db0
cmd/link/internal/ld.(*Link).reloc2.func2(0x4000071880, 0x4000071880, 0x40007c2000, 0x4000071890, 0x40000718d8, 0x403bcb66f0)
/root/upstream/go/src/cmd/link/internal/ld/data2.go:505 +0x68 fp=0x40000377a0 sp=0x4000037750 pc=0x1cd128
runtime.goexit()
/root/upstream/go/src/runtime/asm_arm64.s:1136 +0x4 fp=0x40000377a0 sp=0x40000377a0 pc=0x726b4
created by cmd/link/internal/ld.(*Link).reloc2
/root/upstream/go/src/cmd/link/internal/ld/data2.go:503 +0xe0
goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x403bcb66f8)
/root/upstream/go/src/runtime/sema.go:56 +0x38
sync.(*WaitGroup).Wait(0x403bcb66f0)
/root/upstream/go/src/sync/waitgroup.go:130 +0x6c
cmd/link/internal/ld.(*Link).reloc2(0x4000071880)
/root/upstream/go/src/cmd/link/internal/ld/data2.go:517 +0x12c
cmd/link/internal/ld.Main(0x448740, 0x10, 0x20, 0x1, 0x1f, 0x1e, 0x2a77c4, 0x14, 0x2aaf46, 0x1a, ...)
/root/upstream/go/src/cmd/link/internal/ld/main.go:346 +0x1228
main.main()
/root/upstream/go/src/cmd/link/main.go:68 +0x188
goroutine 80 [syscall]:
syscall.Syscall(0xd7, 0xfffefaed8000, 0x172997cc, 0x0, 0x40006dca28, 0x56c8d05f, 0xec02fd3640213e57)
/root/upstream/go/src/syscall/asm_linux_arm64.s:9 +0x10
syscall.munmap(0xfffefaed8000, 0x172997cc, 0xffff121717cb, 0x4004e85ce8)
/root/upstream/go/src/syscall/zsyscall_linux_arm64.go:957 +0x3c
syscall.(*mmapper).Munmap(0x4447a0, 0xfffefaed8000, 0x172997cc, 0x172997cc, 0x0, 0x0)
/root/upstream/go/src/syscall/syscall_unix.go:94 +0x148
syscall.Munmap(...)
/root/upstream/go/src/syscall/syscall_linux.go:1003
cmd/link/internal/ld.(*OutBuf).munmap(0x4000136070)
/root/upstream/go/src/cmd/link/internal/ld/outbuf_mmap.go:39 +0x54
cmd/link/internal/ld.(*OutBuf).copyHeap(0x4000136070, 0x403941a000)
/root/upstream/go/src/cmd/link/internal/ld/outbuf.go:152 +0x54
cmd/link/internal/ld.(*OutBuf).Close(0x4000136070, 0x1c3ea8, 0x40317b4270)
/root/upstream/go/src/cmd/link/internal/ld/outbuf.go:116 +0x124
cmd/link/internal/ld.linknew.func1()
/root/upstream/go/src/cmd/link/internal/ld/sym.go:61 +0x4c
cmd/link/internal/ld.runAtExitFuncs()
/root/upstream/go/src/cmd/link/internal/ld/util.go:24 +0x40
cmd/link/internal/ld.Exit(0x2)
/root/upstream/go/src/cmd/link/internal/ld/util.go:31 +0x20
cmd/link/internal/ld.Exitf(0x2a4c73, 0xf, 0x0, 0x0, 0x0)
/root/upstream/go/src/cmd/link/internal/ld/util.go:39 +0x100
cmd/link/internal/ld.afterErrorAction(...)
/root/upstream/go/src/cmd/link/internal/ld/util.go:50
cmd/link/internal/ld.Errorf(0x403f1b4280, 0x2b350e, 0x30, 0x40006dcd88, 0x1, 0x1)
/root/upstream/go/src/cmd/link/internal/ld/util.go:66 +0x100
cmd/link/internal/arm64.archreloc(0x4000071880, 0x40000718d8, 0x4077122560, 0x403f1b4280, 0x94000000, 0x910da273, 0x1)
/root/upstream/go/src/cmd/link/internal/arm64/asm.go:634 +0x4dc
cmd/link/internal/ld.relocsym2(0x4000071880, 0x40007c2000, 0x4000071890, 0x40000718d8, 0x403f1b4280)
/root/upstream/go/src/cmd/link/internal/ld/data2.go:147 +0xe38
cmd/link/internal/ld.(*Link).reloc2.func1(0x4000071880, 0x4000071880, 0x40007c2000, 0x4000071890, 0x40000718d8, 0x403bcb66f0)
/root/upstream/go/src/cmd/link/internal/ld/data2.go:498 +0x78
created by cmd/link/internal/ld.(*Link).reloc2
/root/upstream/go/src/cmd/link/internal/ld/data2.go:495 +0x9c
Using linkmode=external seems to work around the issue. This reminds me of the similar issues with the external linker(on non-amd64 architectures) in the past https://github.com/golang/go/issues/17028 . I’m bit surprised that this didn’t got hit by any other “big binary” projects as kube, openshift,…
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 27 (19 by maintainers)
Commits related to this issue
- Workaround golang linux/arm64 link error There is a bug with the golang internal linker for the linux/arm64 target that has yet to be fixed: https://github.com/golang/go/issues/40492 — committed to yselkowitz/openshift-installer by yselkowitz 4 years ago
- Workaround golang linux/arm64 link error There is a bug with the golang internal linker for the linux/arm64 target that has yet to be fixed: https://github.com/golang/go/issues/40492 — committed to yselkowitz/openshift-installer by yselkowitz 4 years ago
- Workaround golang linux/arm64 link error There is a bug with the golang internal linker for the linux/arm64 target that has yet to be fixed: https://github.com/golang/go/issues/40492 — committed to vrutkovs/installer by yselkowitz 4 years ago
- cmd/link: support trampoline insertion on ARM64 Compared to ARM32 or PPC64, ARM64 has larger range for direct jumps. But for very large programs it can still go over the limit. Add trampoline inserti... — committed to golang/go by cherrymui 3 years ago
- cmd/link: support trampoline insertion for PLT calls on ARM64 When internal linking with C objects, some C object relocations may be turned into a CALL via PLT. For very large programs, the PLT stub ... — committed to golang/go by cherrymui 3 years ago
- cmd/link: test trampolines with cgo Updates #40492, #30949. Change-Id: I6d7923ac83275c5ab08958f7a501f7975aea151a Reviewed-on: https://go-review.googlesource.com/c/go/+/314456 Trust: Cherry Zhang <ch... — committed to golang/go by cherrymui 3 years ago
- cmd/link: split large text sections on Darwin/ARM64 when external linking The Darwin linker does not like text sections that are larger than the jump limit (even if we already inserted trampolines). ... — committed to golang/go by cherrymui 3 years ago
I submitted a few CLs related to this. Now it should work on Linux/ARM64 for both internal and external linking, with cgo enabled or disabled.
On Darwin/ARM64 internal linking should work. Cgo can be enabled or disabled. External linking with cgo on still needs more work (the system linker is very picky).
(Note: when building natively, cgo is by default enabled, and external linking is used by default. When cross compile, cgo is disabled by default and internal linking is by default used. It can be overridden by CGO_ENABELD=0/1 and -ldflags=-linkmode=internal/external.)
This issue is originally about internal linking, which now should work on both Linux and Darwin. I’m going to call this fixed. If external linking on Darwin is a concern we can open a separate issue. Thanks.
I plan to do this in 1.17. But if you get ahead of me, feel free to send a CL. Thanks.
Yes, no work is done for this in Go 1.16. My plan is to do this in 1.17.
Note that this was also present in 1.14 and 1.13, so this isn’t a regression. I’ve tentatively put this in the 1.15 milestone, but we probably won’t fix this for the initial release (it may make a point release).