go: cmd/link: invalid TLS access model used with -buildmode c-shared

What version of Go are you using (go version)?

$ link -V
link version go1.18

Does this issue reproduce with the latest release?

Yes, 1.18 is the latest release.

What operating system and processor architecture are you using (go env)?

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/olo/.cache/go-build"
GOENV="/home/olo/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/olo/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/olo/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/golang"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3022165828=/tmp/go-build -gno-record-gcc-switches"

What did you do?

We have a CGo-based library that we build in c-shared mode to use as a dynamic shared object on Linux (x86_64).

The library is, unfortunately, an internal project and I cannot disclose the source code.

For the C compiler, we are using Clang.

The final linking of the shared object file is performed with the following invocation of the go link tool (some irrelevant flags removed):

go link -s -X internalmodule/go/buildinfo.Compiler=clang -o shared_library.so -buildmode c-shared -buildid= -linkmode external -L importdir -extld /path/to/external_clang++_ld  '-extldflags=A BUNCH OF EXT LD FLAGS' package.a

What did you expect to see?

All this setup has been working fine with Clang’s previous version, 9.0.20190721 and the exact same (binary for binary) Golang toolchain version.

We would expect it to work the same after upgrading to a newer Clang version.

What did you see instead?

With a newer Clang version 12.0.20210610, we are observing cmd/link fail with the following error:

ld.lld: error: relocation R_X86_64_TPOFF32 against runtime.tlsg cannot be used with -shared
>>> defined in /tmp/go-link-4046235013/go.o
>>> referenced by cpu_x86.go:65 (/usr/local/go/src/internal/cpu/cpu_x86.go:65)
>>>               /tmp/go-link-4046235013/go.o:(internal/cpu.doinit)

The first suspicion would obviously be on Clang. However, we tracked it down to this change: https://reviews.llvm.org/D33100

As can be seen, Clang has introduced a stricter check on where this relocation is allowed due to correctness reasons.

Further reading of https://akkadia.org/drepper/tls.pdf indicates that this relocation type is intended for the TLS Local Exec model, and exec models are not suitable for usage in dynamic shared objects (only in final built executables).

We’ve analyzed all the intermediate/dependency .a archive files and the .o object files they contain with readelf -r, and none of them contain the R_X86_64_TPOFF32 relocation anywhere.

The temporary go.o file generated by the go link command is the first that appears during the build process that contains it, indicating that it is this command inserting the relocation.

We’ve tracked down the code in cmd/link that generates this relocation to this place: https://github.com/golang/go/blob/go1.18/src/cmd/link/internal/amd64/asm.go#L403

which further confirms that cmd/link is using an invalid TLS access model for the situation (-buildmode c-shared was used).

I do not yet have a good enough understanding of the Go linker and why is it choosing this access model, but its behavior seems to be clearly incorrect.

It’s possible that, as the latest Clang releases get more prevalent, this issue will pop up in other places with other users who rely on -buildmode c-shared.

This issue might also have some relation to issue #48596 and bears some similarity to the old (and fixed) issue #9652.