go: x/build: freebsd-arm-paulzhol builder is failing cmd/link TestDWARF

https://build.golang.org/log/d0025d71cbb3628b7800d2eebe1d8fb58c79969d

ok  	cmd/internal/src	0.034s
ok  	cmd/internal/test2json	1.729s
--- FAIL: TestDWARF (4.86s)
    dwarf_test.go:41: cmd/link is stale - run go install cmd/link
FAIL
FAIL	cmd/link	56.431s
FAIL
go tool dist: Failed: exit status 1

I’ve captured the stdout/stderr output from the buildlet to look at the env. variables as I understand it was related to clearing GOROOT_FINAL in similar issues.

2020/08/28 11:21:06 [0x314ba580] Running /tmp/workdir-host-freebsd-arm-paulzhol/go/bin/go with args ["/tmp/workdir-host-freebsd-arm-paulzhol/go/bin/go" "tool" "dist" "test" "--no-rebuild" "--banner=XXXBANNERXXX:" "go_test:cmd/internal/obj/riscv" "go_test:cmd/internal/obj/x86" "go_test:cmd/internal/objabi"] and env ["BLOCKSIZE=K" "MAIL=/var/mail/paulzhol" "PATH=/tmp/workdir-host-freebsd-arm-paulzhol/go/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/home/paulzhol/bin" "GO_TEST_TIMEOUT_SCALE=30" "CGO_ENABLED=1" "GOARM=7" "XDG_CACHE_HOME=/tmp/cache" "USER=paulzhol" "HOME=/home/paulzhol" "PWD=/" "RC_PID=21" "WORKDIR=/tmp/workdir-host-freebsd-arm-paulzhol" "GOROOT_BOOTSTRAP=/tmp/workdir-host-freebsd-arm-paulzhol/go1.4" "GO_BUILDER_NAME=freebsd-arm-paulzhol" "GOROOT_BOOTSTRAP=/usr/home/paulzhol/go1.4" "GOARM=7" "CGO_ENABLED=1" "GOROOT=/tmp/workdir-host-freebsd-arm-paulzhol/go" "GOPATH=/tmp/workdir-host-freebsd-arm-paulzhol/gopath" "GOPROXY=http://10.240.0.3:30157" "GOPROXY=off" "TMPDIR=/tmp/workdir-host-freebsd-arm-paulzhol/tmp" "GOCACHE=/tmp/workdir-host-freebsd-arm-paulzhol/gocache"] in dir /tmp/workdir-host-freebsd-arm-paulzhol
2020/08/28 11:22:09 [0x314ba580] Run = ok, after 1m3.510284157s

2020/08/28 11:22:09 [0x314bb1e0] Running /tmp/workdir-host-freebsd-arm-paulzhol/go/bin/go with args ["/tmp/workdir-host-freebsd-arm-paulzhol/go/bin/go" "tool" "dist" "test" "--no-rebuild" "--banner=XXXBANNERXXX:" "go_test:cmd/internal/src" "go_test:cmd/internal/test2json" "go_test:cmd/link"] and env ["BLOCKSIZE=K" "MAIL=/var/mail/paulzhol" "PATH=/tmp/workdir-host-freebsd-arm-paulzhol/go/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/home/paulzhol/bin" "GO_TEST_TIMEOUT_SCALE=30" "CGO_ENABLED=1" "GOARM=7" "XDG_CACHE_HOME=/tmp/cache" "USER=paulzhol" "HOME=/home/paulzhol" "PWD=/" "RC_PID=21" "WORKDIR=/tmp/workdir-host-freebsd-arm-paulzhol" "GOROOT_BOOTSTRAP=/tmp/workdir-host-freebsd-arm-paulzhol/go1.4" "GO_BUILDER_NAME=freebsd-arm-paulzhol" "GOROOT_BOOTSTRAP=/usr/home/paulzhol/go1.4" "GOARM=7" "CGO_ENABLED=1" "GOROOT=/tmp/workdir-host-freebsd-arm-paulzhol/go" "GOPATH=/tmp/workdir-host-freebsd-arm-paulzhol/gopath" "GOPROXY=http://10.240.0.3:30157" "GOPROXY=off" "TMPDIR=/tmp/workdir-host-freebsd-arm-paulzhol/tmp" "GOCACHE=/tmp/workdir-host-freebsd-arm-paulzhol/gocache"] in dir /tmp/workdir-host-freebsd-arm-paulzhol
2020/08/28 11:23:49 [0x314bb1e0] Run = exit status 1, after 1m39.66510577s

2020/08/28 11:23:49 [0x3148cdc0] Running /tmp/workdir-host-freebsd-arm-paulzhol/go/bin/go with args ["/tmp/workdir-host-freebsd-arm-paulzhol/go/bin/go" "tool" "dist" "test" "--no-rebuild" "--banner=XXXBANNERXXX:" "go_test:cmd/link/internal/benchmark" "go_test:cmd/link/internal/ld" "go_test:cmd/link/internal/loader"] and env ["BLOCKSIZE=K" "MAIL=/var/mail/paulzhol" "PATH=/tmp/workdir-host-freebsd-arm-paulzhol/go/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/home/paulzhol/bin" "GO_TEST_TIMEOUT_SCALE=30" "CGO_ENABLED=1" "GOARM=7" "XDG_CACHE_HOME=/tmp/cache" "USER=paulzhol" "HOME=/home/paulzhol" "PWD=/" "RC_PID=21" "WORKDIR=/tmp/workdir-host-freebsd-arm-paulzhol" "GOROOT_BOOTSTRAP=/tmp/workdir-host-freebsd-arm-paulzhol/go1.4" "GO_BUILDER_NAME=freebsd-arm-paulzhol" "GOROOT_BOOTSTRAP=/usr/home/paulzhol/go1.4" "GOARM=7" "CGO_ENABLED=1" "GOROOT=/tmp/workdir-host-freebsd-arm-paulzhol/go" "GOPATH=/tmp/workdir-host-freebsd-arm-paulzhol/gopath" "GOPROXY=http://10.240.0.3:30157" "GOPROXY=off" "TMPDIR=/tmp/workdir-host-freebsd-arm-paulzhol/tmp" "GOCACHE=/tmp/workdir-host-freebsd-arm-paulzhol/gocache"] in dir /tmp/workdir-host-freebsd-arm-paulzhol
2020/08/28 11:23:50 Halting in 1 second.

Maybe this is a dup of #33598 but on FreeBSD-arm.

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 1
Comments: 15 (15 by maintainers)

Most upvoted comments

With @bcmills 's CL https://go-review.googlesource.com/c/go/+/322629 , it is passing now (staleness check removed).

I still think it is a good idea to make GOROOT and GOROOT_FINAL settings consistent for the builder.

cherrymui on May 27, 2021

When building the toolchain, cmd/dist sets GOROOT_FINAL to GOROOT, which is set by make.bash. In our case, it is /var/tmp/workdir-host-freebsd-arm-paulzhol/go.

When running the test, the coordinator sets GOROOT to the tree root. In our case, it is /tmp/workdir-host-freebsd-arm-paulzhol/go (note: without /var).

On gomote,

$ gomote run $VM go/bin/go list -f '{{.Stale}}' cmd/link
false
$ gomote run -e GOROOT= $VM go/bin/go list -f '{{.Stale}}' cmd/link
false
$ gomote run -e GOROOT=/var/tmp/workdir-host-freebsd-arm-paulzhol/go $VM go/bin/go list -f '{{.Stale}}' cmd/link
false
$ gomote run -e GOROOT=/tmp/workdir-host-freebsd-arm-paulzhol/go $VM go/bin/go list -f '{{.Stale}}' cmd/link
true

When GOROOT is not set or set to the same path (as go env GOROOT), it reports cmd/link not stale. When GOROOT is set to a different (although equivalent) path, it reports cmd/link stale. I think that’s the cause.

I wonder why the coordinator sets GOROOT when running the tests? Is it necessary?

(Perhaps the go command’s staleness check can be made more resilient to this kind of equivalent but not lexically identical paths? Or maybe not worth it.)

cherrymui on Dec 23, 2020