rules_go: Significant slow down in build time

Using commit 9556bc88d7d240d3bcbf07d24282953eb1687ef3, the build slows down significantly with heavy disk io. Looks like the stdlib code is copying the entire go toolchain on every target.

https://github.com/bazelbuild/rules_go/commit/7c664db6ddf82a24bcc4c49a62e08b7bbbf18e01#diff-b1e7c11591fbac0890652d29a78d6222R93

Running inotifywait -mr . at ~/.cache/bazel/_bazel_user/sha along with bazel build ..., there are lots of outputs:

./bazel-sandbox/different_number/execroot/__main__/bazel-out/host/bin/external/go_stdlib_linux_amd64_cgo/

./bazel-sandbox/3175211375642440610/execroot/__main__/bazel-out/host/bin/external/go_stdlib_linux_amd64_cgo/src/cmd/go/testdata/importcom/src/wrongplace/ DELETE_SELF 
./bazel-sandbox/7145389296667911019/execroot/__main__/bazel-out/host/bin/external/go_stdlib_linux_amd64_cgo/src/runtime/ CREATE timestub.go
./bazel-sandbox/8657867472004421131/execroot/__main__/bazel-out/host/bin/external/go_stdlib_linux_amd64_cgo/src/cmd/go/testdata/src/vend/vendor/vend/dir1/ DELETE_SELF 
./bazel-sandbox/3175211375642440610/execroot/__main__/bazel-out/host/bin/external/go_stdlib_linux_amd64_cgo/src/cmd/go/testdata/importcom/src/ DELETE,ISDIR wrongplace
./bazel-sandbox/3144955954922670518/execroot/__main__/bazel-out/host/bin/external/go_stdlib_linux_amd64_cgo/src/sync/atomic/ CREATE asm_nacl_arm.s

./bazel-sandbox/3144955954922670518/execroot/__main__/bazel-out/host/bin/external/go_stdlib_linux_amd64_cgo/pkg/linux_amd64/math/ DELETE big.a
./bazel-sandbox/3144955954922670518/execroot/__main__/bazel-out/host/bin/external/go_stdlib_linux_amd64_cgo/pkg/linux_amd64/math/ DELETE cmplx.a
./bazel-sandbox/3144955954922670518/execroot/__main__/bazel-out/host/bin/external/go_stdlib_linux_amd64_cgo/pkg/linux_amd64/math/ DELETE rand.a
./bazel-sandbox/3144955954922670518/execroot/__main__/bazel-out/host/bin/external/go_stdlib_linux_amd64_cgo/pkg/linux_amd64/math/ DELETE_SELF 

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 20 (16 by maintainers)

Most upvoted comments

#1049 is now in, @ivyxyz can you test again? If it’s not better, can you produce a profile for me?

#1049 seems to restore us roughly to the “old” build times, yes.