bazel: Regression: Compilation Unnecessarily Restricted By Scheduler in Rolling Release

Hey awesome Team Bazel,

There seems to be a regression around parallelism. With the rolling releases, I’m seeing compilation restricted to 2 or 3 threads, with the rest (I think) held up by the scheduler.

Concretely, during build, I’m seeing output like the following, and moderately slower build times to match:

bazel build //:rule
INFO: ...<omitted>
[1 / 200] 8 actions, 2 running
    Compiling <...>; 5s darwin-sandbox
    Compiling <...>; 7s darwin-sandbox
    [Sched] Compiling <...>; 24s
    [Sched] Compiling <...>; 24s
    [Sched] Compiling <...>; 24s
    [Sched] Compiling <...>; 24s
    [Sched] Compiling <...>; 24s
    [Sched] Compiling <...>; 24s

[My interpretation is that the scheduler is having the jobs wait; please correct me if I’m wrong.]

Setting, e.g. --jobs=HOST_CPUS or --noexperimental_local_memory_estimate doesn’t have any impact. (Hence different from https://github.com/envoyproxy/envoy/issues/12455.) And linking/non-compiling jobs don’t seem to count against the lower cap.

With the latest normal release, 4.2.1 and all previous releases I’ve seen, all cores are utilized, I don’t see [Sched] and the build runs a bit faster. (But not 4x faster or anything, so the bottleneck is somewhere else–the disk perhaps?)

Version details: I’m seeing this on 5.0.0-pre.20210831.2 (and for all the rolling versions we’ve tried, which are just those after the release of 4.2). I’m using the latest macOS, 11.6, and I’m seeing it all the way back at least to 10.14.

Thanks so much, Chris (ex-Googler)

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 1
  • Comments: 16 (13 by maintainers)

Most upvoted comments

I fixed this in commit https://github.com/bazelbuild/bazel/commit/f5196e2ce4e8e45ca49c271053dad8dd32be5080

@cpsauer Could you please check that my fix resolved your issue?

To do this, you could use the bazelisk with last_green version.

One might investigate the impact of 4af342870cd5a777701a3188dffcbba82ae737a5.