bazel: --bes_upload_mode=fully_async causes local BEP files to be written async too

Description of the problem / feature request:

When using --bes_upload_mode=fully_async, Bazel uploads BES asynchronously, so your bazel build command can exit, but the upload still continues in the background. This is nice for performance.

However, it seems to also cause local BEP files, e.g. --build_event_text_file or --build_event_json_file to be written asynchronously. This means that if you bazel build and save the BEP to a file, then immediately inspect the BEP, it might not be fully written yet.

Bugs: what’s the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

bazel clean
bazel build --nobuild --build_event_text_file=/tmp/bep.txt //...
grep targets_configured /tmp/bep.txt

The grep then often doesn’t return anything, because the targets_configured thing is written at the end of the BEP file. If you add a sleep 3 before it, it usually works.

What operating system are you running Bazel on?

Linux

What’s the output of bazel info release?

release 1.2.1

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 15 (13 by maintainers)

Most upvoted comments

We could probably make the URI in the local BEP files pointing to local files instead so there is no network traffic for generating local BEP files and can be blocked waiting after the build regardless --bes_upload_mode.

We would actually like that as an option 😍. BEP having local URI and BES having remote cache URI.

No, I meant that with --build_event_json_file_path_conversion=false it looked like it is completely written to disk. That code seems to confirm it as well. I think this can be closed with this workaround.

What is the expected behavior in this case? It seems to me that having local BEP files written completely before exiting Bazel contradicts to fully_async mode.

That is an excellent question for @coeuvre! 😄