bazel: Previous failing test masks correct result in remote cache

ATTENTION! Please read and follow:

  • if this is a question about how to build / test / query / deploy using Bazel, or a discussion starter, send it to bazel-discuss@googlegroups.com
  • if this is a bug or feature request, fill the form below as best as you can.

Description of the problem / feature request:

Assume we have two branches of a git repository. master with passing unit tests and a unit_test_failure branch with failing unit tests. I have disk_cache enabled. I run:

$ git checkout master $ bazel test //… -> unit tests pass. No cache hit. $ bazel test //… -> unit tests pass. Cache hit. $ git checkout unit_test_failure $ bazel test //… -> unit tests fail $ git checkout master $ bazel test //… -> unit tests pass. No cache hit.

If instead, I add “rm -rf bazel-out/k8-fastbuild/testlogs/” right before running the last “bazel test” on master (or “bazel clean”), then there will be a remote cache hit:

$ git checkout master $ bazel test //… -> unit tests pass. No cache hit. $ bazel test //… -> unit tests pass. Cache hit. $ git checkout unit_test_failure $ bazel test //… -> unit tests fail $ git checkout master

Run “bazel clean” or “rm -rf bazel-out/k8-fastbuild/testlogs/”

$ rm -rf bazel-out/k8-fastbuild/testlogs/ $ bazel test //… -> unit tests pass. Cache hit.

I would expect a remote cache hit also in the first case. But this does not happen.

I suppose this is a bug. If not, please let me know what I’m doing wrong.

Feature requests: what underlying problem are you trying to solve with this feature?

Replace this line with your answer.

Bugs: what’s the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

See in Description. Please use my repo on github: https://github.com/gergelyfabian/bazel-scala-example. The example branches I used in this description (master and unit_test_failure) are there and testable.

What operating system are you running Bazel on?

Ubuntu 18.04.

What’s the output of bazel info release?

INFO: Invocation ID: a2999d7d-0b72-4f0f-b42d-9590c7ea672f release 2.2.0

Tested on Bazel 2.0.0 and 2.2.0.

If bazel info release returns “development version” or “(@non-git)”, tell us how you built Bazel.

Replace this line with your answer.

What’s the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

git@github.com:gergelyfabian/bazel-scala-example.git e1e4ae4da5a3d8c18a68e1c8c6baddb34d187659 9b4a64e1be19b28ff73941446d6729cd98dcd8cc

Have you found anything relevant by searching the web?

There is another issue that seems similar: #9389.

Any other information, logs, or outputs that you want to share?

Case reproducing the bug:

$ git checkout master Already on ‘master’ Your branch is up to date with ‘origin/master’. $ bazel test //… INFO: Invocation ID: 92696c4f-639f-4e5b-87b7-f22adcfd8d3a INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured). INFO: Found 3 targets and 2 test targets… INFO: Elapsed time: 0.167s, Critical Path: 0.00s INFO: 0 processes. INFO: Build completed successfully, 1 total action //example-lib:test (cached) PASSED in 0.0s //example-maven:test (cached) PASSED in 0.0s

Executed 0 out of 2 tests: 2 tests pass. INFO: Build completed successfully, 1 total action $ git checkout unit_test_failure
Switched to branch ‘unit_test_failure’ $ bazel test //… INFO: Invocation ID: 1eef1f1d-7df4-4d3d-ac92-97e638fe17e9 INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured). INFO: Found 3 targets and 2 test targets… FAIL: //example-maven:test (see /home/user/.cache/bazel/_bazel_user/d4795b862c3f0b5e82d41ecc9ec237d2/execroot/scala_example/bazel-out/k8-fastbuild/testlogs/example-maven/test/test.log) INFO: Elapsed time: 1.233s, Critical Path: 1.07s INFO: 2 processes: 1 remote cache hit, 1 linux-sandbox. INFO: Build completed, 1 test FAILED, 3 total actions //example-lib:test (cached) PASSED in 0.0s //example-maven:test FAILED in 1.0s /home/user/.cache/bazel/_bazel_user/d4795b862c3f0b5e82d41ecc9ec237d2/execroot/scala_example/bazel-out/k8-fastbuild/testlogs/example-maven/test/test.log

Executed 1 out of 2 tests: 1 test passes and 1 fails locally. INFO: Build completed, 1 test FAILED, 3 total actions $ git checkout master Switched to branch ‘master’ Your branch is up to date with ‘origin/master’. $ bazel test //… INFO: Invocation ID: 3a21e211-1902-4433-b344-9f04b3d83558 INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured). INFO: Found 3 targets and 2 test targets… INFO: Elapsed time: 1.240s, Critical Path: 1.07s INFO: 2 processes: 1 remote cache hit, 1 linux-sandbox. INFO: Build completed successfully, 3 total actions //example-lib:test (cached) PASSED in 0.0s //example-maven:test PASSED in 1.0s

Executed 1 out of 2 tests: 2 tests pass. INFO: Build completed successfully, 3 total actions


Forcing remote cache hit:

$ git checkout master Already on ‘master’ Your branch is up to date with ‘origin/master’. $ $ bazel test //… INFO: Invocation ID: 71339ed1-128c-412c-a6cd-6e812b38a880 INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured). INFO: Found 3 targets and 2 test targets… INFO: Elapsed time: 0.234s, Critical Path: 0.01s INFO: 0 processes. INFO: Build completed successfully, 1 total action //example-lib:test (cached) PASSED in 0.0s //example-maven:test (cached) PASSED in 1.0s

Executed 0 out of 2 tests: 2 tests pass. INFO: Build completed successfully, 1 total action $ git checkout unit_test_failure
Switched to branch ‘unit_test_failure’ $ bazel test //… INFO: Invocation ID: ec056e85-d024-4072-916b-d89fd017b278 INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured). INFO: Found 3 targets and 2 test targets… FAIL: //example-maven:test (see /home/user/.cache/bazel/_bazel_user/d4795b862c3f0b5e82d41ecc9ec237d2/execroot/scala_example/bazel-out/k8-fastbuild/testlogs/example-maven/test/test.log) INFO: Elapsed time: 1.285s, Critical Path: 1.04s INFO: 2 processes: 1 remote cache hit, 1 linux-sandbox. INFO: Build completed, 1 test FAILED, 3 total actions //example-lib:test (cached) PASSED in 0.0s //example-maven:test FAILED in 1.0s /home/user/.cache/bazel/_bazel_user/d4795b862c3f0b5e82d41ecc9ec237d2/execroot/scala_example/bazel-out/k8-fastbuild/testlogs/example-maven/test/test.log

Executed 1 out of 2 tests: 1 test passes and 1 fails locally. INFO: Build completed, 1 test FAILED, 3 total actions $ git checkout master Switched to branch ‘master’ Your branch is up to date with ‘origin/master’. $ bazel clean INFO: Invocation ID: 0cbb6d3d-a42d-41c2-9ece-dde8feb7d795 INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes. $ bazel test //… INFO: Invocation ID: afbf5193-0f6a-4b0e-8b6d-a301aee6ba11 INFO: Analyzed 5 targets (58 packages loaded, 1299 targets configured). INFO: Found 3 targets and 2 test targets… INFO: From Building external/com_google_protobuf/libprotobuf_java.jar (123 source files, 1 source jar): warning: -parameters is not supported for target value 1.7. Use 1.8 or later. INFO: Elapsed time: 5.215s, Critical Path: 2.67s INFO: 212 processes: 212 remote cache hit. INFO: Build completed successfully, 245 total actions //example-lib:test (cached) PASSED in 0.0s //example-maven:test (cached) PASSED in 0.0s

Executed 0 out of 2 tests: 2 tests pass. INFO: Build completed successfully, 245 total actions

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 18 (12 by maintainers)

Most upvoted comments

Thanks for reporting this! I’ll look into the cause.