bazel: Tests with "exclusive" tag do not get results from remote cache
Description of the problem / feature request / question:
I am trying out the remote cache functionality described here. Currently I’m running against a local hazelcast instance. This works great most of the time. One thing I noticed is that tests with the exclusive
tag, despite being cached as normal in the local cache, will not seem to use the remote cache and will always rerun if there is no passing result in the local cache. This seems like an oversight?
If possible, provide a minimal example to reproduce the problem:
BUILD
cc_test(
name = "test",
srcs = ["test.cc"],
tags = ["exclusive"], # comment this out to fix the problem
)
test.cc
#include <chrono>
#include <thread>
int main() {
std::this_thread::sleep_for(std::chrono::seconds(10));
return 0;
}
.bazelrc
startup --host_jvm_args=-Dbazel.DigestFunction=SHA1
build --spawn_strategy=remote
build --experimental_strict_action_env
build --remote_rest_cache=http://localhost:5701/hazelcast/rest/maps/cache
test --spawn_strategy=remote
test --experimental_strict_action_env
test --remote_rest_cache=http://localhost:5701/hazelcast/rest/maps/cache
Environment info
-
Operating System: Ubuntu
-
Bazel version (output of
bazel info release
): release 0.5.3 -
Hazelcast version 3.8.6
Anything else, information or logs or outputs that would be helpful?
Here’s an example run. The second test will pass in 0.0s if either the exclusive
tag is removed, or the second bazel clean
command is omitted.
$ bazel clean && bazel test //... && bazel clean && bazel test //...
INFO: Reading 'startup' options from /home/stu/tmp/.bazelrc: --host_jvm_args=-Dbazel.DigestFunction=SHA1
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
INFO: Reading 'startup' options from /home/stu/tmp/.bazelrc: --host_jvm_args=-Dbazel.DigestFunction=SHA1
INFO: Analysed target //:test (9 packages loaded).
INFO: Found 1 test target...
Target //:test up-to-date:
bazel-bin/test
INFO: Elapsed time: 10.405s, Critical Path: 10.20s
INFO: Build completed successfully, 7 total actions
//:test PASSED in 10.0s
Executed 1 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
INFO: Reading 'startup' options from /home/stu/tmp/.bazelrc: --host_jvm_args=-Dbazel.DigestFunction=SHA1
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
INFO: Reading 'startup' options from /home/stu/tmp/.bazelrc: --host_jvm_args=-Dbazel.DigestFunction=SHA1
INFO: Analysed target //:test (9 packages loaded).
INFO: Found 1 test target...
Target //:test up-to-date:
bazel-bin/test
INFO: Elapsed time: 10.209s, Critical Path: 10.02s
INFO: Build completed successfully, 7 total actions
//:test PASSED in 10.0s
Executed 1 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 6
- Comments: 24 (14 by maintainers)
Commits related to this issue
- ref_tests: switch to cpu:4 from exclusive (#6312) This tells Bazel that the test requires 4 CPUs to run. On enormous machines, tests will run in parallel, on laptops tests will run serially or maybe... — committed to batfish/batfish by dhalperi 4 years ago
- Flip incompatible_exclusive_test_sandboxed Solves https://github.com/bazelbuild/bazel/issues/3791 https://github.com/bazelbuild/bazel/issues/16871 https://github.com/bazel-contrib/SIG-rules-authors/i... — committed to bazelbuild/bazel by gasparev 2 years ago
Is there any timelines to get it fixed? If not how about introducing the flag to enable #8983 conditionally, in this case many teams could benefit from exclusive tests caching as well as keep internal tests green. Thoughts?
Here is a concrete use-case for why this bug is important: We have a lot of tests that require usage of the GPU. These tests use the exclusive tag to prevent OOM issues that can arise when running multiple tests that require the GPU at once. Because of this bug these tests have to run every single time that we do a test CI build.
In my opinion this bug breaks one of the core features of bazel: targets are being built which do not need to be built because the dependencies did not change.
#8983 is rolled forward with fix as 5e5eb86e5390e9e822848ccc031e0846a520858d:
--incompatible_exclusive_test_sandboxed
flag so users can enable this feature conditionally.It should be contained in release 4.0. You can track the release here #12455.
@RNabel yes. should be in 0.29.0
@gasparev Could you create a separate tracking issue as explained in the docs? You can then ping
@meteorcloudy
to have him add the necessary flags.Hi, I’m trying to fix this but I need to test the change on the downstream projects. From the doc, it seems that the labels
incompatible-change
andmigration-ready
are needed. Can someone add them?