bazel: `--incompatible_remote_build_event_upload_respect_no_cache` still uploads some `no-cache` outputs
Description of the bug:
Some test target outputs are still uploaded to the cache even when --incompatible_remote_build_event_upload_respect_no_cache and --modify_execution_info='.*=+no-remote' are set. rules_apple tests upload the binary and the zip, and swift_test has the runner uploaded (which it shouldn’t), but not the binary…
What’s the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
The following will repro the bug:
git clone https://github.com/buildbuddy-io/rules_xcodeproj.git
cd rules_xcodeproj
# Modify `examples/command_line/Tests/SwiftGreetingsTests.swift` adding something like
# `public let cache = "123"` and changing as needed to produce a new end output
bazel build --config=cache --modify_execution_info='.*=+no-remote' //examples/command_line/Tests:LibSwiftTests
Resulting in bytestream/upload for the test outputs:
"importantOutput": [
{
"name": "examples/command_line/Tests/LibSwiftTests",
"uri": "bytestream://remote.buildbuddy.io/blobs/593de6d72a4f278017c499cffe5cee734144740cffdc7a4cbea2b69eb2d712de/7289",
"pathPrefix": [
"bazel-out",
"darwin_arm64-fastbuild",
"bin"
]
},
{
"name": "examples/command_line/Tests/LibSwiftTests.zip",
"uri": "bytestream://remote.buildbuddy.io/blobs/a1f751591b0fb98ddd985dbf5bd957e59dd76b5c5ce61ccb7bd6030a400da02d/119243",
"pathPrefix": [
"bazel-out",
"applebin_macos-darwin_arm64-fastbuild-ST-3a19c795fefb",
"bin"
]
}
]
The following will somewhat repro the bug:
# Modify `tools/generator/test/AddTargetsTests.swift` adding something like
# `public let cache = "123"` and changing as needed to produce a new end output
bazel build --config=cache --modify_execution_info='.*=+no-remote' //tools/generator/test:tests
Resulting in bytestream/upload for the test runner, but not the binary:
"importantOutput": [
{
"name": "tools/generator/test/tests.test-runner.sh",
"uri": "bytestream://remote.buildbuddy.io/blobs/c4af21acfc33711fe544144629937d18dbd29aff5add8f946907fcf04c72ee21/2278",
"pathPrefix": [
"bazel-out",
"darwin_arm64-fastbuild",
"bin"
]
},
{
"name": "tools/generator/test/tests.xctest/Contents/MacOS/tests",
"uri": "file:///Users/brentley/Developer/rules_xcodeproj/bazel-output-base/execroot/com_github_buildbuddy_io_rules_xcodeproj/bazel-out/darwin_arm64-fastbuild/bin/tools/generator/test/tests.xctest/Contents/MacOS/tests",
"pathPrefix": [
"bazel-out",
"darwin_arm64-fastbuild",
"bin"
]
}
]
Which operating system are you running Bazel on?
macOS 12.4
What is the output of bazel info release?
release 5.3.0
If bazel info release returns development version or (@non-git), tell us how you built Bazel.
No response
What’s the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?
No response
Have you found anything relevant by searching the web?
No response
Any other information, logs, or outputs that you want to share?
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 17 (17 by maintainers)
SGTM. we can try to make this into
6.0and if everything works well we can removeexperimental_prefix and flip tominimalin the next major release.Made a PR for uploading stdout and stderr for actions as well https://github.com/bazelbuild/bazel/pull/17110
Seems that when an action fails, the stderr will be found under a path something like
bazel-out/_tmp/actions/stderr-5. I think it would make sense to upload those too.Thanks for the repro!
The reason why there are still outputs being uploaded even you set
--modify_execution_info='.*=+no-remote'ismodify_execution_infoonly applies to actions that supportexecution_info.In your repro, there are some actions created by the rule
macos_unit_teste.g.SymlinkActionandTemplateExpansionActiondon’t supportexecution_infohence their outputs are still uploaded. Hence, I wouldn’t consider this is a bug for flag--incompatible_remote_build_event_upload_respect_no_cache.However, I do understand that users do want to run all actions of a target locally sometimes, and prevent outputs of that target being uploaded by BEP.
--modify_execution_infowhich works on action granularity might not the best option for this purpose. I don’t have other options in my mind right now but we might need something works on rule granularity.It’s a git clone. I’ve changed the command to use https for easier use for people that don’t have ssh keys setup on GitHub.