bazel: Building iOS resources causes bazel to cache a file for the null sha
Currently if you build an iOS target that has resources (storyboards or asset catalogs in my testing) bazel will end up caching a file for the null sha e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855. With remote caches this results in failed builds if you accept this file in your cache.
You can reproduce this with one of the sample projects in rules_apple:
bazel build //examples/ios/HelloWorld --disk_cache=cache
ls cache | grep e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Here you should see a match. If you remove all the resources portions of this BUILD file, this is no longer the case. After applying this diff:
diff --git i/examples/ios/HelloWorld/BUILD w/examples/ios/HelloWorld/BUILD
index 7f9385d..a3c4753 100644
--- i/examples/ios/HelloWorld/BUILD
+++ w/examples/ios/HelloWorld/BUILD
@@ -15,9 +15,6 @@ objc_library(
"Sources/AppDelegate.m",
"Sources/main.m",
],
- resources = [
- "Resources/Main.storyboard",
- ],
)
apple_bundle_version(
@@ -27,14 +24,12 @@ apple_bundle_version(
ios_application(
name = "HelloWorld",
- app_icons = ["//examples/resources:PhoneAppIcon.xcassets"],
bundle_id = "com.example.hello-world",
families = [
"iphone",
"ipad",
],
infoplists = [":Info.plist"],
- launch_storyboard = "//examples/resources:Launch.storyboard",
minimum_os_version = "8.0",
version = ":HelloWorldVersion",
deps = [":Sources"],
And running:
rm -rf cache
bazel clean
bazel build //examples/ios/HelloWorld --disk_cache=cache
ls cache | grep e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
You should no longer see any matches. I see a few other references to e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 in other bazel issues, but none specifically talking about whether or not it is a problem. It seems like a few remote cache implementations work around this by not caching files that are empty (which seem to only happen for us when the key is also this sha). I can’t find the exact rule that causes this with aquery, but I’d love to know if this is considered an issue, or if we should work around this in our remote cache.
What operating system are you running Bazel on?
macOS
What’s the output of bazel info release?
0.17.1
I migrated this issue from here https://github.com/bazelbuild/rules_apple/issues/228
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 1
- Comments: 21 (19 by maintainers)
Commits related to this issue
- [Remote Cache] Don't delete tree artifacts on failure Previously if a remote cache request failed, bazel would cleanup all directories that were marked as outputs for the action. These ... — committed to luca-digrazia/DatasetCommitsDiffSearch by deleted user 2 years ago
- Remove superfluous code in RemoteExecutionService. The directories that are to contain spawn outputs have already been created before the spawn executes, so it's unnecessary for RemoteExecutionServic... — committed to bazelbuild/bazel by tjgq a year ago
Ok so I can reproduce this with the example I provided above like this (with our remote cache running locally):
In this case I get this error from the iOS toolchain:
Here's the entire execution log from the failed build (click to expand)
We saw the same issue, and we work around with our remote cache to handle empty sha as well
Thanks for all the context here! I’ve submitted https://github.com/bazelbuild/bazel/pull/6851 which I’ve verified locally fixes this issue.
Bazel creates all declared output directories before executing an action. It’s best to not rely on this behavior because it can cause trouble with remote execution. I am saying this fully realizing that many rules do rely on this behavior (which is fair) 😃. See also https://github.com/bazelbuild/bazel/issues/6393.
The issue here probably is that Bazel creates the empty output directories, runs the action, gets a remote cache hit and when the download fails deletes the output directories. It then falls back to local execution which doesn’t actually create the empty directory because it relies on Bazel to create it.
So that then seems to be a bug in the remote cache cleanup logic. We should only delete contents of output directories but not the directories themselves?
The documentation for
declare_directorysays that actions should create (i.e.mkdir) these output directories. But if you take a look at github.com/sergiocampama/tree-artifacts, you’ll see that the documentation is incorrect, as that target builds correctly and the directories get created.@buchgr can you confirm that tree artifact outputs, as created by
ctx.actions.declare_directory, should be created by the rule author or by bazel? Is this an unintended consequence that bazel does it for the rule authors?Now, as to what pertains to this issue, the problem that @keith is seeing is that when using remote cache, in some cases, bazel will fail to create this output tree artifact during an action execution. So, what we’d like to know is whether the fact that this output directory is missing in this circumstance is a bug with bazel & remote cache, or something that rule authors should be aware of and support in their rules.
@buchgr do you have an understanding of what the issue is? We’re seeing this again (I believe) with another similar action. I’d be happy to attempt a patch but I’m not sure where to start