bazel: `--experimental_remote_download_regex` doesn't work with a cache in 6.0.0rc4
Description of the bug:
The flag --experimental_remote_download_regex
which is used by some IDE integrations doesn’t work with --disk_cache
and --remote_download_toplevel
in 6.0.0rc2. I was able to verify that the same behavior works great in 5.3.0, but suddenly broke sometime before 6.0.0rc2. I couldn’t bisect exactly due to some breakages between Bazel and rules_apple, but I don’t think there are many changes that could impact it.
What’s the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
This is a bit tricky to reproduce, but it goes as following:
- Use the sample project at https://github.com/BalestraPatrick/rules_xcodeproj/tree/repro-disk-cache-bug.
cd examples/integration
and generate the project withbazelisk run //:xcodeproj-sim_arm64
.- Run a build with
bazelisk run //:xcodeproj-sim_arm64 --cpu=ios_sim_arm64 -- --generator_output_groups=all_targets 'build --remote_download_toplevel --cpu=ios_sim_arm64 --apple_platform_type=ios --disk_cache=~/.test-cache'
(Xcode 14.0 might be required). - Open
iOSApp.swift
and make some modifications (simply modify the existingSwift.print
statement). Run a no-op build as a test. This is because the errors in importing the indexstore (which is what we download through--remote_download_regex
) will show up only on the following build. Everything should work fine and no errors should be seen because the example project is using Bazel 5.3.0. You can verify that the indexstore has contents in something like/Users/you/src/rules_xcodeproj/examples/integration/bazel-output-base/rules_xcodeproj/build_output_base/execroot/__main__/bazel-out/ios-sim_arm64-min15.0-applebin_ios-ios_sim_arm64-dbg-ST-2ccbe0f541dd/bin/iOSApp/Source/iOSApp.library.indexstore/
. - Change
.bazelversion
to 6.0.0rc2 (or likely earlier revisions will reproduce the issue too). - Repeat the earlier steps to trigger an incremental build. The
indexstore
directory will contain the right files as expected. Then do a no-op build which will be fetched from the disk cache. If you now list the contents of the same directory as before, you’ll see that theindexstore
directory is empty. Instead, it should contain actual files like the build with 5.3.0 or any other build that doesn’t hit the disk cache.
My assumption is that the values specified in --remote_download_regex
(here) are not respected when an action is fetched from the disk cache. When the action is executed locally or fetched from a remote cache, everything seems to work as expected.
Which operating system are you running Bazel on?
macOS 13.0
What is the output of bazel info release
?
No response
If bazel info release
returns development version
or (@non-git)
, tell us how you built Bazel.
6.0.0rc2
What’s the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD
?
No response
Have you found anything relevant by searching the web?
No response
Any other information, logs, or outputs that you want to share?
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 23 (23 by maintainers)
Commits related to this issue
- Correctly match regex with tree artifact It doesn't work when setting `--experimental_remote_download_regex` to match files inside tree artifact after https://github.com/bazelbuild/bazel/commit/e01e7... — committed to coeuvre/bazel by coeuvre 2 years ago
- Correctly match regex with tree artifact (#16958) It doesn't work when setting `--experimental_remote_download_regex` to match files inside tree artifact after https://github.com/bazelbuild/bazel/com... — committed to bazelbuild/bazel by coeuvre 2 years ago
I found the root cause: we didn’t match regex with the files inside tree artifact.
Detailed explaination:
iOSApp.library.indexstore
is a tree artifact.iOSApp/Source/iOSApp.swift
. In the following build, Bazel will rerun the action that generateiOSApp.library.indexstore
.iOSApp.library.indexstore
.iOSApp.library.indexstore
is correctly generated.iOSApp.library.indexstore
are not downloaded because the regex is.*\.indexstore/.*
and we only match it withiOSApp.library.indexstore
not the files inside it.Working on the fix.
Thanks for the repro, i am looking into this issue.
@BalestraPatrick Then we’ll move forward with the 6.0 release, and if we identify the other potential incremental bug, we can fix it in later minor/patch releases.
@coeuvre Yeah, I was testing with
bazelisk run //:xcodeproj -- "clean --expunge"
to clean the output base that the runner script uses. So from what I understand, the problem should be fixed. I will let you know if I’m able to reproduce the issue in rc4 or later, but feel free not to consider this a release blocker now. Thanks for the help!Either way I think this exact bug is fixed. There might be another bug related to incremental state.
@bazel-io fork 6.0.0