bazel: Builds without the Bytes fails to fetch tree artifact inputs

Description of the problem / feature request:

We’re trying to enable --remote_download_minimal/--remote_download_toplevel for our builds (which use disk cache and remote cache), and encountered this error:

ERROR: /path/to/BUILD:71:12: Bundling App failed: Exec failed due to IOException: 181 errors during bulk transfer
com.google.devtools.build.lib.vfs.FileAccessException: /path/to/bazel-output-base/execroot/__main__/bazel-out/applebin_ios-ios_x86_64-dbg-ST-1f63e7a63487/bin/App/App-intermediates/nibs/Bundle.bundle/ViewController/ViewController.nib (Permission denied)
(a lot other similar logs)

(a Bazel release that includes https://github.com/bazelbuild/bazel/commit/113eaca5862c48797654ae2a3acbb6e15d761485 is needed to see the underlying errors)

Upon some investigation, looks like the exception was thrown somewhere between SpawnExecutionContext.prefetchInputsAndWait.

https://github.com/bazelbuild/bazel/blob/88499f615bc03d1b6a3e4e38f2890a4e987a954d/src/main/java/com/google/devtools/build/lib/exec/SpawnRunner.java#L168

Another pattern I noticed is that inputs that were failed to fetch were all tree artifacts.

The build would succeed if I run the same bazel build again, but would fail again if I make some change and rebuild.

Feature requests: what underlying problem are you trying to solve with this feature?

Using --remote_download_minimal/--remote_download_toplevel for local builds.

What operating system are you running Bazel on?

macOS

What’s the output of bazel info release?

release 6.0.0-pre.20220105.5