bazel: reject symlink outputs from cachable actions
The remote caching protocol doesn’t know about symlinks, and the remote spawn strategy currently converts all symlink into regular files when uploading to the cache. This behavior is insidious because local execution preserves symlinks. So, local and remote execution may yield different results if symlinks are involved:
$ mkdir /tmp/my-cache
$ cat BUILD
genrule(
name = 'emit-symlink',
outs = [':a', ':b'],
cmd = '''
touch $(location :a)
ln -s a $(location :b)
''',
)
$ bazel build --experimental_local_disk_cache --experimental_local_disk_cache_path /tmp/my-cache :emit-symlink
$ ls -lh bazel-genfiles/
total 0
-r-xr-xr-x 1 benjamin benjamin 0 Mar 13 02:35 a
lrwxrwxrwx 1 benjamin benjamin 1 Mar 13 02:35 b -> a
$ bazel clean --expunge
$ bazel build --experimental_local_disk_cache --experimental_local_disk_cache_path /tmp/my-cache :emit-symlink
$ ls -lh bazel-genfiles/
total 0
-r-xr-xr-x 1 benjamin benjamin 0 Mar 14 02:36 a
-r-xr-xr-x 1 benjamin benjamin 0 Mar 14 02:36 b
Handling symlinks in remote caching seems tricky. (What if the link target is absolute?) Therefore, I propose that trying to upload a symlink output to the cache should be an error.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 16 (11 by maintainers)
Commits related to this issue
- Only allow regular files and directories spawn outputs to be uploaded to a remote cache. The remote cache protocol only knows about regular files and directories. Currently, during action output uplo... — committed to benjaminp/bazel by benjaminp 6 years ago
- Only allow regular files and directories spawn outputs to be uploaded to a remote cache. The remote cache protocol only knows about regular files and directories. Currently, during action output uplo... — committed to benjaminp/bazel by benjaminp 6 years ago
- Only allow regular files and directories spawn outputs to be uploaded to a remote cache. The remote cache protocol only knows about regular files and directories. Currently, during action output uplo... — committed to benjaminp/bazel by benjaminp 6 years ago
- Only allow regular files and directories spawn outputs to be uploaded to a remote cache. The remote cache protocol only knows about regular files and directories. Currently, during action output uplo... — committed to benjaminp/bazel by benjaminp 6 years ago
- Allow banning symlink action outputs from being uploaded to a remote cache. This is mostly a roll-forward of 4465dae23de989f1452e93d0a88ac2a289103dd9, which was reverted by fa36d2f48965b127e8fd397348... — committed to benjaminp/bazel by benjaminp 6 years ago
Thanks for the guidance.
I think it should be an error. Warnings are fine for informing users about transient conditions like the remote cache being down. An action producing symlinks is similar to an action not producing promised artifacts. It’s an unambiguous deterministic function of the (gen)rule implementation.