dvc: add: can't set the cache type to "copy" if the filesystem supports reflinks
Bug Report
Description
When the underlying filesystem (for instance ZFS 2.3) supports reflinks, DVC will always use them, even if you set the cache type to “copy”.
Reproduce
- Set the DVC cache type to “copy”
- Do a dvc add
- Observe that the reflink() function in dvc_objects/fs/local.py is called
Expected
DVC should use the copy strategy
Environment information
Output of dvc doctor:
$ dvc doctor
DVC version: 3.59.2 (pip)
-------------------------
Platform: Python 3.11.6 on Linux-6.8.0-59-generic-x86_64-with-glibc2.35
Subprojects:
dvc_data = 3.16.10
dvc_objects = 5.1.0
dvc_render = 1.0.2
dvc_task = 0.40.2
scmrepo = 3.3.11
Supports:
http (aiohttp = 3.11.18, aiohttp-retry = 2.9.1),
https (aiohttp = 3.11.18, aiohttp-retry = 2.9.1),
s3 (s3fs = 2025.3.0, boto3 = 1.37.3)
Config:
Global: /home/mermerico/.config/dvc
System: /etc/xdg/dvc
Cache types: reflink
Cache directory: zfs on tank
Caches: local
Remotes: s3
Workspace directory: zfs on tank
Repo: dvc, git
Repo.site_cache_dir: /home/mermerico/dvc_site_cache/repo/54ea0ca1cd8a6432245cbe79e3f79d2f
Additional Information (if any):
There’s some bug in the reflink strategy where I get “resource not available” errors (using ZFS 2.3.2) I’d like to collect more information about that bug and submit a report but I’m using a cache that’s shared with others on this machine. I need to enable reflinks on the FS level and then have all the other users use the “copy” strategy so they aren’t blocked while I’m debugging.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 16
Hey @mermerico, I am closing this as the immediate issue is now resolved, and I don’t think there’s anything we can do to fix this problem.
Hmm, after playing around with zfs_bclone_wait_dirty=0, it looks like it can take several seconds before ficlone is available. With zfs_bclone_wait_dirty=1 the ficlone can complete in 5ms. I don’t know if there is a way to manually flush the dirty data to disk. If not, the default ZFS behavior seems pretty broken. That’s especially true given the discussion you linked- it seems like the consensus is that moving forward people will try to default to reflinking. I can try to ask the ZFS people.
Sure, I can try to put together a PR