dvc: Hardlinks not correctly detected on SMB network share (NTFS?)
Using DVC version 0.80, with repo and cache both on the same network drive. Windows setup.
I’m testing the unprotect
command, but I assume other commands relying on the same check will also exhibit strange failures.
PS Q:\dvc-test> dvc --version
0.80.0
PS Q:\dvc-test> cat .dvc\config
[cache]
dir = 'Q:\DVC cache (test)'
type = "reflink,symlink,hardlink,copy"
protected = true
PS Q:\dvc-test> dir
Directory: Q:\dvc-test
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 06/01/2020 16:58 .dvc
-a---- 06/01/2020 14:57 24 .gitignore
-a---- 06/01/2020 14:57 155 big_file.dvc
-ar--- 27/12/2019 16:42 134217728 big_file
PS Q:\dvc-test> fsutil.exe hardlink list big_file
Error: The request is not supported.
As you can see, I checked out my files from git at 14:57 today; dvc checkout created a protected link to the original 128Mb file created 10 days ago, rather than a copy (as requested), but Windows can’t tell me that it is a link. Trust me, it is 😉
PS Q:\dvc-test> dvc unprotect big_file
PS Q:\dvc-test> dir
Directory: Q:\dvc-test
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 06/01/2020 16:58 .dvc
-a---- 06/01/2020 14:57 24 .gitignore
-a---- 06/01/2020 14:57 155 big_file.dvc
-a---- 27/12/2019 16:42 134217728 big_file
Strange - the unprotect
operation was very fast, and the creation date of the file is still last year rather than today.
PS Q:\dvc-test> echo Hello >> big_file
PS Q:\dvc-test> dvc status
WARNING: corrupted cache file '..\DVC cache (test)\1d\cca63ad430e16fa12716d1a9bb3a6c'.
big_file_3.dvc:
changed outs:
not in cache: big_file_3
Sure enough, modifying big_file
corrupts the cache, since I was not modifying a copy.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 24 (24 by maintainers)
Commits related to this issue
- dvc: add sanity checks for hard/symlinks Fixed #3080 — committed to efiop/dvc by efiop 4 years ago
Bad news 😞
I’ve edited the issue description, I’m not 100% convinced that there actually is an NTFS system on the other side of the network. Windows seems to always report NTFS in the UI, even when there is something else (like an OSX share).
fsutil does work as expected on my local drive. I don’t know what WSL is I’m afraid.
Sure.