dvc: Hardlinks not correctly detected on SMB network share (NTFS?)

Using DVC version 0.80, with repo and cache both on the same network drive. Windows setup.

I’m testing the unprotect command, but I assume other commands relying on the same check will also exhibit strange failures.

PS Q:\dvc-test> dvc --version
0.80.0
PS Q:\dvc-test> cat .dvc\config
[cache]
dir = 'Q:\DVC cache (test)'
type = "reflink,symlink,hardlink,copy"
protected = true
PS Q:\dvc-test> dir


    Directory: Q:\dvc-test


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       06/01/2020     16:58                .dvc
-a----       06/01/2020     14:57             24 .gitignore
-a----       06/01/2020     14:57            155 big_file.dvc
-ar---       27/12/2019     16:42      134217728 big_file

PS Q:\dvc-test> fsutil.exe hardlink list big_file
Error:  The request is not supported.

As you can see, I checked out my files from git at 14:57 today; dvc checkout created a protected link to the original 128Mb file created 10 days ago, rather than a copy (as requested), but Windows can’t tell me that it is a link. Trust me, it is 😉

PS Q:\dvc-test> dvc unprotect big_file
PS Q:\dvc-test> dir


    Directory: Q:\dvc-test


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       06/01/2020     16:58                .dvc
-a----       06/01/2020     14:57             24 .gitignore
-a----       06/01/2020     14:57            155 big_file.dvc
-a----       27/12/2019     16:42      134217728 big_file

Strange - the unprotect operation was very fast, and the creation date of the file is still last year rather than today.

PS Q:\dvc-test> echo Hello >> big_file
PS Q:\dvc-test> dvc status
WARNING: corrupted cache file '..\DVC cache (test)\1d\cca63ad430e16fa12716d1a9bb3a6c'.
big_file_3.dvc:
     changed outs:
                                                                                                                       not in cache:       big_file_3

Sure enough, modifying big_file corrupts the cache, since I was not modifying a copy.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 24 (24 by maintainers)

Commits related to this issue

Most upvoted comments

Bad news 😞

(dvc-3080) PS Q:\dvc-test> dvc -v version
ERROR: unexpected error - (50, 'FindFileNames', 'The request is not supported.')


Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

Sorry, typo 😳 Windows NTFS network share

I’ve edited the issue description, I’m not 100% convinced that there actually is an NTFS system on the other side of the network. Windows seems to always report NTFS in the UI, even when there is something else (like an OSX share).

fsutil does work as expected on my local drive. I don’t know what WSL is I’m afraid.

Sure.

DVC version: 0.80.0
Python version: 3.7.1
Platform: Windows-10-10.0.16299-SP0
Binary: False
Package: pip
Cache: reflink - False, hardlink - True, symlink - False
Filesystem type (cache directory): ('NTFS', 'Q:\\')
Filesystem type (workspace): ('NTFS', 'Q:\\')