rclone: vfs/refresh does not cache entire directory
Hello, Facing a strange issue with rclone VFS.
Version I am running: v1.45-133-g291f2709-beta (although this happened on stable as well)
It seems like the mount is not seeing a large chunk of my directories:
rclone mount 323 dirs rclone vfs/refresh = 88 dirs rclone vfs/forget = 323 dirs
Commands:
ls -lrth /mnt/remote/Media/TV |wc -l 88
rclone rc vfs/forget dir=Media/TV { “forgotten”: [ “Media/TV” ] }
ls -lrth /mnt/remote/Media/TV |wc -l 323
This happens across any directory, here is another example:
ls -lrth /mnt/remote/Media/Movies/HD |wc -l 435
rclone rc vfs/forget dir=Media/Movies/HD { “forgotten”: [ “Media/Movies/HD” ] }
ls -lrth /mnt/remote/Media/Movies/HD |wc -l 628
Any thoughts would be appreciated
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 1
- Comments: 31 (14 by maintainers)
Commits related to this issue
- walk: make NewDirTree always use ListR #2946 This fixes vfs/refresh with recurse=true needing the --fast-list flag — committed to rclone/rclone by ncw 5 years ago
- walk: make NewDirTree always use ListR #2946 This fixes vfs/refresh with recurse=true needing the --fast-list flag — committed to rclone/rclone by ncw 5 years ago
- drive: fix ListR for items with multiple parents Fixes #2946 — committed to B4dM4n/rclone by B4dM4n 5 years ago
- drive: fix ListR for items with multiple parents Fixes #2946 — committed to rclone/rclone by B4dM4n 5 years ago
- walk: make NewDirTree always use ListR #2946 This fixes vfs/refresh with recurse=true needing the --fast-list flag — committed to gcornut/rclone by ncw 5 years ago
- drive: fix ListR for items with multiple parents Fixes #2946 — committed to gcornut/rclone by B4dM4n 5 years ago
- walk: make NewDirTree always use ListR #2946 This fixes vfs/refresh with recurse=true needing the --fast-list flag — committed to gcornut/rclone by ncw 5 years ago
- drive: fix ListR for items with multiple parents Fixes #2946 — committed to gcornut/rclone by B4dM4n 5 years ago
Confirming that this has resolved the issue!
Pre:
Post:
Thank you to everyone involved in figuring this one out 👍
Excellent news!
I think there were quite a few contributors 👏
We’ve merged this to master now which means it will be in the latest beta in 15-30 mins and released in v1.46
I pushed @B4dM4n 's branch to the main repo to make a beta for you to try
https://beta.rclone.org/branch/v1.45-185-g177c274f-drive-multi-parents-beta/ (uploaded in 15-30 mins)
I found the issue in the drive
ListRimplementation and created a fix for it.Once the PR is merged
ListRwill correctly handle items with multiple parents, at the cost of scanning those folders once per parent if needed.In general the handling of items with multiple parents might not behave as expected in all circumstances. e.g. deleting a item(file or folder) with multiple parents will remove it from all parents.