go: os: RemoveAll does not always work on very large directories
os.RemoveAll and other os operations (e.g., os.Rename) fail with large sets of files on Windows.
Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
1.8.3
What operating system and processor architecture are you using (go env)?
Windows amd64 and Windows Subsystem for Linux amd64
What did you do?
An example repo can be found at https://github.com/mattfarina/go-test-windows-files. It’s a repo with 10,000 files to be deleted to use as an example.
What did you expect to see?
I expect os.RemoveAll to delete all the files
What did you see instead?
Some of the files are deleted and then an error is thrown.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 50 (23 by maintainers)
Sorry, catching up with the comments on the original CL has been falling down my priority list. I won’t be picking it up until January at the earliest.
It’s important to point out here that the title is misleading and makes me sad that I mentioned Windows in the original bug report, because it’s too easy to focus on that too much.
This behavior exists for many filesystems on many platforms. The spec allows for concurrent modification and for unbounded directory sizes. There is no tractable implementation for very large directories other than the behavior the original CL aims to fix. Platform doesn’t really matter. You can almost certainly replicate the original bug with sufficiently large directories on both OSX and NFS today.
OK, that was fast. We believe this has been fixed in WSL in the recently released Windows 10 Fall Creators Update (“RS3”, build 16299). Please let us know if you’re able to reproduce this after updating to the latest Windows release.
I think the fix is in https://golang.org/cl/62970 .