go: os: RemoveAll hangs on a path longer than 260 characters on Windows versions before Windows 10 1607

On Go 1.13.5 on Windows 2008 R2, running os.RemoveAll on a directory that contains a path longer than 260 characters hangs.

In the following repro, the directory is created, and then the program hangs on os.RemoveAll.

package main

import (
	"log"
	"os"
	"path/filepath"
)

func main() {
	// make a long path
	a := ""
	b := ""
	for i := 0; i < 150; i++ {
		a += "a"
		b += "b"
	}
	wd, err := os.Getwd()
	if err != nil {
		log.Fatal(err)
	}
	err = os.MkdirAll(filepath.Join(wd, "foo", "bar", a, b), 0755)
	if err != nil {
		log.Fatal(err)
	}

	// remove the root of the long path
	err = os.RemoveAll("foo")
	if err != nil {
		log.Fatal(err)
	}
}

About this issue

Original URL
State: open
Created 4 years ago
Comments: 44 (15 by maintainers)

Commits related to this issue

Revert "os: handle long path in RemoveAll for windows" This reverts CL 214437. Does not fix the issue, and the test was wrong so it did not detect that it did not fix the issue. Updates #36375 Cha... — committed to golang/go by ianlancetaylor 4 years ago
runtime: support long paths without fixup on Windows 10 >= 1607 Windows 10 >= 1607 allows CreateFile and friends to use long paths if bit 0x80 of the PEB's BitField member is set. In time this means... — committed to golang/go by zx2c4 3 years ago
fix(GODT-2326): Fix potential Win32 API deadlock Update gluon so that the store implementation uses `os.Remove` instead of `os.RemoveAll`. The latter has an issue where it can deadlock on windows. Se... — committed to ProtonMail/proton-bridge by LBeernaertProton a year ago

Most upvoted comments

Since I tagged this issue, I’ll explain why: although deletion due to filename-length is probably fixed in newer Windows due to long-filename support, other reasons for os.Remove returning an IsNotExist-matching error while the target file still exists will still produce an unterminating loop in os.RemoveAll.

One such case (the one that brought me here) is if the filename is not value UTF-16: the filename collected by (f *File) Readdirnames will not correctly round-trip back to match that file (utf16.Decode replaces unpaired surrogate code-points with the replacement character), and hence os.Remove will return an IsNotExist-matching error because it was given the name with replacement characters, which doesn’t exist. Hence, a file-not-found error is returned, the loop thinks it deleted something, closes and reopens the directory entry list, and tries to delete the same file again to the same effect, as seen in this bug report.

One example of filenames with invalid surrogate pairs is the files that msys and cygwin create when their POSIX API is used to delete, e.g., the currently-running executable, but the underlying Win32 API cannot perform that deletion.

I’m not sure if this can be addressed in Go, short of implementing a Windows-specific os.RemoveAll that doesn’t pass the filenames through a string, but maintains the []uint16 the whole way. That’s basically what we’re looking at doing in an external library, see https://github.com/microsoft/go-winio/pull/261.

TBBle on Mar 26, 2023

Let me set the environment and read some more about the process for submitting code, and probably will make this my first collaboration to go code

ackFacu on Apr 18, 2020