moby: rename / move file fails on mounted volume in Windows Server 2019

Description

Renaming or moving files within mounted volumes fails with a “path not found” error. This happens when renaming the file on the command line: ren README.md README.md.bak As well as with powershell Move-Item (see the attached screenshot): Move-Item .\README.md .\README.md.bak Moving files in compiled code with the MoveFile function of the Windows API also results in an error (microsoft error code 3 - file not found).

So it looks like renaming/moving files seems to fail at the operating system level. Copying and deleting files works without problems.

Steps to reproduce the issue:

  1. docker run --name Test -it -v C:\Users\User\Documents\volume:c:/vs microsoft/windowsservercore cmd.exe
  2. docker attach Test
  3. ren c:\vs\MariaDB\README.md c:\vs\MariaDB\README.md.bak

Describe the results you received:

Moving / renaming the file resulted in an error message: Could not find a part of the path.

2018-11-22_screenshot

Describe the results you expected:

Successful renamed / moved file.

Additional information you deem important (e.g. issue happens only occasionally):

We stumbled upon this issue when we were trying to start a MariaDB database instance (with InnoDB engine) on the mounted volume. Under certain circumstances, the database tries to move files on startup and shutdown and then stops with an error message. We checked the MariaDB source code and noticed that they were using the Microsoft Windows API MoveFile function. We changed that to do a CopyFile (and DeleteFile) instead and that ran without error. That led us to check the rename commands on the shell and noticed that those also failed.

Output of docker version:

Client:
 Version:           18.03.1-ee-4
 API version:       1.37
 Go version:        go1.10.2
 Git commit:        0ded23c
 Built:             Thu Oct 25 00:41:52 2018
 OS/Arch:           windows/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.03.1-ee-4
  API version:      1.37 (minimum version 1.24)
  Go version:       go1.10.2
  Git commit:       0ded23c
  Built:            Thu Oct 25 00:56:17 2018
  OS/Arch:          windows/amd64
  Experimental:     false

Output of docker info:

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 3
Server Version: 18.03.1-ee-4
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: ics l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: process
Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
Operating System: Windows Server 2019 Standard Version 1809 (OS Build 17763.134)
OSType: windows
Architecture: x86_64
CPUs: 4
Total Memory: 3.978GiB
Name: DG-DOCKER2019
ID: WBWU:XTLJ:KHYI:OWMH:7LLI:ORWM:XAUU:YW2W:5LSP:5ZYH:VC7E:GQRW
Docker Root Dir: C:\ProgramData\docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

Windows Server 2019 Standard / Windows Docker Container Processisolation (without Hyper-V)

About this issue

  • Original URL
  • State: open
  • Created 6 years ago
  • Reactions: 4
  • Comments: 27 (8 by maintainers)

Most upvoted comments

We are now having this issue on one of our machines as well. It used to work until a few days ago.

Docker 2.2.3.0 Windows 10 1909 18363.720

Some observations:

  • It’s possible to rename a file, but not a directory.
  • The problem only happens when mapping a folder on the System Volume (C:) to . When mapping a folder on D:\ it works. Quotas are disabled.

If I can provide you with any information to narrow this down, please let me know.

@rcwoolley : as far as I know, ID stability is only an issue when creating new users inside an image or container. Windows has the concept of “well known user ids”, and there are two users in this category that were created for docker: ContainerUser & ContainerAdministrator. My reproduction uses ContainerAdministrator (which in turn is a member of the well known group Administators).

I do not think this could be related - after all, we can delete the files for exemple, and the issue also appears by itself after a while in the same container. But thanks for trying to make this issue progress!

@jhowardmsft Thanks a lot for trying to reproduce the issue. We just tried again by just updating Microsoft Windows Server 2019 to the latest version and we also couldn’t reproduce the issue. It seems that Microsoft has fixed it in one of the recent updates!

Thanks again for taking the time to check! We probably wouldn’t have tried process isolation again, but now we can use that and avoid the overhead of hyperv isolation. 😃