moby: Windows Server 2019: LockFile lock is not released by OS when process exits

Description

This problem is exhibited by running Mongo 4.0.8 with a named volume. If you restart Mongo it will not start because the WiredTiger.lock file remains locked. This is a general problem demonstrated by a sample Windows CLI program shown below.

Steps to reproduce the issue: Using Visual Studio 2017, compile the Windows CLI application shown below. I used the C++ code generation option “Multi-threaded” so the VS redistributables did not need to be included in the Dockerfile.

#include "pch.h"
#include "Windows.h"
#include <iostream>
int main() {
	const wchar_t* FILE_NAME = L"c:\\data\\Test.lock";
	HANDLE h = ::CreateFile(FILE_NAME, GENERIC_WRITE, 0, NULL, CREATE_NEW, FILE_ATTRIBUTE_NORMAL, NULL);
	if (h == INVALID_HANDLE_VALUE) {
		std::cout << "CreateFile failed";
		exit(1);
	}
	std::cout << "Created file " << FILE_NAME << std::endl;

	const char* buf = "Test";
	DWORD written;
	if (!::WriteFile(h, (LPCVOID)buf, sizeof(buf), &written, NULL)) {
		std::cout << "WriteFile failed" << std::endl;
		exit(1);
	}
	std::cout << "Wrote content to file " << FILE_NAME << std::endl;

	if (!::LockFile(h, 0, 0, 1, 0)) {
		std::cout << "LockFile failed" << std::endl;
		exit(1);
	}
	std::cout << "Locked first byte of file " << FILE_NAME << std::endl;

	std::cout << "Exiting without closing handle, OS must unlock" << std::endl;
}

Use the following Dockerfile:

FROM mcr.microsoft.com/windows/servercore:ltsc2019
COPY LockFileTest.exe /
CMD ["C:\\LockFileTest.exe"]

Build it:

docker build -t locktest .

Create a named volume:

docker volume create locktest

Run it:

docker run -v locktest:c:\data locktest

Describe the results you received: A file called Test.lock is located in the named volume C:\ProgramData\docker\volumes\locktest\_data. Try to open it with Visual Studio Code, it fails with the error EBUSY. The file remains locked even though the owning process has exited.

Describe the results you expected: As described by the documentation for LockFile (https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-lockfile) the OS should release the lock when the process exits. If you run the LockFileTest.exe on the desktop you will find that the file is not locked after the process exits.

Additional information you deem important (e.g. issue happens only occasionally): Workaround is that you can delete the file even though it is locked. So for Mongo you can delete the WiredTiger.lock file on startup if it exists.

Output of docker version:

Client:
 Version:           18.09.3
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        142dfcedca
 Built:             02/28/2019 06:33:17
 OS/Arch:           windows/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.09.3
  API version:      1.39 (minimum version 1.24)
  Go version:       go1.10.8
  Git commit:       142dfcedca
  Built:            02/28/2019 06:31:15
  OS/Arch:          windows/amd64
  Experimental:     false

Output of docker info:

Containers: 10
 Running: 3
 Paused: 0
 Stopped: 7
Images: 26
Server Version: 18.09.3
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: ics l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog
Swarm: active
 NodeID: gith69a6jn2rev6tk5tzkdd4c
 Is Manager: true
 ClusterID: a9kqvee722zhpotilk8gmnsts
 Managers: 1
 Nodes: 1
 Default Address Pool: 10.0.0.0/8
 SubnetSize: 24
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 10
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 10.0.2.15
 Manager Addresses:
  10.0.2.15:2377
Default Isolation: process
Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
Operating System: Windows Server 2019 Standard Version 1809 (OS Build 17763.379)
OSType: windows
Architecture: x86_64
CPUs: 4
Total Memory: 16GiB
Name: WIN-ER8SHU068SD
ID: W4KY:OBEM:VG5C:2KR7:ZYPJ:HKEJ:6SVH:GEY2:EQVS:HHUD:27T7:ZZO6
Docker Root Dir: C:\ProgramData\docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.): Windows 2019 Standard running in VirtualBox.