moby: Process isolation not working for me on Win 10 1809

Description I try to get a process isolated container on Win 10 1809 (17763.107) but it hangs on startup. Might very well be something on my side, I just don’t know where to start looking

Steps to reproduce the issue:

  1. Install Docker Desktop 2.0.0.0
  2. Replace docker.exe and dockerd.exe with the files from https://master.dockerproject.com/ (https://master.dockerproject.com/windows/x86_64/docker.exe and https://master.dockerproject.com/windows/x86_64/dockerd.exe)
  3. Start the daemon like this to get debug output: "C:\Program Files\Docker\Docker\Resources\dockerd.exe" -G FUM-GLOBAL\tfenster -H npipe:////./pipe/docker_engine_windows -D (FUM-GLOBAL\tfenster is my user account)
  4. docker run a container using process isolation like docker run -ti --isolation=process mcr.microsoft.com/windows/nanoserver:1809

Describe the results you received: The container hangs on startup, nothing happens for more than 60 minutes: C:\WINDOWS\system32>docker run -ti --isolation=process mcr.microsoft.com/windows/nanoserver:1809

Hyper-v isolation works fine:

C:\WINDOWS\system32>docker run mcr.microsoft.com/windows/nanoserver:1809
Microsoft Windows [Version 10.0.17763.134]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\>

Describe the results you expected: A container running with process isolation

Additional information you deem important (e.g. issue happens only occasionally): Daemon log looks like this:

DEBU[2018-11-30T17:41:55.826424900+01:00] Calling GET /_ping
DEBU[2018-11-30T17:41:55.828426800+01:00] Calling POST /v1.40/containers/create
DEBU[2018-11-30T17:41:55.828945200+01:00] form data: {"AttachStderr":true,"AttachStdin":true,"AttachStdout":true,"Cmd":null,"Domainname":"","Entrypoint":null,"Env":[],"HostConfig":{"AutoRemove":false,"Binds":null,"BlkioDeviceReadBps":null,"BlkioDeviceReadIOps":null,"BlkioDeviceWriteBps":null,"BlkioDeviceWriteIOps":null,"BlkioWeight":0,"BlkioWeightDevice":[],"CapAdd":null,"CapDrop":null,"Cgroup":"","CgroupParent":"","ConsoleSize":[30,120],"ContainerIDFile":"","CpuCount":0,"CpuPercent":0,"CpuPeriod":0,"CpuQuota":0,"CpuRealtimePeriod":0,"CpuRealtimeRuntime":0,"CpuShares":0,"CpusetCpus":"","CpusetMems":"","DeviceCgroupRules":null,"Devices":[],"DiskQuota":0,"Dns":[],"DnsOptions":[],"DnsSearch":[],"ExtraHosts":null,"GroupAdd":null,"IOMaximumBandwidth":0,"IOMaximumIOps":0,"IpcMode":"","Isolation":"process","KernelMemory":0,"Links":null,"LogConfig":{"Config":{},"Type":""},"MaskedPaths":null,"Memory":0,"MemoryReservation":0,"MemorySwap":0,"MemorySwappiness":-1,"NanoCpus":0,"NetworkMode":"default","OomKillDisable":false,"OomScoreAdj":0,"PidMode":"","PidsLimit":0,"PortBindings":{},"Privileged":false,"PublishAllPorts":false,"ReadonlyPaths":null,"ReadonlyRootfs":false,"RestartPolicy":{"MaximumRetryCount":0,"Name":"no"},"SecurityOpt":null,"ShmSize":0,"UTSMode":"","Ulimits":null,"UsernsMode":"","VolumeDriver":"","VolumesFrom":null},"Hostname":"","Image":"mcr.microsoft.com/windows/nanoserver:1809","Labels":{},"NetworkingConfig":{"EndpointsConfig":{}},"OnBuild":null,"OpenStdin":true,"StdinOnce":true,"Tty":true,"User":"","Volumes":{},"WorkingDir":""}
DEBU[2018-11-30T17:41:55.832462500+01:00] hcsshim::GetLayerMountPath path C:\ProgramData\docker\windowsfilter\ecca0170f401bd3132a4b0fa72abea915e0f6ef94003dff5c00e6d3f93fc149b
DEBU[2018-11-30T17:41:55.832462500+01:00] Calling proc (1)
DEBU[2018-11-30T17:41:55.833955300+01:00] Calling proc (2)
DEBU[2018-11-30T17:41:55.834484400+01:00] hcsshim::GetLayerMountPath succeeded path=C:\ProgramData\docker\windowsfilter\ecca0170f401bd3132a4b0fa72abea915e0f6ef94003dff5c00e6d3f93fc149b mountPath=C:\ProgramData\docker\windowsfilter\ecca0170f401bd3132a4b0fa72abea915e0f6ef94003dff5c00e6d3f93fc149b
DEBU[2018-11-30T17:41:55.835987900+01:00] hcsshim::CreateScratchLayer path C:\ProgramData\docker\windowsfilter\c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668
DEBU[2018-11-30T17:41:55.836428300+01:00] hcsshim::NameToGuid name:ecca0170f401bd3132a4b0fa72abea915e0f6ef94003dff5c00e6d3f93fc149b guid:010df3e1-7918-5616-b7ed-f330ad22e0d8
DEBU[2018-11-30T17:41:55.837448500+01:00] hcsshim::NameToGuid name:7be5fa817a7ea239fa1b0e1a18f4587794dc8a043761eb25b7659026a9bdf0f1 guid:44ee543c-e9f7-5ac3-a463-d6dc23dde7ce
DEBU[2018-11-30T17:41:55.858979500+01:00] hcsshim::CreateScratchLayer - succeeded path=C:\ProgramData\docker\windowsfilter\c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668
DEBU[2018-11-30T17:41:55.894773700+01:00] Calling POST /v1.40/containers/c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668/attach?stderr=1&stdin=1&stdout=1&stream=1
DEBU[2018-11-30T17:41:55.895270900+01:00] attach: stderr: begin
DEBU[2018-11-30T17:41:55.895270900+01:00] attach: stdout: begin
DEBU[2018-11-30T17:41:55.895270900+01:00] attach: stdin: begin
DEBU[2018-11-30T17:41:55.898773300+01:00] Calling POST /v1.40/containers/c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668/wait?condition=next-exit
DEBU[2018-11-30T17:41:55.899770000+01:00] Calling POST /v1.40/containers/c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668/start
DEBU[2018-11-30T17:41:55.899770000+01:00] WindowsGraphDriver Get() id c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668 mountLabel
DEBU[2018-11-30T17:41:55.900773200+01:00] hcsshim::ActivateLayer path C:\ProgramData\docker\windowsfilter\c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668
DEBU[2018-11-30T17:41:55.949367600+01:00] hcsshim::ActivateLayer  - succeeded path=C:\ProgramData\docker\windowsfilter\c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668
DEBU[2018-11-30T17:41:55.949868400+01:00] hcsshim::PrepareLayer path C:\ProgramData\docker\windowsfilter\c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668
DEBU[2018-11-30T17:41:55.960868200+01:00] hcsshim::NameToGuid name:ecca0170f401bd3132a4b0fa72abea915e0f6ef94003dff5c00e6d3f93fc149b guid:010df3e1-7918-5616-b7ed-f330ad22e0d8
DEBU[2018-11-30T17:41:55.962865900+01:00] hcsshim::NameToGuid name:7be5fa817a7ea239fa1b0e1a18f4587794dc8a043761eb25b7659026a9bdf0f1 guid:44ee543c-e9f7-5ac3-a463-d6dc23dde7ce
DEBU[2018-11-30T17:41:56.025371000+01:00] hcsshim::PrepareLayer succeeded path=C:\ProgramData\docker\windowsfilter\c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668
DEBU[2018-11-30T17:41:56.025869300+01:00] hcsshim::GetLayerMountPath path C:\ProgramData\docker\windowsfilter\c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668
DEBU[2018-11-30T17:41:56.028867900+01:00] Calling proc (1)
DEBU[2018-11-30T17:41:56.032366700+01:00] Calling proc (2)
DEBU[2018-11-30T17:41:56.036873900+01:00] hcsshim::GetLayerMountPath succeeded path=C:\ProgramData\docker\windowsfilter\c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668 mountPath=\\?\Volume{31306e05-d5d7-4817-a68b-b130040e464e}
DEBU[2018-11-30T17:41:56.038868100+01:00] container mounted via layerStore: &{\\?\Volume{31306e05-d5d7-4817-a68b-b130040e464e} 0x30a3660 0x30a3660}
DEBU[2018-11-30T17:41:56.041371300+01:00] Assigning addresses for endpoint pensive_mclaren's interface on network nat
DEBU[2018-11-30T17:41:56.041869300+01:00] RequestAddress(172.21.16.0/20, <nil>, map[])
DEBU[2018-11-30T17:41:56.042370400+01:00] endpointStruct.EnableInternalDNS =[false]
DEBU[2018-11-30T17:41:56.043369700+01:00] [POST]=>[/endpoints/] Request : {"VirtualNetwork":"5A07FB7B-7843-457C-A526-06488B0CEDB2","EnableInternalDNS":true}
DEBU[2018-11-30T17:41:56.064871200+01:00] Network Response : {"ActivityId":"0ACFD905-8FF9-4529-8A72-BB1C4022C5A7","AdditionalParams":{},"CreateProcessingStartTime":131880697160558684,"DNSServerList":"172.21.16.1,192.168.1.1","DNSSuffix":"fritz.box","EnableInternalDNS":true,"EnableLowInterfaceMetric":true,"GatewayAddress":"172.21.16.1","Health":{"LastErrorCode":0,"LastUpdateTime":131880697160503678},"ID":"E21F82A3-5D83-499C-BBD8-82F17C4CC000","IPAddress":"172.21.17.9","MacAddress":"00-15-5D-D6-F0-69","Name":"Ethernet","Policies":[],"PrefixLength":20,"Resources":{"AdditionalParams":{},"AllocationOrder":0,"Health":{"LastErrorCode":0,"LastUpdateTime":131880697160503678},"ID":"0ACFD905-8FF9-4529-8A72-BB1C4022C5A7","PortOperationTime":0,"State":1,"SwitchOperationTime":0,"VfpOperationTime":0,"parentId":"84CD3469-F8E3-4173-A661-62168B8B249B"},"SharedContainers":[],"State":1,"Type":"nat","Version":38654705665,"VirtualNetwork":"5A07FB7B-7843-457C-A526-06488B0CEDB2","VirtualNetworkName":"nat"}
DEBU[2018-11-30T17:41:56.082369200+01:00] Assigning addresses for endpoint pensive_mclaren's interface on network nat
DEBU[2018-11-30T17:41:56.097867700+01:00] hcsshim::OpenComputeSystem ID=c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668
DEBU[2018-11-30T17:41:56.106369700+01:00] Programming external connectivity on endpoint pensive_mclaren (5e65471111572ed913df6eee44e2cea33caaa379d06614a57c3447aff3fa2b58)
DEBU[2018-11-30T17:41:56.106369700+01:00] EnableService c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668 START
DEBU[2018-11-30T17:41:56.117368200+01:00] EnableService c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668 DONE
DEBU[2018-11-30T17:41:56.133868500+01:00] hcsshim::NameToGuid name:ecca0170f401bd3132a4b0fa72abea915e0f6ef94003dff5c00e6d3f93fc149b guid:010df3e1-7918-5616-b7ed-f330ad22e0d8
DEBU[2018-11-30T17:41:56.144368900+01:00] hcsshim::NameToGuid name:7be5fa817a7ea239fa1b0e1a18f4587794dc8a043761eb25b7659026a9bdf0f1 guid:44ee543c-e9f7-5ac3-a463-d6dc23dde7ce
DEBU[2018-11-30T17:41:56.145376400+01:00] hcsshim::CreateComputeSystem ID=c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668 config={"SystemType":"Container","Name":"c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668","Owner":"docker","VolumePath":"\\\\?\\Volume{31306e05-d5d7-4817-a68b-b130040e464e}","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"C:\\ProgramData\\docker\\windowsfilter\\c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668","Layers":[{"ID":"010df3e1-7918-5616-b7ed-f330ad22e0d8","Path":"C:\\ProgramData\\docker\\windowsfilter\\ecca0170f401bd3132a4b0fa72abea915e0f6ef94003dff5c00e6d3f93fc149b"},{"ID":"44ee543c-e9f7-5ac3-a463-d6dc23dde7ce","Path":"C:\\ProgramData\\docker\\windowsfilter\\7be5fa817a7ea239fa1b0e1a18f4587794dc8a043761eb25b7659026a9bdf0f1"}],"HostName":"c80bd23ea18a","HvPartition":false,"EndpointList":["E21F82A3-5D83-499C-BBD8-82F17C4CC000"],"AllowUnqualifiedDNSQuery":true}
DEBU[2018-11-30T17:41:56.548869900+01:00] hcsshim::CreateComputeSystem succeeded id=c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668 handle=56164288
DEBU[2018-11-30T17:41:56.549369900+01:00] starting container                            container=c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668 module=libcontainerd namespace=moby
DEBU[2018-11-30T17:41:56.553868900+01:00] hcsshim::ComputeSystem::Start ID=c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668
WARN[2018-11-30T17:45:56.555017400+01:00] StartComputeSystem c80bd23ea18aac62251e61762e2781365ac8d2bfd7b7f5a2c5707332a0fa3668:: Did not complete within 4m0s. This may indicate a platform issue. If it appears to be making no forward progress, obtain the stacks and see is there is a syscall stuck in the platform API for a significant length of time.

Output of docker version:

C:\WINDOWS\system32>docker version
Client:
 Version:           master-dockerproject-2018-11-29
 API version:       1.40
 Go version:        go1.11.1
 Git commit:        504cecf2
 Built:             Thu Nov 29 23:50:48 2018
 OS/Arch:           windows/amd64
 Experimental:      false

Server:
 Engine:
  Version:          master-dockerproject-2018-11-29
  API version:      1.40 (minimum version 1.24)
  Go version:       go1.11.2
  Git commit:       baab736
  Built:            Thu Nov 29 23:59:33 2018
  OS/Arch:          windows/amd64
  Experimental:     false

Output of docker info:

Containers: 3
 Running: 0
 Paused: 0
 Stopped: 3
Images: 1
Server Version: master-dockerproject-2018-11-29
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: ics l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
Operating System: Windows 10 Enterprise Version 1809 (OS Build 17763.107)
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 7.927GiB
Name: DE00INDE044L1
ID: KYRQ:XK2M:COIT:DBL5:TDPF:CJIW:D4OQ:K5PV:W44E:A5FZ:3RCW:QSNQ
Docker Root Dir: C:\ProgramData\docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: -1
 Goroutines: 22
 System Time: 2018-11-30T22:17:58.826596+01:00
 EventsListeners: 0
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.): Physical Surface laptop

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 36 (8 by maintainers)

Most upvoted comments

I am sure it works on a clean PC, enough people are reporting that it does. I sent @jhowardmsft the logs and he got back saying that he needs someone else internally to look at it (https://github.com/moby/moby/issues/38306#issuecomment-443779692), so I don’t think I have the means to find out what is causing my problem.

I’ve also tried with ProcMon (https://github.com/moby/moby/issues/38306#issuecomment-447769178) but I don’t know what the error message means and a web research didn’t bring much relevant result

FWIW, I retried with the nightlies from 2019-01-04 but still the same problem

@tfenster FYI. jhowardmsft is currently out of office but I’m sure that he will look about this when he is back.

Also notice that docker/engine#81 is merged so Docker CE 18.09.1 hopefully contains official, tested version of this.