moby: Unable to start docker service on Win2016 RTM from non-interactive process

Description

I am unable to start the docker service on a Windows Server 2016 RTM instance on AWS (or Azure), from under a non-interactive process.

I am trying to automate some build agents using packer and as part of that, I’m trying to pre-cache some images. However, the docker service is not started, and I cannot start it.

This also happens when I connect to the instance via a WinRM session, and under a TeamCity session.

Steps to reproduce the issue: Create a file: packer.json, with appropriate values for security group and subnet:

{
  "variables": {
    "aws_access_key": "{{env `AWS_ACCESS_KEY`}}",
    "aws_secret_key": "{{env `AWS_SECRET_KEY`}}"
  },
  "builders": [
    {
      "type": "amazon-ebs",
      "access_key": "{{user `aws_access_key`}}",
      "secret_key": "{{user `aws_secret_key`}}",
      "region": "ap-southeast-2",
      "source_ami": "ami-f4774a97",
      "instance_type": "c3.large",
      "ami_name": "2016-with-docker",
      "communicator": "winrm",
      "winrm_username": "Administrator",
      "windows_password_timeout": "20m",
      "subnet_id": "subnet-XXX",
      "associate_public_ip_address": true,
      "security_group_id": "sg-XXX",
      "user_data_file": "setup-winrm.txt"
    }
  ],
  "provisioners": [
    {
      "type":"powershell",
      "inline": [
        "docker version",
        "start-service docker"
      ]
    }
  ]
}

create another file: setup-winrm.txt:

<powershell>
winrm quickconfig -q
winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="2048"}'
winrm set winrm/config '@{MaxTimeoutms="1800000"}'
winrm set winrm/config/service '@{AllowUnencrypted="true"}'
winrm set winrm/config/service/auth '@{Basic="true"}'

netsh advfirewall firewall add rule name="WinRM 5985" protocol=TCP dir=in localport=5985 action=allow
netsh advfirewall firewall add rule name="WinRM 5986" protocol=TCP dir=in localport=5986 action=allow

net stop winrm
& c:\windows\system32\sc.exe config winrm start= auto
net start winrm

Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope LocalMachine
</powershell>

execute packer build --debug packer.json

Describe the results you received:

Service is not started, and does not start:

==> amazon-ebs: Prevalidating AMI Name...
==> amazon-ebs: Pausing after run of step 'StepPreValidate'. Press enter to continue. 
==> amazon-ebs: Inspecting the source AMI (ami-f4774a97)...
==> amazon-ebs: Pausing after run of step 'StepSourceAMIInfo'. Press enter to continue. 
==> amazon-ebs: Creating temporary keypair: packer_XXXXX
    amazon-ebs: Saving key for debug purposes: ec2_amazon-ebs.pem
==> amazon-ebs: Pausing after run of step 'StepKeyPair'. Press enter to continue. 
==> amazon-ebs: Pausing after run of step 'StepSecurityGroup'. Press enter to continue. 
==> amazon-ebs: Pausing after run of step 'stepCleanupVolumes'. Press enter to continue. 
==> amazon-ebs: Launching a source AWS instance...
    amazon-ebs: Instance ID: i-XXXXXXXX
==> amazon-ebs: Waiting for instance (i-XXXXXXXX) to become ready...
    amazon-ebs: Public DNS: ec2-XXX-XXX-XXX-XXX.ap-southeast-2.compute.amazonaws.com
    amazon-ebs: Public IP: XXX.XXX.XXX.XXX
    amazon-ebs: Private IP: XXX.XXX.XXX.XXX
==> amazon-ebs: Pausing after run of step 'StepRunSourceInstance'. Press enter to continue. 
==> amazon-ebs: Pausing after run of step 'stepTagEBSVolumes'. Press enter to continue. 
==> amazon-ebs: Waiting for auto-generated password for instance...
    amazon-ebs: It is normal for this process to take up to 15 minutes,
    amazon-ebs: but it usually takes around 5. Please wait.
    amazon-ebs:  
    amazon-ebs: Password retrieved!
    amazon-ebs: Password (since debug is enabled): XXXXXX
==> amazon-ebs: Pausing after run of step 'StepGetPassword'. Press enter to continue. 
==> amazon-ebs: Waiting for WinRM to become available...
==> amazon-ebs: Connected to WinRM!
==> amazon-ebs: Pausing after run of step 'StepConnect'. Press enter to continue. 
==> amazon-ebs: Provisioning with Powershell...
==> amazon-ebs: Provisioning with shell script: /var/folders/3k/dc8q184975n75tv39hb6s0700000gn/T/packer-powershell-provisioner050913598
    amazon-ebs: Client:
    amazon-ebs: Version:      1.12.2-cs2-ws-beta
    amazon-ebs: API version:  1.25
    amazon-ebs: error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.25/version: open //./pipe/docker_engine: The system cannot find the file specified.
    amazon-ebs: Go version:   go1.7.1
    amazon-ebs: Git commit:   050b611
    amazon-ebs: Built:        Tue Oct 11 02:35:40 2016
    amazon-ebs: OS/Arch:      windows/amd64
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: WARNING: Waiting for service 'Docker Engine (docker)' to start...
    amazon-ebs: start-service : Failed to start service 'Docker Engine (docker)'.
    amazon-ebs: At C:\Windows\Temp\script.ps1:2 char:1
    amazon-ebs: + start-service docker
    amazon-ebs: + ~~~~~~~~~~~~~~~~~~~~
    amazon-ebs: + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service],
    amazon-ebs: ServiceCommandException
    amazon-ebs: + FullyQualifiedErrorId : StartServiceFailed,Microsoft.PowerShell.Commands.StartServiceCommand
    amazon-ebs:
==> amazon-ebs: Pausing before cleanup of step 'StepConnect'. Press enter to continue. 

Describe the results you expected:

I expected the service to have already been started / start.

Additional information you deem important (e.g. issue happens only occasionally):

I tried this with connecting via WinRM to the launched instance, and I couldn’t start the service. I tried to run start-service docker under TeamCity, and it wouldn’t start either. I manually launched an instance, RDP’d to it, disabled the firewall, then WinRM’d to it, and it did work (not sure if the RDP’ing in first did something?).

Output of docker version:

Client:
Version:      1.12.2-cs2-ws-beta
API version:  1.25
error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.25/version: open //./pipe/docker_engine: The system cannot find the file specified.
Go version:   go1.7.1
Git commit:   050b611
Built:        Tue Oct 11 02:35:40 2016
OS/Arch:      windows/amd64

Output of docker info:

PS C:\Users\Administrator\Documents> docker info
docker : Cannot connect to the Docker daemon at npipe:////./pipe/docker_engine. Is the docker daemon running?
    + CategoryInfo          : NotSpecified: (Cannot connect ...daemon running?:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

Additional environment details (AWS, VirtualBox, physical, etc.): AWS