moby: Windows Server 2019 publish ports in swarm not working
Description
When running a container in a single node swarm on Windows Server 2019 you can not access the published ports.
Steps to reproduce the issue:
- Create a fresh VM on Azure using the Windows Server 2019 Datacenter with Containers image
- Fully patch the VM.
- Install latest docker update 18.09.6
- Run swarm init
- Use the following service yml file
version: '3.5'
services:
webtest:
image: mcr.microsoft.com/dotnet/core/samples:aspnetapp
deploy:
replicas: 1
endpoint_mode: dnsrr
ports:
- target: 80
published: 8000
protocol: tcp
mode: host
- Run the following to deploy the service
docker stack deploy -c .\service_test.yml service_test
- Try and browse to
http://localhost:8000orhttp://local ip of machine:8000
Describe the results you received:
The browser sits there forever and never makes a connection. If you look in the resource monitor in Windows you see that dockerd.exe is listening on port 8000 on all interfaces.
When you point a browser at that port you can see in resource monitor that dockerd has accepted a connection on port 8000 but it just doesn’t appear to be doing anything with it.
If you just run the command
docker run -it --rm -p 8000:80 --name aspnetcore_sample mcr.microsoft.com/dotnet/core/samples:aspnetapp
I.E just run is as a container by itself and not on the swarm then everything works fine.
If I do the same steps using the following yml
version: '3.5'
services:
webtest:
image: mcr.microsoft.com/dotnet/core/samples:aspnetapp
deploy:
replicas: 1
ports:
- target: 80
published: 8000
protocol: tcp
It still fails but this time I don’t see any listening ports for dockerd in resource monitor.
If I restart the docker service I find that the following events are logged in event viewer
HNSNetwork Request ={"Name":"uzjewd94ge9hi38jqzpseo0zx","Type":"overlay","Subnets":[{"AddressPrefix":"10.255.0.0/16","GatewayAddress":"10.255.0.1","Policies":[{"Type":"VSID","VSID":4096}]}],"AutomaticDNS":true}
followed by
Failed creating ingress network: hnsCall failed in Win32: An adapter was not found. (0x803b0006)
I found some comments online indicating this had something to do with only one physical network adapter being available on the machine so I added another network adapter in Azure and this error disappears but I still can’t access the container when it is in a swarm.
Describe the results you expected:
I’d expect to be able to access the website inside the container.
Additional information you deem important (e.g. issue happens only occasionally):
Output of docker version:
Client: Docker Engine - Enterprise
Version: 18.09.6
API version: 1.39
Go version: go1.10.8
Git commit: 1578dcadd2
Built: 05/04/2019 02:34:11
OS/Arch: windows/amd64
Experimental: false
Server: Docker Engine - Enterprise
Engine:
Version: 18.09.6
API version: 1.39 (minimum version 1.24)
Go version: go1.10.8
Git commit: 1578dcadd2
Built: 05/04/2019 02:32:24
OS/Arch: windows/amd64
Experimental: false
Output of docker info:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 3
Server Version: 18.09.6
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: ics l2bridge l2tunnel nat null overlay transparent
Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog
Swarm: active
NodeID: z59hrmf76s22pnnjo3t8xz7vt
Is Manager: true
ClusterID: lp7wuobnabi7rj0eow628k2x5
Managers: 1
Nodes: 1
Default Address Pool: 10.0.0.0/8
SubnetSize: 24
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 10.0.5.5
Manager Addresses:
10.0.5.5:2377
Default Isolation: process
Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
Operating System: Windows Server 2019 Datacenter Version 1809 (OS Build 17763.475)
OSType: windows
Architecture: x86_64
CPUs: 2
Total Memory: 4GiB
Name: docker2019
ID: T6FB:4SLL:ASKC:YHC6:3CP3:N2Q6:ULCQ:GAWL:PSEZ:3REE:GYFM:CJ7X
Docker Root Dir: C:\ProgramData\docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional environment details (AWS, VirtualBox, physical, etc.):
Azure
About this issue
- Original URL
- State: open
- Created 5 years ago
- Comments: 36 (5 by maintainers)
Thanks @olljanat & @darrensteadman we will take a look at this and related bugs.
@pradipd FYI
No, it does not work from within an overlay network.
When I do:
docker run --rm mcr.microsoft.com/dotnet/core/runtime:2.2 curl google.comIt works, I get a response.
When I do:
I do not get a response. Neither when I use an IP address:
docker run --network mynet --rm mcr.microsoft.com/dotnet/core/runtime:2.2 curl 13.90.143.69ipconfig /all from within the container:
I also tried this:
docker network create -d overlay -o com.docker.network.windowsshim.enable_outboundnat=true mynetBut that didn’t change anything.
ipconfig /all on the host:
So when I init the swarm I init on --advertise-addr=10.0.0.7, but I have tried the other network as well. Do I need to somehow create another hnsnetwork?
Get-HnsNetwork:
@darrensteadman @vitaliy-leschenko @daschott I just spun up a VM on Azure and stack deployed the following compose to a single node Win2019 server with containers.
Whilst I did not test the port access issue @darrensteadman reported, I was more interested in the error
On the azure VM the service and task start just fine.
However on a fully patched Win10 desktop with docker desktop community 19.03.05 the single node swarm service continually terminates the task and attempts to restart it every 5 seconds, citing the error above.
@daschott Will whatever fixed this in Enterprise trickle down to community version ? Would be nice to keep testing locally.