moby: dnsrr endpoint mode not working with routing mesh
Output of docker version:
docker@node2:~$ docker --version
Docker version 1.12.0-rc4, build e4a0dbc, experimental
Output of docker info:
docker@node2:~$ docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 1.12.0-rc4
Storage Driver: aufs
Root Dir: /mnt/sda1/var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 0
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: null overlay bridge host
Swarm: active
NodeID: a0j7h4vbc32wmzw1cl56cey9h
IsManager: No
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 4.4.8-boot2docker
Operating System: Boot2Docker 1.11.1 (TCL 7.0); master : 7954f54 - Wed Apr 27 17:59:58 UTC 2016
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 995.9 MiB
Name: node2
ID: PWMA:PMXK:J4VC:Y665:XWJ5:J2AG:U43U:CG6T:VNWA:7USA:V3QU:ZGIM
Docker Root Dir: /mnt/sda1/var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 22
Goroutines: 120
System Time: 2016-07-25T17:02:40.657240432Z
EventsListeners: 0
Registry: https://index.docker.io/v1/
Labels:
provider=virtualbox
Experimental: true
Insecure Registries:
127.0.0.0/8
Additional environment details (AWS, VirtualBox, physical, etc.):
Steps to reproduce the issue:
-
Create 2 node swarm cluster docker@node1:~$ docker node ls ID HOSTNAME MEMBERSHIP STATUS AVAILABILITY MANAGER STATUS a0j7h4vbc32wmzw1cl56cey9h node2 Accepted Ready Active
bfqlqktthzijxk0thf371zxx8 * node1 Accepted Ready Active Leader
Create 2 service with dnsrr endpoint and routing mesh docker service create --endpoint-mode dnsrr --replicas 1 --name client --network overlay1 smakam/myubuntu:v4 ping docker.com docker service create --endpoint-mode dnsrr --name vote --network overlay1 --replicas 2 -p 8080:80 instavote/vote 3.
Describe the results you received:
Service gets stuck in new state
docker@node1:~$ docker service ls
ID NAME REPLICAS IMAGE COMMAND
4gljblqy8u0t client 1/1 smakam/myubuntu:v4 ping docker.com
4wbpkae9pdsz vote 0/2 instavote/vote
docker@node1:~$ docker service tasks vote
ID NAME SERVICE IMAGE LAST STATE DESIRED STATE NODE
53ttlrww3c9cfzlmt4y8jrv9m vote.1 vote instavote/vote New 58 seconds ago Running
0xptv7b3mjbgsy1q1c1rpqnps vote.2 vote instavote/vote New 58 seconds ago Running
Describe the results you expected: Service should transition to running
Additional information you deem important (e.g. issue happens only occasionally): dnsrr endpoint works fine without routing mesh. For example, following services got created fine and moved to running state. docker service create --endpoint-mode dnsrr --replicas 1 --name client --network overlay1 smakam/myubuntu:v4 ping docker.com docker service create --endpoint-mode dnsrr --name vote --network overlay1 --replicas 2 instavote/vote
docker@node1:~$ docker service ls
ID NAME REPLICAS IMAGE COMMAND
7ctu5us9gu5f vote 2/2 instavote/vote
da5q0xscsw2d client 1/1 smakam/myubuntu:v4 ping docker.com
Also, routing mesh works fine with vip based endpoint mode.
About this issue
- Original URL
- State: open
- Created 8 years ago
- Comments: 18 (9 by maintainers)
Three years since the ticket was opened, one year since the last comment - and I’m still not sure I understand what the intended behaviour or supported use case is.
For some things like elasticsearch, you will sometimes not want load balancing across them. For example, there are elasticsearch clients which discover which subsections of the data are stored on each node, such that it can go direct to the node that holds the data it’s looking for. Swarm’s load balancing will cause problems here, because you never know which node you are going to be taken to. Similarly, in the context of web servers, this makes things like sticky-session load balancing impossible.
As noted in this ticket multiple times, if you disable the load balancing (https://docs.docker.com/network/overlay/#bypass-the-routing-mesh-for-a-swarm-service says to use
--endpoint-mode dnsrr), it’s impossible to expose any ports (at least if using a compose file with swarm). I’m really struggling to understand the point of having a way to bypass the routing mesh, if it’s impossible to expose any ports to reach!@RehanSaeed use
--publish target=,published=,mode=hostworks for meI’m getting the following error when using
endpoint-mode: dnsrrusing Docker Swarm:Is there a workaround in the latest 17.06 or 17.09 releases? I’m trying to start RabbitMQ which has it’s own internal clustering ability with
dnsrrand ports exposed.cc @mrjana @al I think we should remove this restriction also to support swarm-mode for windows where routing-mesh will not be enabled and the user should be able to use DNS-RR on the back-end networks with published ports.
@smakam @tyoh @guenhter @johnharris85 @pascalandy I opened https://github.com/docker/swarmkit/issues/1693 to take care of this issue. But @mrjana has some valid concerns with the use-cases. Could you please comment in https://github.com/docker/swarmkit/issues/1693 so that we understand the use-case better ?