weave: After 1.5.0 Update DNS Name Resolution Using the Network Plugin Does Not Work

I am using the Docker Network Plugin (–net=weave) to communicate between containers that reside on different hosts. Docker + Weave up & running on all hosts.

root@host1:/# weave status
        Version: 1.5.0 (up to date; next check at 2016/04/19 21:31:55)

        Service: router
       Protocol: weave 1..2
           Name: <host name & address>
     Encryption: disabled
  PeerDiscovery: enabled
        Targets: 1
    Connections: 2 (2 established)
          Peers: 3 (with 6 established connections)
 TrustedSubnets: none

        Service: ipam
         Status: ready
          Range: 10.32.0.0-10.47.255.255
  DefaultSubnet: 10.32.0.0/12

        Service: dns
         Domain: weave.local.
       Upstream: <DNS addresses>
            TTL: 1
        Entries: 0

        Service: proxy
        Address: unix:///var/run/weave/weave.sock

        Service: plugin
     DriverName: weave

root@host1:/# docker --version
Docker version 1.11.0, build 4dc5990

My test example is going to be very similar to the instructions in the documentation.

root@host1:/# docker run -it --name a1 --net=weave --hostname a1.weave.local $(weave dns-args) ubuntu:trusty
root@host2:/# docker run -it --name a2 --net=weave --hostname a2.weave.local $(weave dns-args) ubuntu:trusty

At this point weave status dns shows:

root@host1:/# weave status dns
a1           10.32.0.1       <container ID> <MAC address>
a2           10.40.0.1       <container ID> <MAC address>

But when I try to ping one of the containers (from the other container, of course) using their hostname, I get ping: unknown host a1 (or a2). Pinging using the IP address works.

The most strange part is that if I use the Docker API Proxy, it all works as expected. However I don’t want to go that route because I automatize all this process and I don’t want to think about environment variables, whether I do it all with sudo or in a root shell, whether I am in the same shell when I eval $(weave env) and run my docker container, etc.

root@host1:/# eval $(weave env)
root@host1:/# docker run -it --name a1 ubuntu:trusty
root@host2:/# eval $(weave env)
root@host2:/# docker run -it --name a2 ubuntu:trusty
root@a2:/# ping a1
PING a1.weave.local (10.32.0.1) 56(84) bytes of data.
64 bytes from a1.weave.local (10.32.0.1): icmp_seq=1 ttl=64 time=0.491 ms
64 bytes from a1.weave.local (10.32.0.1): icmp_seq=2 ttl=64 time=0.455 ms
^C
--- a1.weave.local ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.455/0.473/0.491/0.018 ms

Not to mention I didn’t have this problem with version 1.4.6. Multiple reinstallations of weave and creating new containers didn’t help.

Here is the log of weaveplugin:

root@host1:/# docker logs weaveplugin
INFO: 2016/04/19 19:18:12.253230 Weave plugin 1.5.0 Command line options: [--log-level=debug]
INFO: 2016/04/19 19:18:12.254587 Docker API on unix:///var/run/weave/weave.sock: &[KernelVersion=3.19.0-58-generic BuildTime=2016-04-13T18:34:23.204131695+00:00 Version=1.11.0 ApiVersion=1.23 GitCommit=4dc5990 GoVersion=go1.5.4 Os=linux Arch=amd64]
INFO: 2016/04/19 19:18:12.254736 Listening on /run/docker/plugins/weave.sock for global scope
INFO: 2016/04/19 19:18:12.254841 Listening on /run/docker/plugins/weavemesh.sock for local scope
DEBU: 2016/04/19 19:18:12.264378 [net] ContainerStarted: fe335f5ffe09ff330e68d3bdaa3e2c48cc003fb51803eb34692768461ae0805b
DEBU: 2016/04/19 19:18:12.264413 [net] ContainerStarted: fe335f5ffe09ff330e68d3bdaa3e2c48cc003fb51803eb34692768461ae0805b
INFO: 2016/04/19 19:18:12.333025 [ipam] RequestPool weavelocal   map[]
DEBU: 2016/04/19 19:18:12.333064 weave GET to http://127.0.0.1:6784/ipinfo/defaultsubnet with map[]
DEBU: 2016/04/19 19:18:12.333930 [ipam] RequestPool result weave-10.32.0.0/12-10.32.0.0/12 10.32.0.0/12 map[com.docker.network.gateway:10.32.0.0/12]
DEBU: 2016/04/19 19:18:12.334500 [net] CreateNetwork &{NetworkID:14a5ff7efbc8528c3bbad06863abd2ee8679d5cf555f93cab4e62193e117a0d6 Options:map[com.docker.network.enable_ipv6:false com.docker.network.generic:map[]] IPv4Data:[{AddressSpace: Pool:10.32.0.0/12 Gateway:10.32.0.0/12 AuxAddresses:map[]}] IPv6Data:[]}
INFO: 2016/04/19 19:18:12.334524 [net] CreateNetwork 14a5ff7efbc8528c3bbad06863abd2ee8679d5cf555f93cab4e62193e117a0d6
DEBU: 2016/04/19 19:18:53.266363 [net] ContainerStarted: 40b822726743a654ebe2765ebb549104d2d20ac90a87c17a59fae50a4abe1c65
DEBU: 2016/04/19 19:18:53.266363 [net] ContainerStarted: 40b822726743a654ebe2765ebb549104d2d20ac90a87c17a59fae50a4abe1c65
INFO: 2016/04/19 19:18:53.536062 [ipam] RequestAddress weave-10.32.0.0/12-10.32.0.0/12 <nil> map[]
DEBU: 2016/04/19 19:18:53.536148 weave POST to http://127.0.0.1:6784/ip/_/10.32.0.0/12 with map[]
DEBU: 2016/04/19 19:18:53.545610 [ipam] RequestAddress result 10.32.0.1/12
DEBU: 2016/04/19 19:18:53.546429 [net] CreateEndpoint &{NetworkID:14a5ff7efbc8528c3bbad06863abd2ee8679d5cf555f93cab4e62193e117a0d6 EndpointID:adc859ded150df380cf9a3a897d3050d593410fa45038184fe3aefc66e872932 Interface:0xc820269c20 Options:map[com.docker.network.endpoint.exposedports:[] com.docker.network.portmap:[]]}
INFO: 2016/04/19 19:18:53.546461 [net] CreateEndpoint adc859ded150df380cf9a3a897d3050d593410fa45038184fe3aefc66e872932
DEBU: 2016/04/19 19:18:53.546511 [net] CreateEndpoint &{Response:{Err:} Interface:<nil>}
DEBU: 2016/04/19 19:18:53.573604 [net] JoinEndpoint &{NetworkID:14a5ff7efbc8528c3bbad06863abd2ee8679d5cf555f93cab4e62193e117a0d6 EndpointID:adc859ded150df380cf9a3a897d3050d593410fa45038184fe3aefc66e872932 SandboxKey:/var/run/docker/netns/670a19229df7 Options:map[com.docker.network.endpoint.exposedports:[] com.docker.network.portmap:[]]}
INFO: 2016/04/19 19:18:53.573654 [net] JoinEndpoint 14a5ff7efbc8528c3bbad06863abd2ee8679d5cf555f93cab4e62193e117a0d6:adc859ded150df380cf9a3a897d3050d593410fa45038184fe3aefc66e872932 to /var/run/docker/netns/670a19229df7
DEBU: 2016/04/19 19:18:53.574995 [net] JoinEndpoint &{Response:{Err:} InterfaceName:0xc8202e85d0 Gateway: GatewayIPv6: StaticRoutes:[{Destination:224.0.0.0/4 RouteType:1 NextHop:}] DisableGatewayService:false}
DEBU: 2016/04/19 19:18:53.814220 [net] ContainerStarted: 97cc46f484101b2af749e3de5a34982a3d7b32bf2b3ab62336e1892bfe82ce12
DEBU: 2016/04/19 19:18:53.814220 [net] ContainerStarted: 97cc46f484101b2af749e3de5a34982a3d7b32bf2b3ab62336e1892bfe82ce12
DEBU: 2016/04/19 19:18:53.816522 weave PUT to http://127.0.0.1:6784/name/97cc46f484101b2af749e3de5a34982a3d7b32bf2b3ab62336e1892bfe82ce12/10.32.0.1 with map[fqdn:[a1.weave.local.]]
DEBU: 2016/04/19 19:18:54.156120 [net] ContainerStarted: 8570a92c4da69108435d27ff90d8fa7c620f578e4f04af11818457d2d002406c
DEBU: 2016/04/19 19:18:54.156138 [net] ContainerStarted: 8570a92c4da69108435d27ff90d8fa7c620f578e4f04af11818457d2d002406c
DEBU: 2016/04/19 19:19:49.576965 [net] LeaveEndpoint &{NetworkID:14a5ff7efbc8528c3bbad06863abd2ee8679d5cf555f93cab4e62193e117a0d6 EndpointID:adc859ded150df380cf9a3a897d3050d593410fa45038184fe3aefc66e872932}
INFO: 2016/04/19 19:19:49.577035 [net] LeaveEndpoint 14a5ff7efbc8528c3bbad06863abd2ee8679d5cf555f93cab4e62193e117a0d6:adc859ded150df380cf9a3a897d3050d593410fa45038184fe3aefc66e872932
DEBU: 2016/04/19 19:19:49.733979 [net] DeleteEndpoint &{NetworkID:14a5ff7efbc8528c3bbad06863abd2ee8679d5cf555f93cab4e62193e117a0d6 EndpointID:adc859ded150df380cf9a3a897d3050d593410fa45038184fe3aefc66e872932}
INFO: 2016/04/19 19:19:49.734024 [net] DeleteEndpoint adc859ded150df380cf9a3a897d3050d593410fa45038184fe3aefc66e872932
INFO: 2016/04/19 19:19:49.734572 [ipam] ReleaseAddress weave-10.32.0.0/12-10.32.0.0/12 10.32.0.1
DEBU: 2016/04/19 19:19:49.734619 weave DELETE to http://127.0.0.1:6784/ip/10.32.0.1 with map[]

Thank you.