moby: Daemon DNS Lookup Issues: no such host
Hello,
We’ve been seeing some strange issues with Ubuntu 14.04 and the 3.13 kernel on AWS. Occasionally pulls will start throwing errors like this(xxx is me redacting):
Error response from daemon: unable to ping registry endpoint https://xxx/v0/
v2 ping attempt failed with error: Get https://xxx/v2/: dial tcp: lookup xxx: no such host
v1 ping attempt failed with error: Get https://xxx/v1/_ping: dial tcp: lookup xxx: no such host
I was finally able to get it to repeat it about 8 times out of 10 on a host and turned this up via strace on the daemon:
[pid 19381] connect(218, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.0.0.2")}, 16) = -1 ENETUNREACH (Network is unreachable)
That IP address is the DNS server in my VPC. I spent about 30 minutes messing around with this and I can tell you 10.0.0.2 was 100% reachable. Not only was I able to connect to it via dig and other programs, the client was able to connect to it every time:
connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.0.0.2")}, 16) = 0
So here we have the client using the same call and being able to connect every time, while the daemon gets ENETUNREACH almost every time.
$uname -a
Linux staging-rails-app-1 3.13.0-71-generic #114-Ubuntu SMP Tue Dec 1 02:34:22 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$ docker version
Client:
Version: 1.9.1
API version: 1.21
Go version: go1.4.2
Git commit: a34a1d5
Built: Fri Nov 20 13:12:04 UTC 2015
OS/Arch: linux/amd64
About this issue
- Original URL
- State: open
- Created 9 years ago
- Reactions: 1
- Comments: 49 (7 by maintainers)
Adding the host to /etc/hosts seems to solve the issue, so something must be wrong with the docker/Go DNS lookup mechanism.
Still happens as of today, using Docker version 1.12.3, build 6b644ec
Solved it by doing this : $ VBoxManage list vms “default” {f13e52ac-22f6-4171-a48a-f3c25578c764} $ VBoxManage modifyvm f13e52ac-22f6-4171-a48a-f3c25578c651 --natdnshostresolver1 on
That did it …
Hi,
I can confirm the problem with
As @hookenz suggests in
https://github.com/docker/docker/issues/18842#issuecomment-187935887
in my case this seems related to the DNS response size.
Using
dig, with my usual resolver I haveMSG SIZE rcvd: 679, while with Google’s8.8.8.8I haveMSG SIZE rcvd: 271.I have then used
GODEBUG=netdns=cgoas suggested inhttps://github.com/docker/docker/issues/18842#issuecomment-189946305
and the issue is gone.
Does anyone know whether a later version of Docker fixes this or how safe it is to revert to the old behavior with
GODEBUG=netdns=cgo?Thanks!
I still get this happening on 1.12.2 and my colleague has it happen on 1.12.3 from time to time. Never got around to setting
GODEBUG=netdns=cgobut settingnameserver 8.8.8.8always works too.I also have this issue and have figured out a workaround.
docker version:
Docker version 1.10.1, build 9e83765docker info:
uname -a:
Linux xen 3.16.0-41-generic #57~14.04.1-Ubuntu SMP Thu Jun 18 18:01:13 UTC 2015 x86_64 x86_64 x86_64 GNU/LinuxDocker login fails. So does docker push.
Workaround: set the /etc/resolv.conf file to contain nameserver 8.8.8.8
Could it be the size of the DNS response message?
Change nameserver in /etc/resolv.conf to 8.8.8.8