minikube: hyperkit conflict with local DNS daemon (dns: read udp src->dst:53: read: connection refused)

Minikube version (use minikube version): v0.33.1

OS (e.g. from /etc/os-release): macOS Mojave version 10.14.2
VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): hyperkit
ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION): minikube-v0.33.1.iso
Install tools: brew
Others:

Command “minkube start --vm-driver hyperkit” fails with the following messages:

Starting local Kubernetes v1.13.2 cluster… Starting VM… Getting VM IP address… E0123 16:14:34.829156 21704 start.go:243] Error parsing version semver: Version string empty Moving files into cluster… Setting up certs… Connecting to cluster… Setting up kubeconfig… Stopping extra container runtimes… Starting cluster components… E0123 16:14:37.615834 21704 start.go:376] Error starting cluster: kubeadm init: sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable–etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable–data-minikube --ignore-preflight-errors=Port-10250 --ignore-preflight-errors=FileAvailable–etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable–etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable–etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable–etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI

[init] Using Kubernetes version: v1.13.2 [preflight] Running pre-flight checks [WARNING Swap]: running with swap on is not supported. Please disable swap [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using ‘kubeadm config images pull’ error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-apiserver:v1.13.2: output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 192.168.64.1:53: read udp 192.168.64.7:42770->192.168.64.1:53: read: connection refused , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-controller-manager:v1.13.2: output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 192.168.64.1:53: read udp 192.168.64.7:45819->192.168.64.1:53: read: connection refused , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-scheduler:v1.13.2: output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 192.168.64.1:53: read udp 192.168.64.7:46142->192.168.64.1:53: read: connection refused , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-proxy:v1.13.2: output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 192.168.64.1:53: read udp 192.168.64.7:36363->192.168.64.1:53: read: connection refused , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/pause:3.1: output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 192.168.64.1:53: read udp 192.168.64.7:37734->192.168.64.1:53: read: connection refused , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/etcd:3.2.24: output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 192.168.64.1:53: read udp 192.168.64.7:35622->192.168.64.1:53: read: connection refused , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/coredns:1.2.6: output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 192.168.64.1:53: read udp 192.168.64.7:58397->192.168.64.1:53: read: connection refused , error: exit status 1 [preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...

: Process exited with status 1

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 3
Comments: 17 (6 by maintainers)

Most upvoted comments

I was running into this same issue because I was running dnsmasq. sudo brew services stop dnsmasq solved things for me.

mhumeSF on Mar 31, 2019

Ok, so I cannot figure out how to add the IP (I am assuming Minikube IP) to dnscrypt-proxy.

Here’s what I am seeing:

sudo lsof -n -i :53

COMMAND     PID   USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
dnscrypt- 85639 nobody   62u  IPv4 0x7507542b71a3d337      0t0  UDP 127.0.0.1:domain
dnscrypt- 85639 nobody   63u  IPv4 0x7507542b5c25c5ff      0t0  TCP 127.0.0.1:domain (LISTEN)

Then I did this…

pstree | grep -i 85639
 | |   \--- 85639 nobody /opt/cisco/anyconnect/bin/dnscrypt-proxy --user nobody --local-address=127.0.0.1:53 --plugin=/opt/cisco/anyconnect/lib/libdcplugin_erc.so -d 0101A75A568AB7E5
 |       \--- 14399 anasharm grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn -i 85639

Not sure where specifically do I go to “add the IP”. Also, since the dnscrypt here was installed by Cisco AnyConnect, not sure if I have a lot of wiggle room.

Thoughts?

indrayam on May 29, 2019

Adding the IP to dnscrypt-proxy solved it, it seems. I feel stupid. I hope that this might help others. Also, maybe it should have been added to the hyperkit page. I’ll see if I submit a PR for it. 😄

MindTooth on May 29, 2019

This seems to also occur with DNSCrypt Proxy as well. Would love for a way to make minikube work with DNSMasq, DNSCrypt Proxy, etc.

MindTooth on May 19, 2019

My interpretation of this message is that the hyperkit DNS forwarder isn’t available, possibly due to a port conflict, perhaps with another DNS daemon. If you run into this, do you mind sharing the output of:

sudo lsof -ni:53

I’ve tried to search for documentation on how hyperkit’s DNS implementation works, but haven’t come up with much: it all seems to be VPNKit magic. Anyone have any tips?

tstromberg on Mar 28, 2019

It did not work for me.

Hyperkit version:

hyperkit -v
hyperkit: 0.20180403

Homepage: https://github.com/docker/hyperkit
License: BSD

MacOS version: 10.14.5

minikube profile knative

minikube start -p knative --memory=8192 --cpus=4 \
  --kubernetes-version=v1.12.0 \
  --vm-driver=hyperkit \
  --disk-size=50g \
  --extra-config=apiserver.enable-admission-plugins="LimitRanger,NamespaceExists,NamespaceLifecycle,ResourceQuota,ServiceAccount,DefaultStorageClass,MutatingAdmissionWebhook"

✅  minikube profile was successfully set to knative
😄  minikube v1.1.0 on darwin (amd64)
💿  Downloading Minikube ISO ...
 131.28 MB / 131.28 MB [============================================] 100.00% 0s
🔥  Creating hyperkit VM (CPUs=4, Memory=8192MB, Disk=50000MB) ...
🐳  Configuring environment for Kubernetes v1.12.0 on Docker 18.09.6
    ▪ apiserver.enable-admission-plugins=LimitRanger,NamespaceExists,NamespaceLifecycle,ResourceQuota,ServiceAccount,DefaultStorageClass,MutatingAdmissionWebhook
💾  Downloading kubeadm v1.12.0
💾  Downloading kubelet v1.12.0
🚜  Pulling images ...
❌  Unable to pull images, which may be OK: running cmd: sudo kubeadm config images pull --config /var/lib/kubeadm.yaml: command failed: sudo kubeadm config images pull --config /var/lib/kubeadm.yaml
stdout:
stderr: failed to pull image "k8s.gcr.io/kube-apiserver:v1.12.0": output: Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 192.168.64.1:53: read udp 192.168.64.4:37133->192.168.64.1:53: read: connection refused
, error: exit status 1
: Process exited with status 1
🚀  Launching Kubernetes ...
⌛  Verifying: apiserver proxy etcd scheduler controller dns
🏄  Done! kubectl is now configured to use "knative"

When I run mk ssh and do a nslookup k8s.gcr.io, it fails.

Here’s my lsof output:

sudo lsof -n -i :53

COMMAND     PID   USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
dnscrypt- 24368 nobody   62u  IPv4 0x7507542b6aa22d77      0t0  UDP 127.0.0.1:domain
dnscrypt- 24368 nobody   63u  IPv4 0x7507542b845a12ff      0t0  TCP 127.0.0.1:domain (LISTEN)

indrayam on May 29, 2019