kind: Kind clusters using ipvs kubeProxyMode doesn't work On v1.21.1
What happened:
Clusters using image kindest/node:v1.21.1
cannot resolve DNS names.
It works, for example, with kindest/node:v1.20.7
What you expected to happen:
DNS should work
How to reproduce it (as minimally and precisely as possible):
mnencia@sirio ➤ kind create cluster --image=kindest/node:v1.21.1
Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.21.1) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:
kubectl cluster-info --context kind-kind
Have a nice day! 👋
mnencia@sirio ➤ kubectl run -i -t busybox --image=busybox --restart=Never --rm
If you don't see a command prompt, try pressing enter.
/ # ping wwww.google.com
ping: bad address 'wwww.google.com'
/ #
Anything else we need to know?:
As a workaround, you can patch the CoreDNS configuration and restart the CoreDNS deployment
--- coredns.yaml.orig 2021-06-24 11:40:42.000000000 +0200
+++ coredns.yaml 2021-06-24 11:41:00.000000000 +0200
@@ -1,30 +1,30 @@
apiVersion: v1
data:
Corefile: |
.:53 {
errors
health {
lameduck 5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus :9153
- forward . /etc/resolv.conf {
+ forward . dns://1.1.1.1 {
max_concurrent 1000
}
cache 30
loop
reload
loadbalance
}
kind: ConfigMap
metadata:
creationTimestamp: "2021-06-24T09:28:09Z"
name: coredns
namespace: kube-system
resourceVersion: "238"
uid: c2def70f-5890-41cb-8184-b4dd33e8cece
mnencia@sirio ➤ kubectl apply -f coredns.yaml
Warning: resource configmaps/coredns is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
configmap/coredns configured
mnencia@sirio ➤ kubectl rollout restart -n kube-system deployment coredns
deployment.apps/coredns restarted
mnencia@sirio ➤ kubectl run -i -t busybox --image=busybox --restart=Never --rm
If you don't see a command prompt, try pressing enter.
/ # ping www.google.com
PING www.google.com (142.250.180.68): 56 data bytes
64 bytes from 142.250.180.68: seq=0 ttl=36 time=11.237 ms
64 bytes from 142.250.180.68: seq=1 ttl=36 time=59.229 ms
64 bytes from 142.250.180.68: seq=2 ttl=36 time=9.575 ms
64 bytes from 142.250.180.68: seq=3 ttl=36 time=10.221 ms
^C
--- www.google.com ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 9.575/22.565/59.229 ms
/ #
pod "busybox" deleted
Environment:
- kind version:
kind v0.11.1 go1.16.4 darwin/amd64
- Kubernetes version:
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-12T14:18:45Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-18T01:10:20Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}
- Docker version:
Client: Context: default Debug Mode: false Plugins: buildx: Build with BuildKit (Docker Inc., v0.5.1-docker) compose: Docker Compose (Docker Inc., 2.0.0-beta.3) scan: Docker Scan (Docker Inc., v0.8.0) Server: Containers: 2 Running: 2 Paused: 0 Stopped: 0 Images: 10 Server Version: 20.10.7 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc Default Runtime: runc Init Binary: docker-init containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7 init version: de40ad0 Security Options: seccomp Profile: default Kernel Version: 5.10.25-linuxkit Operating System: Docker Desktop OSType: linux Architecture: x86_64 CPUs: 6 Total Memory: 3.844GiB Name: docker-desktop ID: MVJ3:ZO5L:5OPX:VUGX:64A7:SHR5:3CUR:GS47:446R:UGPZ:Y2BS:2MRN Docker Root Dir: /var/lib/docker Debug Mode: true File Descriptors: 58 Goroutines: 57 System Time: 2021-06-24T09:37:41.819583449Z EventsListeners: 3 HTTP Proxy: http.docker.internal:3128 HTTPS Proxy: http.docker.internal:3128 Registry: https://index.docker.io/v1/ Labels: Experimental: true Insecure Registries: 127.0.0.0/8 Registry Mirrors: https://mirror.gcr.io/ Live Restore Enabled: false
- OS:
ProductName: macOS ProductVersion: 11.4 BuildVersion: 20F71
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 33 (20 by maintainers)
Commits related to this issue
- test: fix for Pod errors due to “too many open files” See https://kind.sigs.k8s.io/docs/user/known-issues/#pod-errors-due-to-too-many-open-files Also, remove workaround for https://github.com/kubern... — committed to cloudnative-pg/cloudnative-pg by mnencia 2 years ago
- test: fix for Pod errors due to “too many open files” See https://kind.sigs.k8s.io/docs/user/known-issues/#pod-errors-due-to-too-many-open-files Also, remove workaround for https://github.com/kubern... — committed to cloudnative-pg/cloudnative-pg by mnencia 2 years ago
- test: fix for Pod errors due to “too many open files” See https://kind.sigs.k8s.io/docs/user/known-issues/#pod-errors-due-to-too-many-open-files Also, remove workaround for https://github.com/kubern... — committed to cloudnative-pg/cloudnative-pg by mnencia 2 years ago
- test: fix for Pod errors due to “too many open files” See https://kind.sigs.k8s.io/docs/user/known-issues/#pod-errors-due-to-too-many-open-files Also, remove workaround for https://github.com/kubern... — committed to cloudnative-pg/cloudnative-pg by mnencia 2 years ago
Uff, you are right. I did it many times and I didn’t notice that this very instance had a typo. The result is the same.
It is the DNS that is set by docker. However, it get rewritten on the host by iptables.
I’ve tried both it with the default image (without specifying the version) and using the current one (sha256:f4eaad3d61dfe1228b43407dc9db980cd0a2e254588d133762750e605f5f368a)
Regarding the version of Docker Desktop, I’ve tried with 3.5.0, 3.4.0, 3.2.2, and 3.0.3. The result is always the same.