kind: WSL2 ERROR: `kind create cluster` fails on Starting control-plane

What happened:

On WSL2 (Ubuntu-22.04), creation of kind cluster using:

kind create cluster

failed on:

ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged kind-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1

PLease note: It used to work fine last week!

Logs attached: 3165244420.zip

What you expected to happen:

Creating cluster โ€œkindโ€ โ€ฆ โœ“ Ensuring node image (kindest/node:v1.26.3) ๐Ÿ–ผ โœ“ Preparing nodes ๐Ÿ“ฆ ๐Ÿ“ฆ โœ“ Writing configuration ๐Ÿ“œ โœ“ Starting control-plane ๐Ÿ•น๏ธ โœ“ Installing CNI ๐Ÿ”Œ โœ“ Installing StorageClass ๐Ÿ’พ โœ“ Joining worker nodes ๐Ÿšœ Set kubectl context to โ€œkind-kindโ€ You can now use your cluster with:

kubectl cluster-info --context kind-kind

Thanks for using kind! ๐Ÿ˜Š

How to reproduce it (as minimally and precisely as possible):

kind create cluster

Anything else we need to know?:

Environment:

  • kind version: (use kind version): kind v0.18.0 go1.20.2 linux/amd64
  • Runtime info: (use docker info or podman info):

Client: Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.10.4 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.17.2 Path: /usr/libexec/docker/cli-plugins/docker-compose

Server: Containers: 65 Running: 3 Paused: 0 Stopped: 62 Images: 101 Server Version: 23.0.3 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: 2806fc1057397dbaeefbea0e4e17bddfbd388f38 runc version: v1.1.5-0-gf19387a init version: de40ad0 Security Options: seccomp Profile: builtin cgroupns Kernel Version: 5.15.90.1-microsoft-standard-WSL2 Operating System: Ubuntu 22.04.2 LTS OSType: linux Architecture: x86_64 CPUs: 14 Total Memory: 54.92GiB Name: w-PF42KZH6 ID: ITXK:IZJO:LRJS:ZZ5M:FYBZ:3ZJN:HRD7:AOLG:O7SA:SFZT:AZHX:7UDC Docker Root Dir: /var/lib/docker Debug Mode: false Registry: https://index.docker.io/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Default Address Pools: Base: 10.180.128.0/17, Size: 22

  • OS (e.g. from /etc/os-release):

PRETTY_NAME=โ€œUbuntu 22.04.2 LTSโ€ NAME=โ€œUbuntuโ€ VERSION_ID=โ€œ22.04โ€ VERSION=โ€œ22.04.2 LTS (Jammy Jellyfish)โ€ VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL=โ€œhttps://www.ubuntu.com/โ€ SUPPORT_URL=โ€œhttps://help.ubuntu.com/โ€ BUG_REPORT_URL=โ€œhttps://bugs.launchpad.net/ubuntu/โ€ PRIVACY_POLICY_URL=โ€œhttps://www.ubuntu.com/legal/terms-and-policies/privacy-policyโ€ UBUNTU_CODENAME=jammy

  • Kubernetes version: (use kubectl version):

Client Version: version.Info{Major:โ€œ1โ€, Minor:โ€œ24โ€, GitVersion:โ€œv1.24.3โ€, GitCommit:โ€œaef86a93758dc3cb2c658dd9657ab4ad4afc21cbโ€, GitTreeState:โ€œcleanโ€, BuildDate:โ€œ2022-07-13T14:30:46Zโ€, GoVersion:โ€œgo1.18.3โ€, Compiler:โ€œgcโ€, Platform:โ€œlinux/amd64โ€} Kustomize Version: v4.5.4 Server Version: version.Info{Major:โ€œ1โ€, Minor:โ€œ25โ€, GitVersion:โ€œv1.25.6โ€, GitCommit:โ€œff2c119726cc1f8926fb0585c74b25921e866a28โ€, GitTreeState:โ€œcleanโ€, BuildDate:โ€œ2023-01-18T19:15:26Zโ€, GoVersion:โ€œgo1.19.5โ€, Compiler:โ€œgcโ€, Platform:โ€œlinux/amd64โ€}

  • Any proxies or other special environment settings?:

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 1
  • Comments: 15 (8 by maintainers)

Most upvoted comments

From the node serial logs:

INFO: ensuring we can execute mount/umount even with userns-remap chown: cannot access โ€˜โ€™: No such file or directory

Uh, hmmm. Thatโ€™s an odd but familiar issue

Thanks,

"default-cgroupns-mode": "host"

in /etc/docker/daemon.json solved the problem, it seems.

After setting that value, restarting docker service, pruning docker system (just to make sure) and restarting WSL, kind cluster can successfully be created ๐Ÿ˜ƒ

$ kind create cluster
Creating cluster "kind" ...
 โœ“ Ensuring node image (kindest/node:v1.26.3) ๐Ÿ–ผ
 โœ“ Preparing nodes ๐Ÿ“ฆ
 โœ“ Writing configuration ๐Ÿ“œ
 โœ“ Starting control-plane ๐Ÿ•น๏ธ
 โœ“ Installing CNI ๐Ÿ”Œ
 โœ“ Installing StorageClass ๐Ÿ’พ
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Not sure what to do next? ๐Ÿ˜…  Check out https://kind.sigs.k8s.io/docs/user/quick-start/

Thatโ€™s, of course, much better solution and it works like a charm even with default --cgroupns=private!

Phew, ok, so it is just a bug in WSL ๐Ÿ˜… Thank you for confirming!

So to clarify: WSL2 1.2.0.0 has this and a fix is in 1.2.3/1.2.4 preview? I think we need to pin an issue outlining the situation and fix.

I was going to link #3165 (comment)

Thatโ€™s, of course, much better solution and it works like a charm even with default --cgroupns=private!

Thanks a lot!

Here is a solution i found to a similar error while trying to run โ€œkind create clusterโ€

I am running WSL2, but got the same error the op mentioned.

After some frustration trying to find a solution (and some help from ChatGPT), i was able to solve with the following process:


  1. Manually pull the Kindest Node image:
docker pull kindest/node:v1.27.3
Manually create the kind-control-plane container:

docker run -it --privileged --name kind-control-plane kindest/node:v1.27.3

Check the container logs:

docker logs kind-control-plane

If the container starts successfully, stop and remove it:

docker stop kind-control-plane
docker rm kind-control-plane

Retry creating the Kind cluster:

kind create cluster

If successful, check the logs:

docker logs kind-control-plane


The key point is was to manually create the kind-control-plane container.

Hope this helps.

Thank you! For now Iโ€™m pinning https://github.com/kubernetes-sigs/kind/issues/3180 to the top of the issue tracker outlining the situation and linking back to this and https://github.com/kubernetes-sigs/kind/issues/3165#issuecomment-1513665114

Thatโ€™s, of course, much better solution and it works like a charm even with default --cgroupns=private!

Phew, ok, so it is just a bug in WSL ๐Ÿ˜… Thank you for confirming!

So to clarify: WSL2 1.2.0.0 has this and a fix is in 1.2.3/1.2.4 preview? I think we need to pin an issue outlining the situation and fix.

Yes, after I ran wsl --update --pre-release, my WSL was updated from 1.2.0.0 to 1.2.4.0. Then I removed the workaround from /etc/docker/daemon.json, restarted docker and it works.