k3d: [BUG] Failed waiting for log message 'start worker processes' from node 'k3d-cluster3Bplus-serverlb'

What did you do

• How was the cluster created?

  • k3d cluster create cluster3Bplus

• What did you do afterwards?

  • Nothing

What did you expect to happen

Create without any errors !

Screenshots or terminal output

~ % k3d cluster create cluster3Bplus --trace
DEBU[0000] Selected runtime is 'docker.Docker'
DEBU[0000] API-Port Mapping didn't specify hostPort, choosing one randomly...
DEBU[0000] Got free port for API: '35307'
TRAC[0000] PortFilterMap: map[]
TRAC[0000] LabelFilterMap: map[]
DEBU[0000] '--update-default-kubeconfig set: enabling wait-for-server
INFO[0000] Created network 'k3d-cluster3Bplus'
INFO[0000] Created volume 'k3d-cluster3Bplus-images'
INFO[0001] Creating node 'k3d-cluster3Bplus-server-0'
TRAC[0001] Creating node from spec
&{Name:k3d-cluster3Bplus-server-0 Role:server Image:docker.io/rancher/k3s:v1.18.9-k3s1 Volumes:[k3d-cluster3Bplus-images:/k3d/images] Env:[K3S_TOKEN=kMdwgZZPGeacyOqeuBmc] Cmd:[] Args:[] Ports:[] Restart:false Labels:map[k3d.cluster:cluster3Bplus k3d.cluster.imageVolume:k3d-cluster3Bplus-images k3d.cluster.network:cb83c98f6b5d394d49c2ce90f107a24ff93e8ba64ec1a339f952d0c824373fc8 k3d.cluster.network.external:false k3d.cluster.token:kMdwgZZPGeacyOqeuBmc k3d.cluster.url:https://k3d-cluster3Bplus-server-0:6443] Network:cb83c98f6b5d394d49c2ce90f107a24ff93e8ba64ec1a339f952d0c824373fc8 ExtraHosts:[] ServerOpts:{IsInit:false ExposeAPI:{Host:0.0.0.0 HostIP:0.0.0.0 Port:35307}} AgentOpts:{} State:{Running:false Status:}}
TRAC[0001] Creating docker container with translated config
&{ContainerConfig:{Hostname:k3d-cluster3Bplus-server-0 Domainname: User: AttachStdin:false AttachStdout:false AttachStderr:false ExposedPorts:map[] Tty:false OpenStdin:false StdinOnce:false Env:[K3S_TOKEN=kMdwgZZPGeacyOqeuBmc K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml] Cmd:[server --tls-san 0.0.0.0] Healthcheck:<nil> ArgsEscaped:false Image:docker.io/rancher/k3s:v1.18.9-k3s1 Volumes:map[] WorkingDir: Entrypoint:[] NetworkDisabled:false MacAddress: OnBuild:[] Labels:map[app:k3d k3d.cluster:cluster3Bplus k3d.cluster.imageVolume:k3d-cluster3Bplus-images k3d.cluster.network:cb83c98f6b5d394d49c2ce90f107a24ff93e8ba64ec1a339f952d0c824373fc8 k3d.cluster.network.external:false k3d.cluster.token:kMdwgZZPGeacyOqeuBmc k3d.cluster.url:https://k3d-cluster3Bplus-server-0:6443 k3d.role:server k3d.server.api.host:0.0.0.0 k3d.server.api.hostIP:0.0.0.0 k3d.server.api.port:35307] StopSignal: StopTimeout:<nil> Shell:[]} HostConfig:{Binds:[k3d-cluster3Bplus-images:/k3d/images] ContainerIDFile: LogConfig:{Type: Config:map[]} NetworkMode: PortBindings:map[] RestartPolicy:{Name: MaximumRetryCount:0} AutoRemove:false VolumeDriver: VolumesFrom:[] CapAdd:[] CapDrop:[] Capabilities:[] CgroupnsMode: DNS:[] DNSOptions:[] DNSSearch:[] ExtraHosts:[] GroupAdd:[] IpcMode: Cgroup: Links:[] OomScoreAdj:0 PidMode: Privileged:true PublishAllPorts:false ReadonlyRootfs:false SecurityOpt:[] StorageOpt:map[] Tmpfs:map[/run: /var/run:] UTSMode: UsernsMode: ShmSize:0 Sysctls:map[] Runtime: ConsoleSize:[0 0] Isolation: Resources:{CPUShares:0 Memory:0 NanoCPUs:0 CgroupParent: BlkioWeight:0 BlkioWeightDevice:[] BlkioDeviceReadBps:[] BlkioDeviceWriteBps:[] BlkioDeviceReadIOps:[] BlkioDeviceWriteIOps:[] CPUPeriod:0 CPUQuota:0 CPURealtimePeriod:0 CPURealtimeRuntime:0 CpusetCpus: CpusetMems: Devices:[] DeviceCgroupRules:[] DeviceRequests:[] KernelMemory:0 KernelMemoryTCP:0 MemoryReservation:0 MemorySwap:0 MemorySwappiness:<nil> OomKillDisable:<nil> PidsLimit:<nil> Ulimits:[] CPUCount:0 CPUPercent:0 IOMaximumIOps:0 IOMaximumBandwidth:0} Mounts:[] MaskedPaths:[] ReadonlyPaths:[] Init:0x2406624} NetworkingConfig:{EndpointsConfig:map[cb83c98f6b5d394d49c2ce90f107a24ff93e8ba64ec1a339f952d0c824373fc8:0x242e060]}}
DEBU[0001] Created container k3d-cluster3Bplus-server-0 (ID: e7f0984f1cc3c5297be07a9225683417ab8cd8da35150e01d633397731b53853)
DEBU[0003] Created node 'k3d-cluster3Bplus-server-0'
DEBU[0003] Waiting for server node 'k3d-cluster3Bplus-server-0' to get ready
DEBU[0049] Finished waiting for log message 'k3s is up and running' from node 'k3d-cluster3Bplus-server-0'
INFO[0049] Creating LoadBalancer 'k3d-cluster3Bplus-serverlb'
TRAC[0049] Creating node from spec
&{Name:k3d-cluster3Bplus-serverlb Role:loadbalancer Image:docker.io/rancher/k3d-proxy:v3.1.5 Volumes:[] Env:[SERVERS=k3d-cluster3Bplus-server-0 PORTS=6443 WORKER_PROCESSES=1] Cmd:[] Args:[] Ports:[0.0.0.0:35307:6443/tcp] Restart:false Labels:map[app:k3d k3d.cluster:cluster3Bplus] Network:cb83c98f6b5d394d49c2ce90f107a24ff93e8ba64ec1a339f952d0c824373fc8 ExtraHosts:[] ServerOpts:{IsInit:false ExposeAPI:{Host: HostIP: Port:}} AgentOpts:{} State:{Running:false Status:}}
TRAC[0049] Creating docker container with translated config
&{ContainerConfig:{Hostname:k3d-cluster3Bplus-serverlb Domainname: User: AttachStdin:false AttachStdout:false AttachStderr:false ExposedPorts:map[6443/tcp:{}] Tty:false OpenStdin:false StdinOnce:false Env:[SERVERS=k3d-cluster3Bplus-server-0 PORTS=6443 WORKER_PROCESSES=1 K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml] Cmd:[] Healthcheck:<nil> ArgsEscaped:false Image:docker.io/rancher/k3d-proxy:v3.1.5 Volumes:map[] WorkingDir: Entrypoint:[] NetworkDisabled:false MacAddress: OnBuild:[] Labels:map[app:k3d k3d.cluster:cluster3Bplus k3d.role:loadbalancer] StopSignal: StopTimeout:<nil> Shell:[]} HostConfig:{Binds:[] ContainerIDFile: LogConfig:{Type: Config:map[]} NetworkMode: PortBindings:map[6443/tcp:[{HostIP:0.0.0.0 HostPort:35307}]] RestartPolicy:{Name: MaximumRetryCount:0} AutoRemove:false VolumeDriver: VolumesFrom:[] CapAdd:[] CapDrop:[] Capabilities:[] CgroupnsMode: DNS:[] DNSOptions:[] DNSSearch:[] ExtraHosts:[] GroupAdd:[] IpcMode: Cgroup: Links:[] OomScoreAdj:0 PidMode: Privileged:true PublishAllPorts:false ReadonlyRootfs:false SecurityOpt:[] StorageOpt:map[] Tmpfs:map[/run: /var/run:] UTSMode: UsernsMode: ShmSize:0 Sysctls:map[] Runtime: ConsoleSize:[0 0] Isolation: Resources:{CPUShares:0 Memory:0 NanoCPUs:0 CgroupParent: BlkioWeight:0 BlkioWeightDevice:[] BlkioDeviceReadBps:[] BlkioDeviceWriteBps:[] BlkioDeviceReadIOps:[] BlkioDeviceWriteIOps:[] CPUPeriod:0 CPUQuota:0 CPURealtimePeriod:0 CPURealtimeRuntime:0 CpusetCpus: CpusetMems: Devices:[] DeviceCgroupRules:[] DeviceRequests:[] KernelMemory:0 KernelMemoryTCP:0 MemoryReservation:0 MemorySwap:0 MemorySwappiness:<nil> OomKillDisable:<nil> PidsLimit:<nil> Ulimits:[] CPUCount:0 CPUPercent:0 IOMaximumIOps:0 IOMaximumBandwidth:0} Mounts:[] MaskedPaths:[] ReadonlyPaths:[] Init:0x56ba804} NetworkingConfig:{EndpointsConfig:map[cb83c98f6b5d394d49c2ce90f107a24ff93e8ba64ec1a339f952d0c824373fc8:0x56b87e0]}}

DEBU[0050] Created container k3d-cluster3Bplus-serverlb (ID: 0cd82d0dbf4f2ffd1bffe18690af6ce533488dfe73ba0e86ebac7e5596aeb54c)
DEBU[0051] Starting to wait for loadbalancer node 'k3d-cluster3Bplus-serverlb'
ERRO[0052] Failed waiting for log message 'start worker processes' from node 'k3d-cluster3Bplus-serverlb'
ERRO[0052] Failed to bring up all server nodes (and loadbalancer) in time. Check the logs:
ERRO[0052] >>> Node 'k3d-cluster3Bplus-serverlb' (container '0cd82d0dbf4f2ffd1bffe18690af6ce533488dfe73ba0e86ebac7e5596aeb54c') not running
ERRO[0052] Failed to bring up cluster
ERRO[0052] Failed to create cluster >>> Rolling Back
INFO[0052] Deleting cluster 'cluster3Bplus'
DEBU[0052] Cluster Details: &{Name:cluster3Bplus Network:{Name:cb83c98f6b5d394d49c2ce90f107a24ff93e8ba64ec1a339f952d0c824373fc8 External:false} Token:kMdwgZZPGeacyOqeuBmc Nodes:[0x21246e0 0x4e3c5a0] InitNode:<nil> ExternalDatastore:{Endpoint: CAFile: CertFile: KeyFile: Network:} CreateClusterOpts:0x2380330 ExposeAPI:{Host:0.0.0.0 HostIP:0.0.0.0 Port:35307} ServerLoadBalancer:0x2124640 ImageVolume:}
INFO[0052] Deleted k3d-cluster3Bplus-server-0
INFO[0052] Deleted k3d-cluster3Bplus-serverlb
INFO[0052] Deleting cluster network 'cb83c98f6b5d394d49c2ce90f107a24ff93e8ba64ec1a339f952d0c824373fc8'
FATA[0052] Cluster creation FAILED, all changes have been rolled back!

Which OS & Architecture

• Raspbian (10 / Buster) on raspberrypi 3B+ -> arm

Which version of k3d

k3d version v3.1.5
k3s version v1.18.9-k3s1 (default)

Which version of docker

• output of docker version and docker info

~ % docker version
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46
 Built:             Wed Sep 16 17:07:02 2020
 OS/Arch:           linux/arm
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46
  Built:            Wed Sep 16 17:00:52 2020
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
~ % docker info
Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 19.03.13
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fba4e9a7d01810a393d5d25a3621dc101981175
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.4.72-v7+
 Operating System: Raspbian GNU/Linux 10 (buster)
 OSType: linux
 Architecture: armv7l
 CPUs: 4
 Total Memory: 924.8MiB
 Name: raspberrypi3Bplus
 ID: I6NQ:UBMA:BYKN:IGA7:BME5:U3M3:77QV:XXIR:6FLS:FWAC:M2UY:CFT6
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No kernel memory TCP limit support
WARNING: No oom kill disable support