kind: Yet another "... Container * is not running" as version

Related to #1288

Microsoft Windows 10 Enterprise 10.0.18363 Build 18363

docker-machine.exe version 0.15.0, build b48dc28d

Docker version 17.06.0-ce, build 02c1d87

kind version 0.9.0

VirtualBox:

Linux default 4.14.154-boot2docker #1 SMP Thu Nov 14 19:19:08 UTC 2019 x86_64 GNU/Linux

Docker version 19.03.5, build 633a0ea838

CMD:

> kind create cluster -v 1 --retain
Creating cluster "kind" ...
 • Ensuring node image (kindest/node:v1.19.1) �  ...
DEBUG: docker/images.go:58] Image: kindest/node:v1.19.1@sha256:98cf5288864662e37115e362b23e4369c8c4a408f99cbc06e58ac30ddc721600 present locally
 ✓ Ensuring node image (kindest/node:v1.19.1) �
 • Preparing nodes �   ...
 ✓ Preparing nodes �
 • Writing configuration �  ...
 ✗ Writing configuration �
ERROR: failed to create cluster: failed to generate kubeadm config content: could not parse "Error response from daemon: Container cec3b6664d735d495ceabf1052810508e955a226c27223f1a220ff1719c114eb is not running" as version

Stack Trace:
sigs.k8s.io/kind/pkg/errors.Wrap
        C:/Users/stefanz/go/pkg/mod/sigs.k8s.io/kind@v0.9.0/pkg/errors/errors.go:39
sigs.k8s.io/kind/pkg/cluster/internal/create/actions/config.(*Action).Execute.func1.1
        C:/Users/stefanz/go/pkg/mod/sigs.k8s.io/kind@v0.9.0/pkg/cluster/internal/create/actions/config/config.go:85
sigs.k8s.io/kind/pkg/errors.UntilErrorConcurrent.func1
        C:/Users/stefanz/go/pkg/mod/sigs.k8s.io/kind@v0.9.0/pkg/errors/concurrent.go:30
runtime.goexit
        c:/go/src/runtime/asm_amd64.s:1374

Unfortunately, the related /mnt/sda1/var/lib/docker/containers/cec3b6664d735d495ceabf1052810508e955a226c27223f1a220ff1719c114eb/ cec3b6664d735d495ceabf1052810508e955a226c27223f1a220ff1719c114eb-json.log is empty

How to find any detailed information about what went wrong?

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 16 (12 by maintainers)

Most upvoted comments

The error field of inspecting the container has the issue.

https://github.com/microsoft/WSL/issues/4189#issuecomment-518277265

https://github.com/docker/for-linux/issues/219

+1
BenTheElder on Oct 2, 2020

Thanks @BenTheElder for the quick reply!

What I actually meant is: Related to https://github.com/kubernetes-sigs/kind/issues/1288#issuecomment-631677585 😉

docker logs kind-control-plane has empty output, and all the .log files created by kind export logs only contain:

Error response from daemon: Container cec3b6664d735d495ceabf1052810508e955a226c27223f1a220ff1719c114eb is not running

Can I run the kind-control-plane container manually in some debug mode to get the missing logging data?

docker-info.txt contains:

Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 52
Server Version: 19.03.5
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
init version: fec3683
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.14.154-boot2docker
Operating System: Boot2Docker 19.03.5 (TCL 10.1)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.67GiB
Name: default
ID: 4B4L:7WVK:UP3R:ZPOS:ZATA:SUB5:J7JX:S7LJ:ITGK:Q3XG:JXNW:XLFC
Docker Root Dir: /mnt/sda1/var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

inspect.json contains:

[
    {
        "Id": "cec3b6664d735d495ceabf1052810508e955a226c27223f1a220ff1719c114eb",
        "Created": "2020-10-01T18:57:03.506215881Z",
        "Path": "/usr/local/bin/entrypoint",
        "Args": [
            "/sbin/init"
        ],
        "State": {
            "Status": "created",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 128,
            "Error": "cgroups: cannot find cgroup mount destination: unknown",
            "StartedAt": "0001-01-01T00:00:00Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:37ddbc9063d2ae1a9587f79fd184bef370fd49bb853cf0521aef6ddd17399c99",
        "ResolvConfPath": "/mnt/sda1/var/lib/docker/containers/cec3b6664d735d495ceabf1052810508e955a226c27223f1a220ff1719c114eb/resolv.conf",
        "HostnamePath": "/mnt/sda1/var/lib/docker/containers/cec3b6664d735d495ceabf1052810508e955a226c27223f1a220ff1719c114eb/hostname",
        "HostsPath": "/mnt/sda1/var/lib/docker/containers/cec3b6664d735d495ceabf1052810508e955a226c27223f1a220ff1719c114eb/hosts",
        "LogPath": "/mnt/sda1/var/lib/docker/containers/cec3b6664d735d495ceabf1052810508e955a226c27223f1a220ff1719c114eb/cec3b6664d735d495ceabf1052810508e955a226c27223f1a220ff1719c114eb-json.log",
        "Name": "/kind-control-plane",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/lib/modules:/lib/modules:ro"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "kind",
            "PortBindings": {
                "6443/tcp": [
                    {
                        "HostIp": "127.0.0.1",
                        "HostPort": "57765"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "on-failure",
                "MaximumRetryCount": 1
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Capabilities": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "shareable",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": true,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "seccomp=unconfined",
                "apparmor=unconfined",
                "label=disable"
            ],
            "Tmpfs": {
                "/run": "",
                "/tmp": ""
            },
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": null,
            "ReadonlyPaths": null
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/mnt/sda1/var/lib/docker/overlay2/3992cde94ea8cf3b2b7c8a578e346aa75d6eab06f9ecc4831c1dd2a2c81902bc-init/diff:/mnt/sda1/var/lib/docker/overlay2/7d69fe6ec2e1c537a4a35a95aae3f1fd3828bdaf760e9701a9f2a1004df32e51/diff:/mnt/sda1/var/lib/docker/overlay2/30e3098feffba1be8a684e076e0537febaa93c3c252cc6c39432bf138e6b125a/diff:/mnt/sda1/var/lib/docker/overlay2/87fa1831b77d25891034274d80b39bc4c0c5dc23adbe8d09f470b905884fdcd7/diff:/mnt/sda1/var/lib/docker/overlay2/4fc1e5d28eb9d9432d98c5e9503885ea894e5efb4cbf96c49def2357dcfa1f7a/diff:/mnt/sda1/var/lib/docker/overlay2/3c61016116368449f4d988c8cf2dab542a885ceb6eb2e063f9679851b7d602ae/diff:/mnt/sda1/var/lib/docker/overlay2/217edbc4da74c24a08729a80dedb3e59abc8718346188aef03e00ba9312d138b/diff:/mnt/sda1/var/lib/docker/overlay2/487af58ea9f453c48c8d08144b4271cd4a37de37e8a2c698efd04b7fae826253/diff",
                "MergedDir": "/mnt/sda1/var/lib/docker/overlay2/3992cde94ea8cf3b2b7c8a578e346aa75d6eab06f9ecc4831c1dd2a2c81902bc/merged",
                "UpperDir": "/mnt/sda1/var/lib/docker/overlay2/3992cde94ea8cf3b2b7c8a578e346aa75d6eab06f9ecc4831c1dd2a2c81902bc/diff",
                "WorkDir": "/mnt/sda1/var/lib/docker/overlay2/3992cde94ea8cf3b2b7c8a578e346aa75d6eab06f9ecc4831c1dd2a2c81902bc/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "volume",
                "Name": "e85862a93d0ccd42dfe6cae95025561d890e09e4e661f398d219d5e22d07f418",
                "Source": "/mnt/sda1/var/lib/docker/volumes/e85862a93d0ccd42dfe6cae95025561d890e09e4e661f398d219d5e22d07f418/_data",
                "Destination": "/var",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "/lib/modules",
                "Destination": "/lib/modules",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "kind-control-plane",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "6443/tcp": {}
            },
            "Tty": true,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "container=docker"
            ],
            "Cmd": null,
            "Image": "kindest/node:v1.19.1@sha256:98cf5288864662e37115e362b23e4369c8c4a408f99cbc06e58ac30ddc721600",
            "Volumes": {
                "/var": {}
            },
            "WorkingDir": "",
            "Entrypoint": [
                "/usr/local/bin/entrypoint",
                "/sbin/init"
            ],
            "OnBuild": null,
            "Labels": {
                "io.x-k8s.kind.cluster": "kind",
                "io.x-k8s.kind.role": "control-plane"
            },
            "StopSignal": "SIGRTMIN+3"
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "2194745dd8c4218681497e8dc54f0bcb2c0fa01857d7b48a3b387378922fd79a",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "6443/tcp": [
                    {
                        "HostIp": "127.0.0.1",
                        "HostPort": "57765"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/2194745dd8c4",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "kind": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "cec3b6664d73"
                    ],
                    "NetworkID": "54a89324fb73656d1a0cf8f00dc83fcf9c93c53d42d11e264578e7eb97cf8afe",
                    "EndpointID": "ab0f51ebed00b3889f0cb0bb625ceea2b5b666b6dc8e2187545ddf61fbe51b1e",
                    "Gateway": "172.19.0.1",
                    "IPAddress": "172.19.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "fc00:f853:ccd:e793::1",
                    "GlobalIPv6Address": "fc00:f853:ccd:e793::2",
                    "GlobalIPv6PrefixLen": 64,
                    "MacAddress": "02:42:ac:13:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]
+1
purplezimmermann on Oct 2, 2020
Read more comments on GitHub
← kind: [WSL2] Sync failed errors in kube-proxy for Service with SessionAffinity: ClientIP
kind: [zfs] Failed to create cluster →