kubernetes: CrashLoopBackOff and confusing status
Pods are crashing as reported by:
$ kubectl get po
NAME READY STATUS RESTARTS AGE
elasticsearch-attiy 0/1 CrashLoopBackOff 25 1h
elasticsearch-lkl4i 0/1 CrashLoopBackOff 25 1h
frontend-81lqv 1/1 Running 0 1h
frontend-ddanc 1/1 Running 0 1h
But the message isn’t clear when trying to exec:
$ kubectl exec -it elasticsearch-attiy sh
error: Error executing remote command: Error executing command in container: container not found ("elasticsearch")
And the describe gives an even more confusion “2/2 replicas created” and it did find the image:
$ kubectl describe po elastic
search-attiy
Name: elasticsearch-attiy
Namespace: default
Image(s): eu.gcr.io/foo/foo_elasticsearch:latest
Node: gke-small-cluster-7834f89-node-wke5/10.210.0.1
Labels: name=elasticsearch
Status: Running
Reason:
Message:
IP: 10.180.1.11
Replication Controllers: elasticsearch (2/2 replicas created)
Containers:
elasticsearch:
Image: eu.gcr.io/foo/foo_elasticsearch:latest
State: Waiting
Reason: CrashLoopBackOff
Ready: False
Restart Count: 26
Conditions:
Type Status
Ready False
Events:
FirstSeen LastSeen Count From S
...
Wed, 18 Nov 2015 21:24:42 +0100 Wed, 18 Nov 2015 23:09:03 +0100 26 {kubelet gke-small-cluster-7834f89-node-wke5} spec.containers{elasticsearch} Pulled Successfully pulled image "eu.gcr.io/foo/foo_elasticsearch:latest"
Wed, 18 Nov 2015 23:09:03 +0100 Wed, 18 Nov 2015 23:09:03 +0100 1 {kubelet gke-small-cluster-7834f89-node-wke5} spec.containers{elasticsearch} Created Created with docker id a7429f37f133
Wed, 18 Nov 2015 23:09:04 +0100 Wed, 18 Nov 2015 23:09:04 +0100 1 {kubelet gke-small-cluster-7834f89-node-wke5} spec.containers{elasticsearch} Started Started with docker id a7429f37f133
It would help to have clearer messages to help understand causes of crashes.
The exec command could say that the given container isn’t running properly on the Pod (with a status or a command that could describe more in details the reasons).
The describe command here could clarify that replicas have been created but aren’t healthy. More importantly I found missing the reason of why the container got restarted. I do suspect that my command statement is the cause.
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Reactions: 14
- Comments: 15 (7 by maintainers)
The actually useful command to execute would have been:
I was having this problem, until I changed the FROM line in my Dockerfile from:
FROM microsoft/aspnetcore:2.0.0 … to … FROM microsoft/aspnetcore:2.0.5
(I’m running in a linux docker image. Not sure what it would be if windows)