minikube: none: coredns CrashLoopBackOff: dial tcp ip:443: connect: no route to host
The exact command to reproduce the issue:
kubectl get pods --all-namespaces
The full output of the command that failed:
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-fb8b8dccf-tn8vz 0/1 CrashLoopBackOff 4 110s
kube-system coredns-fb8b8dccf-z28dc 0/1 CrashLoopBackOff 4 110s
kube-system etcd-minikube 1/1 Running 0 58s
kube-system kube-addon-manager-minikube 1/1 Running 0 48s
kube-system kube-apiserver-minikube 1/1 Running 0 52s
kube-system kube-controller-manager-minikube 1/1 Running 0 41s
kube-system kube-proxy-wb9bj 1/1 Running 0 110s
kube-system kube-scheduler-minikube 1/1 Running 0 41s
kube-system storage-provisioner 1/1 Running 0 109s
The output of the minikube logs command:
[root@fabsnuc ~]# minikube logs
==> coredns <==
E0524 17:47:43.607851 1 reflector.go:134] github.com/coredns/coredns/plugin/kubernetes/controller.go:322: Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: connect: no route to host
E0524 17:47:43.607851 1 reflector.go:134] github.com/coredns/coredns/plugin/kubernetes/controller.go:322: Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: connect: no route to host
log: exiting because of error: log: cannot create log: open /tmp/coredns.coredns-fb8b8dccf-tn8vz.unknownuser.log.ERROR.20190524-174743.1: no such file or directory
==> dmesg <==
dmesg: invalid option -- '='
Usage:
dmesg [options]
Options:
-C, --clear clear the kernel ring buffer
-c, --read-clear read and clear all messages
-D, --console-off disable printing messages to console
-d, --show-delta show time delta between printed messages
-e, --reltime show local time and time delta in readable format
-E, --console-on enable printing messages to console
-F, --file <file> use the file instead of the kernel log buffer
-f, --facility <list> restrict output to defined facilities
-H, --human human readable output
-k, --kernel display kernel messages
-L, --color colorize messages
-l, --level <list> restrict output to defined levels
-n, --console-level <level> set level of messages printed to console
-P, --nopager do not pipe output into a pager
-r, --raw print the raw message buffer
-S, --syslog force to use syslog(2) rather than /dev/kmsg
-s, --buffer-size <size> buffer size to query the kernel ring buffer
-T, --ctime show human readable timestamp (could be
inaccurate if you have used SUSPEND/RESUME)
-t, --notime don't print messages timestamp
-u, --userspace display userspace messages
-w, --follow wait for new messages
-x, --decode decode facility and level to readable string
-h, --help display this help and exit
-V, --version output version information and exit
Supported log facilities:
kern - kernel messages
user - random user-level messages
mail - mail system
daemon - system daemons
auth - security/authorization messages
syslog - messages generated internally by syslogd
lpr - line printer subsystem
news - network news subsystem
Supported log levels (priorities):
emerg - system is unusable
alert - action must be taken immediately
crit - critical conditions
err - error conditions
warn - warning conditions
notice - normal but significant condition
info - informational
debug - debug-level messages
For more details see dmesg(q).
==> kernel <==
12:48:49 up 12:26, 3 users, load average: 0.21, 0.23, 0.20
Linux fabsnuc.intel.com 3.10.0-957.12.2.el7.x86_64 #1 SMP Tue May 14 21:24:32 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
==> kube-addon-manager <==
WRN: == Error getting default service account, retry in 0.5 second ==
Error from server (NotFound): serviceaccounts "default" not found
WRN: == Error getting default service account, retry in 0.5 second ==
Error from server (NotFound): serviceaccounts "default" not found
WRN: == Error getting default service account, retry in 0.5 second ==
Error from server (NotFound): serviceaccounts "default" not found
WRN: == Error getting default service account, retry in 0.5 second ==
Error from server (NotFound): serviceaccounts "default" not found
WRN: == Error getting default service account, retry in 0.5 second ==
INFO: == Default service account in the kube-system namespace has token default-token-qb9ck ==
find: '/etc/kubernetes/admission-controls': No such file or directory
INFO: == Entering periodical apply loop at 2019-05-24T17:44:29+00:00 ==
INFO: Leader is fabsnuc.intel.com
clusterrolebinding.rbac.authorization.k8s.io/storage-provisioner created
storageclass.storage.k8s.io/standard created
INFO: == Kubernetes addon ensure completed at 2019-05-24T17:44:29+00:00 ==
INFO: == Reconciling with deprecated label ==
error: no objects passed to apply
INFO: == Reconciling with addon-manager label ==
serviceaccount/storage-provisioner created
pod/storage-provisioner created
INFO: == Kubernetes addon reconcile completed at 2019-05-24T17:44:32+00:00 ==
INFO: Leader is fabsnuc.intel.com
INFO: == Kubernetes addon ensure completed at 2019-05-24T17:45:29+00:00 ==
INFO: == Reconciling with deprecated label ==
error: no objects passed to apply
INFO: == Reconciling with addon-manager label ==
serviceaccount/storage-provisioner unchanged
INFO: == Kubernetes addon reconcile completed at 2019-05-24T17:45:30+00:00 ==
INFO: Leader is fabsnuc.intel.com
INFO: == Kubernetes addon ensure completed at 2019-05-24T17:46:29+00:00 ==
INFO: == Reconciling with deprecated label ==
error: no objects passed to apply
INFO: == Reconciling with addon-manager label ==
serviceaccount/storage-provisioner unchanged
INFO: == Kubernetes addon reconcile completed at 2019-05-24T17:46:31+00:00 ==
INFO: Leader is fabsnuc.intel.com
INFO: == Kubernetes addon ensure completed at 2019-05-24T17:47:29+00:00 ==
INFO: == Reconciling with deprecated label ==
error: no objects passed to apply
INFO: == Reconciling with addon-manager label ==
serviceaccount/storage-provisioner unchanged
INFO: == Kubernetes addon reconcile completed at 2019-05-24T17:47:30+00:00 ==
INFO: Leader is fabsnuc.intel.com
INFO: == Kubernetes addon ensure completed at 2019-05-24T17:48:29+00:00 ==
INFO: == Reconciling with deprecated label ==
error: no objects passed to apply
INFO: == Reconciling with addon-manager label ==
serviceaccount/storage-provisioner unchanged
INFO: == Kubernetes addon reconcile completed at 2019-05-24T17:48:31+00:00 ==
==> kube-apiserver <==
I0524 17:44:20.171744 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:node
I0524 17:44:20.213052 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:attachdetach-controller
I0524 17:44:20.252888 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:clusterrole-aggregation-controller
I0524 17:44:20.293107 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:cronjob-controller
I0524 17:44:20.333042 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:daemon-set-controller
I0524 17:44:20.373102 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:deployment-controller
I0524 17:44:20.413045 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:disruption-controller
I0524 17:44:20.453050 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:endpoint-controller
I0524 17:44:20.493174 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:expand-controller
I0524 17:44:20.532906 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:generic-garbage-collector
I0524 17:44:20.572937 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:horizontal-pod-autoscaler
I0524 17:44:20.613066 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:job-controller
I0524 17:44:20.652852 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:namespace-controller
I0524 17:44:20.693204 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:node-controller
I0524 17:44:20.731879 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:persistent-volume-binder
I0524 17:44:20.773076 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:pod-garbage-collector
I0524 17:44:20.816392 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:replicaset-controller
I0524 17:44:20.852923 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:replication-controller
I0524 17:44:21.145954 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:resourcequota-controller
I0524 17:44:21.149326 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:route-controller
I0524 17:44:21.152939 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:service-account-controller
I0524 17:44:21.156458 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:service-controller
I0524 17:44:21.159898 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:statefulset-controller
I0524 17:44:21.163538 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:ttl-controller
I0524 17:44:21.167086 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:certificate-controller
I0524 17:44:21.172823 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:pvc-protection-controller
I0524 17:44:21.212745 1 storage_rbac.go:223] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:pv-protection-controller
I0524 17:44:21.251844 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0524 17:44:21.253898 1 storage_rbac.go:254] created role.rbac.authorization.k8s.io/extension-apiserver-authentication-reader in kube-system
I0524 17:44:21.292135 1 storage_rbac.go:254] created role.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-system
I0524 17:44:21.331692 1 storage_rbac.go:254] created role.rbac.authorization.k8s.io/system:controller:cloud-provider in kube-system
I0524 17:44:21.372887 1 storage_rbac.go:254] created role.rbac.authorization.k8s.io/system:controller:token-cleaner in kube-system
I0524 17:44:21.388219 1 controller.go:606] quota admission added evaluator for: endpoints
I0524 17:44:21.412944 1 storage_rbac.go:254] created role.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager in kube-system
I0524 17:44:21.453044 1 storage_rbac.go:254] created role.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler in kube-system
I0524 17:44:21.495019 1 storage_rbac.go:254] created role.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-public
I0524 17:44:21.531542 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
I0524 17:44:21.533395 1 storage_rbac.go:284] created rolebinding.rbac.authorization.k8s.io/system::extension-apiserver-authentication-reader in kube-system
I0524 17:44:21.573147 1 storage_rbac.go:284] created rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager in kube-system
I0524 17:44:21.613109 1 storage_rbac.go:284] created rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler in kube-system
I0524 17:44:21.653065 1 storage_rbac.go:284] created rolebinding.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-system
I0524 17:44:21.692961 1 storage_rbac.go:284] created rolebinding.rbac.authorization.k8s.io/system:controller:cloud-provider in kube-system
I0524 17:44:21.732967 1 storage_rbac.go:284] created rolebinding.rbac.authorization.k8s.io/system:controller:token-cleaner in kube-system
I0524 17:44:21.773047 1 storage_rbac.go:284] created rolebinding.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-public
W0524 17:44:21.791574 1 lease.go:222] Resetting endpoints for master service "kubernetes" to [192.168.0.159]
I0524 17:44:22.361003 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0524 17:44:22.965355 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0524 17:44:23.308886 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0524 17:44:29.163799 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0524 17:44:29.282538 1 controller.go:606] quota admission added evaluator for: replicasets.apps
==> kube-proxy <==
W0524 17:44:30.645444 1 proxier.go:498] Failed to load kernel module ip_vs_rr with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
W0524 17:44:30.651474 1 server_others.go:267] Flag proxy-mode="" unknown, assuming iptables proxy
I0524 17:44:30.656952 1 server_others.go:146] Using iptables Proxier.
W0524 17:44:30.657029 1 proxier.go:319] clusterCIDR not specified, unable to distinguish between internal and external traffic
I0524 17:44:30.657135 1 server.go:562] Version: v1.14.2
I0524 17:44:30.669112 1 conntrack.go:52] Setting nf_conntrack_max to 262144
I0524 17:44:30.669356 1 config.go:202] Starting service config controller
I0524 17:44:30.669383 1 controller_utils.go:1027] Waiting for caches to sync for service config controller
I0524 17:44:30.670178 1 config.go:102] Starting endpoints config controller
I0524 17:44:30.670192 1 controller_utils.go:1027] Waiting for caches to sync for endpoints config controller
I0524 17:44:30.769721 1 controller_utils.go:1034] Caches are synced for service config controller
I0524 17:44:30.770421 1 controller_utils.go:1034] Caches are synced for endpoints config controller
==> kube-scheduler <==
I0524 17:44:15.952574 1 serving.go:319] Generated self-signed cert in-memory
W0524 17:44:16.280112 1 authentication.go:249] No authentication-kubeconfig provided in order to lookup client-ca-file in configmap/extension-apiserver-authentication in kube-system, so client certificate authentication won't work.
W0524 17:44:16.280124 1 authentication.go:252] No authentication-kubeconfig provided in order to lookup requestheader-client-ca-file in configmap/extension-apiserver-authentication in kube-system, so request-header client certificate authentication won't work.
W0524 17:44:16.280153 1 authorization.go:146] No authorization-kubeconfig provided, so SubjectAccessReview of authorization tokens won't work.
I0524 17:44:16.281836 1 server.go:142] Version: v1.14.2
I0524 17:44:16.281876 1 defaults.go:87] TaintNodesByCondition is enabled, PodToleratesNodeTaints predicate is mandatory
W0524 17:44:16.283073 1 authorization.go:47] Authorization is disabled
W0524 17:44:16.283082 1 authentication.go:55] Authentication is disabled
I0524 17:44:16.283093 1 deprecated_insecure_serving.go:49] Serving healthz insecurely on [::]:10251
I0524 17:44:16.283369 1 secure_serving.go:116] Serving securely on 127.0.0.1:10259
E0524 17:44:18.410541 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0524 17:44:18.410618 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0524 17:44:18.411874 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0524 17:44:18.414829 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0524 17:44:18.414876 1 reflector.go:126] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:223: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0524 17:44:18.426899 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0524 17:44:18.426920 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0524 17:44:18.427736 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0524 17:44:18.428376 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0524 17:44:18.428377 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0524 17:44:19.411860 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0524 17:44:19.412766 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0524 17:44:19.414768 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0524 17:44:19.415861 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0524 17:44:19.423611 1 reflector.go:126] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:223: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0524 17:44:19.428104 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0524 17:44:19.429304 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0524 17:44:19.430785 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0524 17:44:19.432002 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0524 17:44:19.432981 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
I0524 17:44:21.284704 1 controller_utils.go:1027] Waiting for caches to sync for scheduler controller
I0524 17:44:21.384829 1 controller_utils.go:1034] Caches are synced for scheduler controller
I0524 17:44:21.384968 1 leaderelection.go:217] attempting to acquire leader lease kube-system/kube-scheduler...
I0524 17:44:21.389935 1 leaderelection.go:227] successfully acquired lease kube-system/kube-scheduler
==> kubelet <==
-- Logs begin at Fri 2019-05-24 00:22:03 CDT, end at Fri 2019-05-24 12:48:49 CDT. --
May 24 12:44:33 fabsnuc.intel.com kubelet[26430]: E0524 12:44:33.120690 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 10s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:44:34 fabsnuc.intel.com kubelet[26430]: E0524 12:44:34.147984 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 10s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:44:34 fabsnuc.intel.com kubelet[26430]: E0524 12:44:34.159168 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 10s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:44:35 fabsnuc.intel.com kubelet[26430]: E0524 12:44:35.176560 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 10s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:44:36 fabsnuc.intel.com kubelet[26430]: E0524 12:44:36.125943 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 10s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:44:38 fabsnuc.intel.com kubelet[26430]: E0524 12:44:38.222372 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 10s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:44:52 fabsnuc.intel.com kubelet[26430]: E0524 12:44:52.325022 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:44:53 fabsnuc.intel.com kubelet[26430]: E0524 12:44:53.345496 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 20s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:44:56 fabsnuc.intel.com kubelet[26430]: E0524 12:44:56.126216 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 20s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:44:58 fabsnuc.intel.com kubelet[26430]: E0524 12:44:58.222897 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:09 fabsnuc.intel.com kubelet[26430]: E0524 12:45:09.417015 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 20s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:09 fabsnuc.intel.com kubelet[26430]: E0524 12:45:09.417037 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:22 fabsnuc.intel.com kubelet[26430]: E0524 12:45:22.754070 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:22 fabsnuc.intel.com kubelet[26430]: E0524 12:45:22.776061 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:23 fabsnuc.intel.com kubelet[26430]: E0524 12:45:23.792888 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:23 fabsnuc.intel.com kubelet[26430]: E0524 12:45:23.804715 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:26 fabsnuc.intel.com kubelet[26430]: E0524 12:45:26.126308 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:28 fabsnuc.intel.com kubelet[26430]: E0524 12:45:28.222872 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:38 fabsnuc.intel.com kubelet[26430]: E0524 12:45:38.417057 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:42 fabsnuc.intel.com kubelet[26430]: E0524 12:45:42.416826 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:51 fabsnuc.intel.com kubelet[26430]: E0524 12:45:51.416901 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:45:55 fabsnuc.intel.com kubelet[26430]: E0524 12:45:55.416863 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:46:06 fabsnuc.intel.com kubelet[26430]: E0524 12:46:06.134784 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:46:08 fabsnuc.intel.com kubelet[26430]: E0524 12:46:08.174227 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:46:09 fabsnuc.intel.com kubelet[26430]: E0524 12:46:09.187954 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:46:16 fabsnuc.intel.com kubelet[26430]: E0524 12:46:16.126184 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:46:22 fabsnuc.intel.com kubelet[26430]: E0524 12:46:22.417038 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:46:31 fabsnuc.intel.com kubelet[26430]: E0524 12:46:31.416842 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:46:34 fabsnuc.intel.com kubelet[26430]: E0524 12:46:34.418387 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:46:42 fabsnuc.intel.com kubelet[26430]: E0524 12:46:42.416809 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:46:46 fabsnuc.intel.com kubelet[26430]: E0524 12:46:46.416890 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:46:53 fabsnuc.intel.com kubelet[26430]: E0524 12:46:53.416860 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:01 fabsnuc.intel.com kubelet[26430]: E0524 12:47:01.417019 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:04 fabsnuc.intel.com kubelet[26430]: E0524 12:47:04.417086 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:13 fabsnuc.intel.com kubelet[26430]: E0524 12:47:13.416857 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:18 fabsnuc.intel.com kubelet[26430]: E0524 12:47:18.416919 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:27 fabsnuc.intel.com kubelet[26430]: E0524 12:47:27.416850 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:31 fabsnuc.intel.com kubelet[26430]: E0524 12:47:31.759366 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:36 fabsnuc.intel.com kubelet[26430]: E0524 12:47:36.126234 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:43 fabsnuc.intel.com kubelet[26430]: E0524 12:47:43.868686 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:48 fabsnuc.intel.com kubelet[26430]: E0524 12:47:48.222803 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:48 fabsnuc.intel.com kubelet[26430]: E0524 12:47:48.416945 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:47:59 fabsnuc.intel.com kubelet[26430]: E0524 12:47:59.416959 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:48:01 fabsnuc.intel.com kubelet[26430]: E0524 12:48:01.416976 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:48:11 fabsnuc.intel.com kubelet[26430]: E0524 12:48:11.417023 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:48:15 fabsnuc.intel.com kubelet[26430]: E0524 12:48:15.416908 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:48:26 fabsnuc.intel.com kubelet[26430]: E0524 12:48:26.416923 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:48:27 fabsnuc.intel.com kubelet[26430]: E0524 12:48:27.416851 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
May 24 12:48:37 fabsnuc.intel.com kubelet[26430]: E0524 12:48:37.416896 26430 pod_workers.go:190] Error syncing pod 94a7de57-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-tn8vz_kube-system(94a7de57-7e4b-11e9-82df-54b20311c753)"
May 24 12:48:40 fabsnuc.intel.com kubelet[26430]: E0524 12:48:40.416996 26430 pod_workers.go:190] Error syncing pod 94a8990f-7e4b-11e9-82df-54b20311c753 ("coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"), skipping: failed to "StartContainer" for "coredns" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=coredns pod=coredns-fb8b8dccf-z28dc_kube-system(94a8990f-7e4b-11e9-82df-54b20311c753)"
==> storage-provisioner <==
The operating system version:
[root@fabsnuc ~]# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
[root@fabsnuc ~]# docker version
Client:
Version: 18.09.6
API version: 1.39
Go version: go1.10.8
Git commit: 481bc77156
Built: Sat May 4 02:34:58 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 18.09.6
API version: 1.39 (minimum version 1.12)
Go version: go1.10.8
Git commit: 481bc77
Built: Sat May 4 02:02:43 2019
OS/Arch: linux/amd64
Experimental: false
[root@fabsnuc ~]#
[root@fabsnuc ~]# getenforce
Disabled
[root@fabsnuc ~]#
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 27 (7 by maintainers)
use command
systemctl stop firewalldI’ve worked out the problem.It’s not solved this problem…
@fabstao I was facing the exact same issue on my CentOS VM. I got it fixed by following the instructions in this comment: https://github.com/kubernetes/kubeadm/issues/193#issuecomment-330060848 to flush the iptables
I have this problem in prod env after runing some days, server in pod can not access outer network , exececute blow commands reloved, i want to know how happened before encounter this problem , and how to prevent this ; systemctl stop kubelet systemctl stop docker iptables --flush iptables -tnat --flush systemctl start kubelet systemctl start docker
How can flushing or disabling the firewall be an accepted solution - this is disastrous. Please provide details on which firewall ports need to be opened and if any Kubernetes related interfaces (docker, flannel, …) need to be assigned specific zones in order for CoreDNS to be able to connect to the API.
how to fix permanently without disabling firewalld or any workaround?
@tacerus: You can’t reopen an issue/PR unless you authored it or you are a collaborator.
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
In my case it was an issue with dashboard. 💣 http://127.0.0.1:37577/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ is not accessible: Temporary Error: unexpected response code: 503
if you have firewalld enabled you can add docker0 bridge interface to trusted zone which should allow docker containers to host communication
Unfortunately, the fix in @albinsuresh 's reply is a work-around. Does anyone know what he true fix is if you’re running a customized local firewall? I’ll do some digging and post again if I find it.