kubernetes: kubelet can't get node

When I Use kubeadm init --config /etc/kubernetes/kubeadm.yml to install kubernetes, it hangs and reports:

# kubeadm init --config /etc/kubernetes/kubeadm.yml
[init] using Kubernetes version: v1.12.2
[preflight] running pre-flight checks
[preflight/images] Pulling images required for setting up a Kubernetes cluster
[preflight/images] This might take a minute or two, depending on the speed of your internet connection
[preflight/images] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[preflight] Activating the kubelet service
[certificates] Generated etcd/ca certificate and key.
[certificates] Generated etcd/server certificate and key.
[certificates] etcd/server serving cert is signed for DNS names [k8s-master-001 localhost] and IPs [127.0.0.1 ::1]
[certificates] Generated apiserver-etcd-client certificate and key.
[certificates] Generated etcd/peer certificate and key.
[certificates] etcd/peer serving cert is signed for DNS names [k8s-master-001 localhost] and IPs [10.1.254.103 127.0.0.1 ::1]
[certificates] Generated etcd/healthcheck-client certificate and key.
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [k8s-master-001 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.1.254.103]
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] valid certificates and keys now exist in "/etc/kubernetes/pki"
[certificates] Generated sa key and public key.
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
[controlplane] wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests"
[init] this might take a minute or longer if the control plane images have to be pulled

Unfortunately, an error has occurred:
	timed out waiting for the condition

This error is likely caused by:
	- The kubelet is not running
	- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
	- 'systemctl status kubelet'
	- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
	- 'docker ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'docker logs CONTAINERID'
couldn't initialize a Kubernetes cluster

kubelet logs as follows:

Oct 28 09:32:14 k8s-master-001 kubelet[18343]: E1028 09:32:14.913402   18343 kubelet.go:2236] node "k8s-master-001" not found
Oct 28 09:32:15 k8s-master-001 kubelet[18343]: E1028 09:32:15.013584   18343 kubelet.go:2236] node "k8s-master-001" not found
Oct 28 09:32:15 k8s-master-001 kubelet[18343]: E1028 09:32:15.113773   18343 kubelet.go:2236] node "k8s-master-001" not found
Oct 28 09:32:15 k8s-master-001 kubelet[18343]: E1028 09:32:15.147608   18343 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Node: Get https://10.1.254.103:6443/api/v1/nodes?fieldSelector=metadata.name%3Dk8s-master-001&limit=500&resourceVersion=0: dial tcp 10.1.254.103:6443: connect: connection refused
Oct 28 09:32:15 k8s-master-001 kubelet[18343]: E1028 09:32:15.148489   18343 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://10.1.254.103:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dk8s-master-001&limit=500&resourceVersion=0: dial tcp 10.1.254.103:6443: connect: connection refused
Oct 28 09:32:15 k8s-master-001 kubelet[18343]: E1028 09:32:15.149459   18343 reflector.go:134] k8s.io/kubernetes/pkg/kubelet/kubelet.go:442: Failed to list *v1.Service: Get https://10.1.254.103:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.1.254.103:6443: connect: connection refused
Oct 28 09:32:15 k8s-master-001 kubelet[18343]: E1028 09:32:15.213963   18343 kubelet.go:2236] node "k8s-master-001" not found

But in my /etc/hosts, it has the record:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

10.1.254.103   k8s-master-001

and I can ping k8s-master-001 successful, the uname -n is also k8s-master-001.

docker ps -a | grep kube get nothing.

Why kubelet can’t recognize my host, but apiserver and etcd can recognize it. it’s so strange, can somebody explain it, thanks!

Environment:

• Kubernetes version (use kubectl version): v1.12.2
• OS (e.g. from /etc/os-release): Red Hat Enterprise Linux Server release 7.5
• Kernel (e.g. uname -a): 3.10.0-862.14.4.el7.x86_64