metrics-server: metric server not able to deploy on kind (M1 Mac)

What happened:

Metric server not deployed successfully:

metrics-server-dbf765b9b-mhqm7                 0/1     Running   0          11m

What you expected to happen:

Successful deployment

Anything else we need to know?:

Deployed using

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

But the metric server is not deployed:

 $ kubectl describe pod metrics-server-dbf765b9b-mhqm7  -n kube-system
Name:                 metrics-server-dbf765b9b-mhqm7
Namespace:            kube-system
Priority:             2000000000
Priority Class Name:  system-cluster-critical
Node:                 fluvio-control-plane/172.18.0.2
Start Time:           Mon, 29 Nov 2021 16:26:46 -0800
Labels:               k8s-app=metrics-server
                      pod-template-hash=dbf765b9b
Annotations:          <none>
Status:               Running
IP:                   10.244.0.10
IPs:
  IP:           10.244.0.10
Controlled By:  ReplicaSet/metrics-server-dbf765b9b
Containers:
  metrics-server:
    Container ID:  containerd://5baac8b28acbe186008baf0e6d6806f4ae391c0090c5c17b7c31dc26cc057401
    Image:         k8s.gcr.io/metrics-server/metrics-server:v0.5.2
    Image ID:      k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10
    Port:          4443/TCP
    Host Port:     0/TCP
    Args:
      --cert-dir=/tmp
      --secure-port=4443
      --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
      --kubelet-use-node-status-port
      --metric-resolution=15s
    State:          Running
      Started:      Mon, 29 Nov 2021 16:26:56 -0800
    Ready:          False
    Restart Count:  0
    Requests:
      cpu:        100m
      memory:     200Mi
    Liveness:     http-get https://:https/livez delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:    http-get https://:https/readyz delay=20s timeout=1s period=10s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /tmp from tmp-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-gprkb (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  tmp-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  kube-api-access-gprkb:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                  From               Message
  ----     ------     ----                 ----               -------
  Normal   Scheduled  2m48s                default-scheduler  Successfully assigned kube-system/metrics-server-dbf765b9b-mhqm7 to fluvio-control-plane
  Normal   Pulling    2m48s                kubelet            Pulling image "k8s.gcr.io/metrics-server/metrics-server:v0.5.2"
  Normal   Pulled     2m40s                kubelet            Successfully pulled image "k8s.gcr.io/metrics-server/metrics-server:v0.5.2" in 8.416767462s
  Normal   Created    2m39s                kubelet            Created container metrics-server
  Normal   Started    2m39s                kubelet            Started container metrics-server
  Warning  Unhealthy  9s (x14 over 2m19s)  kubelet            Readiness probe failed: HTTP probe failed with statuscode: 500

Environment:

  • Kubernetes distribution (GKE, EKS, Kubeadm, the hard way, etc.):
$ kind version
kind v0.11.1 go1.17.2 darwin/arm64
  • Container Network Setup (flannel, calico, etc.):

  • Kubernetes version (use kubectl version):

 kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.4", GitCommit:"b695d79d4f967c403a96986f1750a35eb75e75f1", GitTreeState:"clean", BuildDate:"2021-11-17T15:41:42Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-21T23:06:30Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/arm64"}
  • Metrics Server manifest
spoiler for Metrics Server manifest:
  • Kubelet config:
spoiler for Kubelet config:
  • Metrics server logs:
spoiler for Metrics Server logs:
  • Status of Metrics API:
spolier for Status of Metrics API:
kubectl describe apiservice v1beta1.metrics.k8s.io

/kind bug

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 3
  • Comments: 31 (18 by maintainers)

Most upvoted comments

@yangjunmyfm192085 you are right, changing it from default 15s to 30s solved the issue

Ok, as you wrote that you are using kind cluster the solution is clear. Kind uses self signed certificates for nodes.

Solution is clear, please provide flag --kubelet-insecure-tls to Metrics Server.

i have the same problem, even i have provided flag --kubelet-insecure-tls to Metrics Server.

Ok, as you wrote that you are using kind cluster the solution is clear. Kind uses self signed certificates for nodes.

Solution is clear, please provide flag --kubelet-insecure-tls to Metrics Server.

Very valuable analysis. Could you try to set the --metric-resolution flag of metrics-server to 30s or longer to verify whether the data cannot be obtained due to the time-consuming access to the metrics/resource endpoint? If it is this issue, need to focus on analyzing the time-consuming problem of access