kubernetes: Metrics Server problems
/kind bug Hi, I did a hpa autoscaling practice, and used the metrics server, but there is also no value about resources of pods, here is what happened:
- Create deployments
# kubectl create -f httpd-deployment.yml
deployment.apps/httpd created
# cat httpd-deployment.yml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: httpd
spec:
replicas: 1
template:
metadata:
labels:
run: httpd
spec:
containers:
- name: httpd
image: httpd:2.2.32
ports:
- containerPort: 80
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "500m"
memory: "256Mi"
- Create service
# kubectl create -f httpd-service.yml
service/httpd-svc created
# cat httpd-service.yml
apiVersion: v1
kind: Service
metadata:
name: httpd-svc
spec:
type: NodePort
selector:
run: httpd
ports:
- protocol: TCP
port: 8080
targetPort: 80
- Create hpa
# kubectl create -f hpa_httpd_test.yml
horizontalpodautoscaler.autoscaling/hpa-httpd created
# cat hpa_httpd_test.yml
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
name: hpa-httpd
spec:
scaleTargetRef:
apiVersion: apps/v1beta1
kind: Deployment
name: httpd
minReplicas: 1
maxReplicas: 10
#targetCPUUtilizationPercentage: 50
metrics:
- type: Resource
resource:
name: cpu
targetAverageUtilization: 50
- type: Resource
resource:
name: memory
targetAverageValue: 200Mi
- Get hpa information
# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
hpa-httpd Deployment/httpd <unknown>/200Mi, <unknown>/50% 1 10 1 1m
Problems: there is no value about the resources of my pod!
- Describe the hpa
# kubectl describe hpa hpa-httpd
Name: hpa-httpd
Namespace: default
Labels: <none>
Annotations: <none>
CreationTimestamp: Tue, 21 Aug 2018 16:05:27 +0800
Reference: Deployment/httpd
Metrics: ( current / target )
resource memory on pods: <unknown> / 200Mi
resource cpu on pods (as a percentage of request): <unknown> / 50%
Min replicas: 1
Max replicas: 10
Deployment pods: 1 current / 0 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True SucceededGetScale the HPA controller was able to get the target's current scale
ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: unable to get metrics for resource memory: no metrics returned from resource metrics API
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedGetResourceMetric 9s (x8 over 3m) horizontal-pod-autoscaler unable to get metrics for resource memory: no metrics returned from resource metrics API
Warning FailedComputeMetricsReplicas 9s (x8 over 3m) horizontal-pod-autoscaler failed to get memory utilization: unable to get metrics for resource memory: no metrics returned from resource metrics API
while the metrics server is running without errors.
# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-78fcdf6894-5psw7 1/1 Running 0 6d
coredns-78fcdf6894-77qhn 1/1 Running 0 6d
etcd-cluster-2-master 1/1 Running 0 6d
kube-apiserver-cluster-2-master 1/1 Running 0 6d
kube-controller-manager-cluster-2-master 1/1 Running 0 6d
kube-flannel-ds-dtjpq 1/1 Running 0 6d
kube-flannel-ds-lflpz 1/1 Running 0 6d
kube-proxy-pcwkd 1/1 Running 0 6d
kube-proxy-ttcrf 1/1 Running 0 6d
kube-scheduler-cluster-2-master 1/1 Running 0 6d
metrics-server-56dc49db67-5c7qx 1/1 Running 0 1d
- my kubernetes version
# kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.1", GitCommit:"b1b29978270dc22fecc592ac55d903350454310a", GitTreeState:"clean", BuildDate:"2018-07-17T18:53:20Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:08:19Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
OS:
# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
Kernel:
Linux cluster-2-master 3.10.0-862.9.1.el7.x86_64 #1 SMP Mon Jul 16 16:29:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
And I didn’t find the parameter “–horizontal-pod-autoscaler-use-rest-clients=false” in the file: “/etc/kubernetes/manifests/kube-controller-manager.yaml”. Also, I didn’t run the heapster.
Could you please give me some suggestions? Thanks!
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 15 (1 by maintainers)
Il n ‘y a pas de question dans la version 1.11, mais dans les kubernets 1.13.1, j’ ai trouvé la question.
command: - /metrics-server - --metric-resolution=30s - --kubelet-insecure-tls - --kubelet-preferred-address-types=InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP
int 0.3.1 try this command
@Bessonov, Umm, yeah - I was trying to say - --kubelet-preferred-address-types=InternalIP works for me. I tried it today only - but on minikube 😃 . I referred this video - https://www.youtube.com/watch?v=uxuyPru3_Lc Check if it’s useful for you or not…
Usage of
kubelet-insecure-tls
orinsecure=true
doesn’t seems like a solution. Can you please open the issue again? @hechuan73