Description of problem:
When increase cpu load, the pod can't auto scale, hpa can't works as expect.
Version-Release number of selected component (if applicable):
[root@ip-172-18-12-2 horizontal-pod-autoscaling]# kubectl version
Client Version: version.Info{Major:"1", Minor:"2+", GitVersion:"v1.2.0-alpha.2.90+360d0e0a7e92dc", GitCommit:"360d0e0a7e92dcc4d00b4c555e166a2a71a7e713", GitTreeState:"clean"}
Server Version: version.Info{Major:"1", Minor:"2+", GitVersion:"v1.2.0-alpha.2.90+360d0e0a7e92dc", GitCommit:"360d0e0a7e92dcc4d00b4c555e166a2a71a7e713", GitTreeState:"clean"}
How reproducible:
Always.
Steps to Reproduce:
1. Enable experimental for kube-apiserver
1). For "kube-apiserver" add --runtime-config=extensions/v1beta1=true
2. Run & expose php-apache server
$ kubectl create -f rc-php-apache.yaml
$ kubectl expose rc php-apache --port=80
$ kubectl get pods
$ kubectl get svc
$ curl http://$php-server-ip
3. Create horizontal pod autoscaler
$ kubectl create -f hpa-php-apache.yaml
$ kubectl get hpa
4. Increase cpu load and check rc
$ while true; do curl http://$svcip:port; done
$ kubectl get hpa
$ kubectl get rc
$ kubectl get pod
Actual results:
4. When increase cpu load, the pod can't auto scale.
[root@ip-172-18-12-2 kubernetes]# kubectl get rc
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE
php-apache php-apache horizontal-pod-autoscaling:latest run=php-apache 1 58m
[root@ip-172-18-12-2 kubernetes]# kubectl get hpa
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
php-apache ReplicationController/default/php-apache/ 1% <waiting> 1 10 46m
[root@ip-172-18-12-2 kubernetes]# kubectl get pod
NAME READY STATUS RESTARTS AGE
php-apache-wuwzo 1/1 Running 0 58m
Expected results:
4. When increase cpu load, the pod should auto scale.
Additional info:
1.$ cat hpa-php-apache.yaml
apiVersion: extensions/v1beta1
kind: HorizontalPodAutoscaler
metadata:
name: php-apache
namespace: default
spec:
maxReplicas: 10
minReplicas: 1
scaleRef:
kind: ReplicationController
name: php-apache
namespace: default
cpuUtilization:
targetPercentage: 1
2.$ cat rc-php-apache.yaml
apiVersion: v1
kind: ReplicationController
metadata:
labels:
run: php-apache
name: php-apache
namespace: default
spec:
replicas: 1
selector:
run: php-apache
template:
metadata:
labels:
run: php-apache
spec:
containers:
- image: horizontal-pod-autoscaling:latest
imagePullPolicy: IfNotPresent
name: php-apache
resources:
requests:
cpu: 200m
restartPolicy: Always
3. Image "horizontal-pod-autoscaling:latest" is build from
https://github.com/kubernetes/kubernetes/tree/master/examples/extensions/horizontal-pod-autoscaling/image
4. The php-apache cpu usage can refer:
http://fpaste.org/281762/44541276/
@jszczepkowski For my env is local all-in-one. I setup the env just run command “hack/local-up-cluster.sh”. Then create heapster by json “cluster/addons/cluster-monitoring/standalone/heapster-controller.yaml” and hpa. Because kubelet listen on localhost 127.0.0.1 and heapster in container connect to 127.0.0.1, it will never connect to the master. At first, I don’t know how hpa works with heapster. Honestly after read the code, I know hpa get heapster from “kube-system” namesapce, it’s hard code and no doc describe this. So I make this mistake. I resolve this problem by create heapster in correctly namespace, change kube default listen ip, configure it in heapster yaml with "- --source=kubernetes:‘http://$kube-apiserver’ ".