autoscaler: [Azure AKS] Can't scale-up nodes
I have deployed cluster-autoscaler (v1.3.1) in AKS (with k8s 1.10.3) yesterday and to test, I created a pod with a large CPU/RAM request. I was expecting a new node to be created but that didn’t happened and all I can see is the output below:
I0713 10:34:43.451225 1 azure_manager.go:261] Refreshed ASG list, next refresh after 2018-07-13 10:35:43.451205157 +0000 UTC m=+577.450491985
I0713 10:34:43.706164 1 utils.go:503] No pod using affinity / antiaffinity found in cluster, disabling affinity predicate for this loop
I0713 10:34:43.706875 1 scale_up.go:249] Pod default/frontend is unschedulable
I0713 10:34:43.926322 1 scale_up.go:376] No expansion options
I manually scaled up my cluster with 1 extra node and the pod was scheduled and it is running fine. Interestingly, I still see the same output as above in cluster-autoscaler log even after the pod is running. In every ASG refresh, cluster-autoscaler still repeats the same log output.
Before scaling up the nodes manually, the following message was present in CA logs (it is not anymore):
Node group agentpool is not ready for scaleup
Am I missing something here? I didn’t changed any default options in CA other than the required values.
Thanks!
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 24 (12 by maintainers)
There is a known issue in AKS such that autoscaler will get 400 response from AKS RP when the cluster was created with Custom VNet due to autoscaler using older api-version
2017-08-31. We are working on the fix. It should go out in 1-2 weeks.New CA releases with AKS API 2018-03-31: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.2.3-beta.1 and https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.3.2-beta.1.
@guitmz @abizake Could you check whether new releases fix the issue on your cluster?
@guitmz Thanks for the info. Could you share the status of CA? .e.g by
kubectl -n kube-system describe configmaps cluster-autoscaler-status