autoscaler: [Azure AKS] Can't scale-up nodes

I have deployed cluster-autoscaler (v1.3.1) in AKS (with k8s 1.10.3) yesterday and to test, I created a pod with a large CPU/RAM request. I was expecting a new node to be created but that didn’t happened and all I can see is the output below:

I0713 10:34:43.451225       1 azure_manager.go:261] Refreshed ASG list, next refresh after 2018-07-13 10:35:43.451205157 +0000 UTC m=+577.450491985
I0713 10:34:43.706164       1 utils.go:503] No pod using affinity / antiaffinity found in cluster, disabling affinity predicate for this loop
I0713 10:34:43.706875       1 scale_up.go:249] Pod default/frontend is unschedulable
I0713 10:34:43.926322       1 scale_up.go:376] No expansion options

I manually scaled up my cluster with 1 extra node and the pod was scheduled and it is running fine. Interestingly, I still see the same output as above in cluster-autoscaler log even after the pod is running. In every ASG refresh, cluster-autoscaler still repeats the same log output.

Before scaling up the nodes manually, the following message was present in CA logs (it is not anymore):

Node group agentpool is not ready for scaleup

Am I missing something here? I didn’t changed any default options in CA other than the required values.

Thanks!

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 24 (12 by maintainers)

Most upvoted comments

There is a known issue in AKS such that autoscaler will get 400 response from AKS RP when the cluster was created with Custom VNet due to autoscaler using older api-version 2017-08-31. We are working on the fix. It should go out in 1-2 weeks.

@guitmz Thanks for the info. Could you share the status of CA? .e.g by kubectl -n kube-system describe configmaps cluster-autoscaler-status