kubernetes: Kubernetes HPA returning incorrect API version

What happened:

When applying a HPA autoscaling/v2beta1 manifest, kubernetes accepts the manifest and returns the autoscaling/v1 version when querying the manifest.

What you expected to happen: Manifest to maintain autoscaling/v2beta1 api version

How to reproduce it (as minimally and precisely as possible):

kubectl create -f hpa-demo.yaml

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: hpa-demo
spec:
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: hpa-demo-master
  minReplicas: 1
  maxReplicas: 5
  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 80
  - type: Resource
    resource:
      name: memory
      targetAverageUtilization: 80

kubectl get hpa -o yaml hpa-demo

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  annotations:
    autoscaling.alpha.kubernetes.io/conditions: '[{"type":"AbleToScale","status":"True","lastTransitionTime":"2018-11-29T10:53:31Z","reason":"ReadyForNewScale","message":"the
      last scale time was sufficiently old as to warrant a new scale"},{"type":"ScalingActive","status":"True","lastTransitionTime":"2018-11-29T10:53:31Z","reason":"ValidMetricFound","message":"the
      HPA was able to successfully calculate a replica count from memory resource
      utilization (percentage of request)"},{"type":"ScalingLimited","status":"False","lastTransitionTime":"2018-11-29T10:53:31Z","reason":"DesiredWithinRange","message":"the
      desired count is within the acceptable range"}]'
    autoscaling.alpha.kubernetes.io/current-metrics: '[{"type":"Resource","resource":{"name":"memory","currentAverageUtilization":38,"currentAverageValue":"4018176"}},{"type":"Resource","resource":{"name":"cpu","currentAverageUtilization":20,"currentAverageValue":"1m"}}]'
    autoscaling.alpha.kubernetes.io/metrics: '[{"type":"Resource","resource":{"name":"memory","targetAverageUtilization":80}}]'
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"autoscaling/v2beta1","kind":"HorizontalPodAutoscaler","metadata":{"annotations":{},"name":"hpa-demo","namespace":"staging0"},"spec":{"maxReplicas":5,"metrics":[{"resource":{"name":"cpu","targetAverageUtilization":80},"type":"Resource"},{"resource":{"name":"memory","targetAverageUtilization":80},"type":"Resource"}],"minReplicas":1,"scaleTargetRef":{"apiVersion":"extensions/v1beta1","kind":"Deployment","name":"hpa-demo-master"}}}
  creationTimestamp: 2018-11-29T10:52:01Z
  name: hpa-demo
  namespace: staging0
  resourceVersion: "718045"
  selfLink: /apis/autoscaling/v1/namespaces/staging0/horizontalpodautoscalers/hpa-demo
  uid: cd513382-f3c4-11e8-9fa7-02bf9ef29fa4
spec:
  maxReplicas: 5
  minReplicas: 1
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: hpa-demo-master
  targetCPUUtilizationPercentage: 80
status:
  currentCPUUtilizationPercentage: 20
  currentReplicas: 1
  desiredReplicas: 1

As you can see from the above, the manifest is accepted by kubernetes on creation, when retrieving the mainfest it’s returned in v1 format.

If we explicitly define the apiversion, we get the correct response. kubectl get hpa.v2beta1.autoscaling -o yaml hpa-demo

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"autoscaling/v2beta1","kind":"HorizontalPodAutoscaler","metadata":{"annotations":{},"name":"hpa-demo","namespace":"staging0"},"spec":{"maxReplicas":5,"metrics":[{"resource":{"name":"cpu","targetAverageUtilization":80},"type":"Resource"},{"resource":{"name":"memory","targetAverageUtilization":80},"type":"Resource"}],"minReplicas":1,"scaleTargetRef":{"apiVersion":"extensions/v1beta1","kind":"Deployment","name":"hpa-demo-master"}}}
  creationTimestamp: 2018-11-29T10:52:01Z
  name: hpa-demo
  namespace: staging0
  resourceVersion: "718045"
  selfLink: /apis/autoscaling/v2beta1/namespaces/staging0/horizontalpodautoscalers/hpa-demo
  uid: cd513382-f3c4-11e8-9fa7-02bf9ef29fa4
spec:
  maxReplicas: 5
  metrics:
  - resource:
      name: memory
      targetAverageUtilization: 80
    type: Resource
  - resource:
      name: cpu
      targetAverageUtilization: 80
    type: Resource
  minReplicas: 1
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: hpa-demo-master
status:
  conditions:
  - lastTransitionTime: 2018-11-29T10:53:31Z
    message: the last scale time was sufficiently old as to warrant a new scale
    reason: ReadyForNewScale
    status: "True"
    type: AbleToScale
  - lastTransitionTime: 2018-11-29T10:53:31Z
    message: the HPA was able to successfully calculate a replica count from memory
      resource utilization (percentage of request)
    reason: ValidMetricFound
    status: "True"
    type: ScalingActive
  - lastTransitionTime: 2018-11-29T10:53:31Z
    message: the desired count is within the acceptable range
    reason: DesiredWithinRange
    status: "False"
    type: ScalingLimited
  currentMetrics:
  - resource:
      currentAverageUtilization: 38
      currentAverageValue: "4018176"
      name: memory
    type: Resource
  - resource:
      currentAverageUtilization: 20
      currentAverageValue: 1m
      name: cpu
    type: Resource
  currentReplicas: 1
  desiredReplicas: 1

Anything else we need to know?:

Environment:

  • Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-21T09:17:39Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.10", GitCommit:"be1a908c6aa47e0ae1b1dc861a1de6ccfe963aa2", GitTreeState:"clean", BuildDate:"2018-11-13T11:33:04Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
  • Cloud provider or hardware configuration: AWS
  • OS (e.g. from /etc/os-release): Container Linux by CoreOS 1520.9.0 (Ladybug) 4.13.16-coreos-r1 docker://1.12.6
  • Kernel (e.g. uname -a):a
  • Install tools:
  • Others:

/kind bug

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 21 (15 by maintainers)

Most upvoted comments

This is working as intended. v1 is the default version of HPA so that’s what you get if you don’t request a specific version. You can get autoscaling/v2beta1 representation of your HPA by running kubectl get hpa.v2beta1.autoscaling -o yaml hpa-demo

A bit of explanaition: Internally all HPA objects are stored in the same way, regardless which version of API you used to create them. When you get HPA Kubernetes API server converts this internal representation to the version you requested.

I’m not saying the default, I’m saying the version that was applied

Is there a way to make hpa.v2beta1.autoscaling the default output in my cluster, for cluster operators this is confusing as they aren’t seeing the true configuration.

Is there a way to make hpa.v2beta1.autoscaling the default output in my cluster, for cluster operators this is confusing as they aren’t seeing the true configuration.

no, the API group and version ordering is pinned in the API server.

note that this only affects how kubectl decides to resolve ambiguous get requests… if you want a specific version, include that in the kubectl get request.

Or is this saying that nothing actually has to be done to older versions, but that this deprecation warning is only for clients that use the API and have expectations of the structure of the data based on the apiVersion?

Kubernetes does the right thing with the data. Users have to do the right thing with their clients (i.e., upgrade them to use new versions).

We’d strongly prefer that scripts specify the exact version they want-- @liggitt demonstrated the syntax above.

Piggybacking off of the realgambo, is there anyway to make the beta version the default? Because when deployed the hpa through helm, helm doesn’t allow the use of beta versions so reverts back to v1 when we would like to be using v2beta2 @MaciekPytel

thanks.