istio-operator: Unexpected periodical istiod scale down

Describe the bug Hello community,

We are experiencing periodical unexpected istiod deployment scale down that is far below an HPA recommendations and below a minimum replicas count value. It happens every 12 hour in our case.

That behaviour leads to a control plane traffic amount increase and thus for an outbound traffic cost’ increase for us.

If we stop an operator (scale its deployment down to 0 replicas) we don’t see such a behaviour.

Example of replicas spec

replicas:
          count: 30
          max: 160
          min: 30

Please also find an example screenshot attached.

Please let me know if any additional info is required.

Thank you!

Expected behavior To not have such a drastic scale down. Rollout restart can be used instead.

Screenshots image

Additional context Platform 1.23.16-gke.1400 Operator version v2.16.1 Pilot version 1.15.3

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Reactions: 1
  • Comments: 15

Most upvoted comments

@vitalii-buchyn-exa no problem. Feel free to reach out. We’ll take a look at what is possible to do. You can try to change PodDisruptionBudget settings for now. Maybe the tips from here will come in handy as well: https://github.com/kubernetes/kubernetes/issues/72231

sure, --devel-mode and --verbose added will wait till the next downscale event and share logs