autoscaler: VPA restarts my pods but does not modify CPU or memory settings
Which component are you using?: vertical-pod-autoscaler
What version of the component are you using?: 0.10.0
Component version:
What k8s version are you using (kubectl version)?:
[root@mc8k516pe4mve7s1jroc0 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.12", GitCommit:"4bf2e32bb2b9fdeea19ff7cdc1fb51fb295ec407", GitTreeState:"clean", BuildDate:"2021-10-27T17:12:26Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.12", GitCommit:"4bf2e32bb2b9fdeea19ff7cdc1fb51fb295ec407", GitTreeState:"clean", BuildDate:"2021-10-27T17:07:18Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"}
kubectl version Output
$ kubectl version
What environment is this in?:
[root@mc8k516pe4mve7s1jroc0 ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
hamster-7844f5fbd8-8jw84 1/1 Running 0 47s
hamster-7844f5fbd8-98qrz 1/1 Running 0 2m47s
[root@mc8k516pe4mve7s1jroc0 ~]# kubectl -n kube-system get pod |grep vpa
vpa-admission-controller-79858b644f-tfq5x 1/1 Running 0 36m
vpa-recommender-6b86988ffd-gcsk6 1/1 Running 0 42m
vpa-updater-7ddcc6fc9f-6s6cw 1/1 Running 0 42m
[root@mc8k516pe4mve7s1jroc0 ~]# kubectl describe vpa hamster-vpa
Name: hamster-vpa
Namespace: default
Labels: <none>
Annotations: <none>
API Version: autoscaling.k8s.io/v1
Kind: VerticalPodAutoscaler
Metadata:
Creation Timestamp: 2022-03-14T06:03:51Z
Generation: 42
Managed Fields:
API Version: autoscaling.k8s.io/v1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:resourcePolicy:
.:
f:containerPolicies:
f:targetRef:
.:
f:apiVersion:
f:kind:
f:name:
f:updatePolicy:
.:
f:updateMode:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2022-03-14T06:03:51Z
API Version: autoscaling.k8s.io/v1
Fields Type: FieldsV1
fieldsV1:
f:status:
.:
f:conditions:
f:recommendation:
.:
f:containerRecommendations:
Manager: recommender
Operation: Update
Time: 2022-03-14T06:04:43Z
Resource Version: 1652258
UID: 62fa448b-bbe7-403e-bc95-6279ff212d2a
Spec:
Resource Policy:
Container Policies:
Container Name: *
Controlled Resources:
cpu
memory
Max Allowed:
Cpu: 1
Memory: 500Mi
Min Allowed:
Cpu: 100m
Memory: 50Mi
Target Ref:
API Version: apps/v1
Kind: Deployment
Name: hamster
Update Policy:
Update Mode: Auto
Status:
Conditions:
Last Transition Time: 2022-03-14T06:04:43Z
Status: True
Type: RecommendationProvided
Recommendation:
Container Recommendations:
Container Name: hamster
Lower Bound:
Cpu: 547m
Memory: 262144k
Target:
Cpu: 587m
Memory: 262144k
Uncapped Target:
Cpu: 587m
Memory: 262144k
Upper Bound:
Cpu: 1
Memory: 415402439
Events: <none>
[root@mc8k516pe4mve7s1jroc0 ~]# kubectl describe mutatingWebhookConfiguration vpa-webhook-config
Name: vpa-webhook-config
Namespace:
Labels: <none>
Annotations: <none>
API Version: admissionregistration.k8s.io/v1
Kind: MutatingWebhookConfiguration
Metadata:
Creation Timestamp: 2022-03-14T06:09:23Z
Generation: 1
Managed Fields:
API Version: admissionregistration.k8s.io/v1
Fields Type: FieldsV1
fieldsV1:
f:webhooks:
.:
k:{"name":"vpa.k8s.io"}:
.:
f:admissionReviewVersions:
f:clientConfig:
.:
f:caBundle:
f:service:
.:
f:name:
f:namespace:
f:port:
f:failurePolicy:
f:matchPolicy:
f:name:
f:namespaceSelector:
f:objectSelector:
f:reinvocationPolicy:
f:rules:
f:sideEffects:
f:timeoutSeconds:
Manager: admission-controller
Operation: Update
Time: 2022-03-14T06:09:23Z
Resource Version: 1642069
UID: 80b7be1d-02e0-4fc7-b3a9-8ba5bcf6d4f2
Webhooks:
Admission Review Versions:
v1
Client Config:
Ca Bundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCekNDQWUrZ0F3SUJBZ0lKQU1DNnlRN2JEQ3RPTUEwR0NTcUdTSWIzRFFFQkN3VUFNQmt4RnpBVkJnTlYKQkFNTURuWndZVjkzWldKb2IyOXJYMk5oTUNBWERUSXlNRE14TkRBMk1ETXpPVm9ZRHpJeU9UVXhNakk0TURZdwpNek01V2pBWk1SY3dGUVlEVlFRRERBNTJjR0ZmZDJWaWFHOXZhMTlqWVRDQ0FTSXdEUVlKS29aSWh2Y05BUUVCCkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUxVM1ZoNmdlUFZ4RmNuT1JMRE90anBEWk9QWUZiSUQybVBWdDhzYzVsd3YKeHlWeGZ5VXkwNzFWQnBBZVlzbUdSQ2FpVE1lNDNBaGx0NzZkWGF3d3htdVlWYWYzTFFNRjQzaktCdHpNZitOTwpRUno3MjBCSmNiakRaamhZRDYrZjdWL2U3NEhtR3FCK2tqT2tLSjk2Qy9IbVFyTk5scm1ySnQyb0UreGtoQlZuCnlJUXNnT2I1dlVGQmVvcjRNVjNXMTkrRHJXQXdnMXo1SlZvTE9jSHJuTUFDbUIyV1FIczIrc0R0czlSR21UcEwKcEhtM2lDaUY5RXB6ejVldkxISUhvMXFtZ0I0S1crN2tyd09ucG14K3lZbFpHd3dTRTRXT1ZJSjBDMnQ3MkNqbwprQ09MYVVnS0xOMklxOWJNSGFBVFRMNE02VzBqRzE2K3Z2VXlDQWYvbnpNQ0F3RUFBYU5RTUU0d0hRWURWUjBPCkJCWUVGQjN3M2ZwajBOY0x4Tkg2Q1hqVmNsNXhRQ0RCTUI4R0ExVWRJd1FZTUJhQUZCM3czZnBqME5jTHhOSDYKQ1hqVmNsNXhRQ0RCTUF3R0ExVWRFd1FGTUFNQkFmOHdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBQnl5VTN0YwpYeEpXTjRIYXNJT2xyU280YzZPY3NoWWpWMlpnMkc5TXZLZFpxT2hBOVdWWWhuUEJORmxGelBscHJKd1VVTy9PCkR5eE0xTzdTQXFFd1l0MXJNeXlzRjdlazJscmVRVnlIb2JlQTFmVDBNdDZtdTVJeGZzRi9YWkNOMFN4NWxPNzkKeklZeHBINTFFVEFVekR3aVY5bTVoUHZxeENZMjFJRHNURUVpUnZaeDc1ajJZazc2N092QXVBWHpUUzNtNEJlbQpkYzNKVDlFNkEzQW9wSWhTbWQzalg1NHdVM1lTT2luV09uWTVnbG15U0UzSnNwMjByRm54ZFFUUXluMTdkcWk3CmxJRWxLbjZXaThZOEswWU1Na3hkRHQzZHFnaVY3anFJOXhsWlpSdFJCS3NUazJqK1FqRjNrOVBJYTFoaVBsamYKNlp6NVA3NUZhd0tGVTFNPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
Service:
Name: vpa-webhook
Namespace: kube-system
Port: 443
Failure Policy: Ignore
Match Policy: Equivalent
Name: vpa.k8s.io
Namespace Selector:
Object Selector:
Reinvocation Policy: Never
Rules:
API Groups:
API Versions:
v1
Operations:
CREATE
Resources:
pods
Scope: *
API Groups:
autoscaling.k8s.io
API Versions:
*
Operations:
CREATE
UPDATE
Resources:
verticalpodautoscalers
Scope: *
Side Effects: None
Timeout Seconds: 30
Events: <none>
What did you expect to happen?:
What happened instead?:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 20 (2 by maintainers)
We observed this issue while testing gardener with VPA 0.10.0 as well. We saw the following error in
kube-apiserverlogs:We realized that our
vpa-webhook-configMutatingWebhookConfigurationonly listedv1beta1inadmissionReviewVersions, but VPA 0.10.0 is returningv1:Once we changed the above version to
v1, the errors in thekube-apiserverlogs were gone, and the issue itself disappeared.It seems that 0.10.0 requires changing to (or adding) the
v1admission review version in theMutatingWebhookConfiguration. Perhaps this should be considered a breaking change and documented properly?/cc @voelzmo