kyverno: [Bug] slow with multiple CRD

Kyverno Version

1.7.2

Kubernetes Version

1.22.x

Kubernetes Platform

GKE

Kyverno Rule Type

Other

Description

When using many CRD, kyverno seem to hang to get all object.

Steps to reproduce

  1. GKE cluster
  2. activate Config Connector

Expected behavior

Maybe I would like to filter what kind / object I want kyverno to watch, with a system of allowlist/denylist.

Screenshots

No response

Kyverno logs

I0721 14:30:20.841366       1 request.go:665] Waited for 1.03801592s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/datafusion.cnrm.cloud.google.com/v1beta1
I0721 14:30:22.041219       1 request.go:665] Waited for 2.237810898s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/iam.cnrm.cloud.google.com/v1beta1
I0721 14:30:40.041630       1 request.go:665] Waited for 1.103295875s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/secretmanager.cnrm.cloud.google.com/v1beta1
I0721 14:30:41.241477       1 request.go:665] Waited for 2.303027443s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/storage.cnrm.cloud.google.com/v1beta1
I0721 14:30:59.241210       1 request.go:665] Waited for 1.173411424s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/secretmanager.cnrm.cloud.google.com/v1beta1
I0721 14:31:00.441604       1 request.go:665] Waited for 2.37350095s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/storage.cnrm.cloud.google.com/v1beta1
I0721 14:31:18.441629       1 request.go:665] Waited for 1.098312604s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/infra.operator.themecloud.io/v1alpha1
I0721 14:31:19.641165       1 request.go:665] Waited for 2.297668825s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/scalingpolicy.kope.io/v1alpha1
I0721 14:31:37.640732       1 request.go:665] Waited for 1.003108372s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/secretmanager.cnrm.cloud.google.com/v1beta1
I0721 14:31:38.640763       1 request.go:665] Waited for 2.002961783s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/sql.cnrm.cloud.google.com/v1beta1
I0721 14:31:56.841499       1 request.go:665] Waited for 1.110686517s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/cloudbuild.cnrm.cloud.google.com/v1beta1
I0721 14:31:58.040964       1 request.go:665] Waited for 2.310099785s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/compute.cnrm.cloud.google.com/v1beta1
I0721 14:32:16.041490       1 request.go:665] Waited for 1.103124737s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/osconfig.cnrm.cloud.google.com/v1beta1
I0721 14:32:17.241301       1 request.go:665] Waited for 2.302774477s due to client-side throttling, not priority and fairness, request: GET:https://10.7.240.1:443/apis/sourcerepo.cnrm.cloud.google.com/v1beta1

Slack discussion

No response

Troubleshooting

  • I have read and followed the documentation AND the troubleshooting guide.
  • I have searched other issues in this repository and mine is not recorded.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 29 (22 by maintainers)

Most upvoted comments

I think this will probably be resolved once we switch over to only loading CRDs if a policy calls for it. See #3966