rancher: [BUG] After Rancher 2.6.11 upgrade, rancher-v2.6.11-secret-migrator userAgent is constantly updating cluster resources
Rancher Server Setup
- Rancher version: 2.6.11
- Installation option (Docker install/Helm Chart): Helm Chart on RKE1 K8s v1.21.10
- Proxy/Cert Details: TCP load balancer; Rancher terminates TLS with company certificate
Information about the Cluster
- Kubernetes version: 1.21.10
- Cluster Type (Local/Downstream): local
User Information
- What is the role of the user logged in? n/a
Describe the bug
After upgrading Rancher from 2.6.9 to 2.6.11, we saw a huge increase in kube-apiserver calls in the local cluster. A substantial contributor of these API calls are coming from the previously-unseen rancher-2.6.11-secrets-migrator
userAgent, at the rate of ~2000-3000 PUT calls per hour:
I0417 18:14:01.727838 1 httplog.go:94] "HTTP" verb="PUT" URI="/apis/management.cattle.io/v3/clusters/c-bq829" latency="24.283621ms" userAgent="rancher-v2.6.11-secret-migrator" srcIP="172.19.2.95:49578" resp=200
To Reproduce
- Install Rancher 2.6.11
- Add some downstream clusters
Result
Incessant PUTs to the clusters.management.cattle.io resources from the rancher-v2.6.11-secret-migrator
userAgent
Expected Result Perhaps a one-time spike as “secrets” were “migrated”, but little to no calls after that. I’m not sure what this particular userAgent is actually doing – is it syncing Project-scoped Secrets? Cattle-cluster-agent secrets? Tokens? There’s vanishingly little documentation about what this thing does.
Screenshots
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 20 (4 by maintainers)
Test Environment:
Rancher version: v2.6-head 9c4e37f Rancher cluster type: HA Docker version: 20.10
Downstream cluster type: RKE2
Testing:
Tested this issue with the following steps:
Result I am not seeing the spike in the secrets migrator resource anymore. This issue seems to be not reproducible anymore.