secrets-store-csi-driver-provider-azure: Volume not being mounted to pods
What steps did you take and what happened: [A clear and concise description of what the bug is.] I’m trying to deploy pods mounting Azure KeyVault secrets in an Azure AKS cluster. When developing the app on docker-desktop, everything was working fine. After deploying the secrets-store-csi-driver-provider-azure to my live production cluster, pods are no longer able to mount the csi-secrets volume from the driver.
As soon as pods try to mount a volume, the secrets store node registrar is informing 2020-11-26T14:51:53.862734037Z E1126 14:51:53.862480 1 connection.go:129] Lost connection to unix:///csi/csi.sock.
and the pod gets a restart.
When using the exact same values.yaml files for both my application and the Azure secrets store on docker-desktop under WSL2, the pods start up containing the secrets.
What did you expect to happen:
The volume would be mounted properly, without any secrets-store-csi-driver-XYZ
restarts
Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
https://gist.github.com/Dyllaann/61864f59f500fd1e2a5a7e046171ce93 gist files containing setup.
I was able to see the following CPU Throttling while this was happening, but even increasing the limits of the secrets-store to 1000m
, it was still throttling 100%.
Which access mode did you use to access the Azure Key Vault instance: [e.g. Service Principal, Pod Identity, User Assigned Managed Identity, System Assigned Managed Identity] Service Principal. Credentials are verified working by using local installation.
Environment:
- Secrets Store CSI Driver version: (use the image tag): v0.0.16
- Azure Key Vault provider version: (use the image tag): 0.0.9
**- Kubernetes version: (use kubectl version
and kubectl get nodes -o wide
): **
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.15", GitCommit:"2a313bf59aa0e1c98bbee2aea6ae727d3f6da6fd", GitTreeState:"clean", BuildDate:"2020-09-03T05:20:07Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
- Cluster type: (e.g. AKS, aks-engine, etc): AKS. Using a NodePool of Standard_DS2_v2
instances
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 1
- Comments: 26 (13 by maintainers)
We’ve made a number of optimizations based on load testing in our latest releases. Please checkout:
https://github.com/kubernetes-sigs/secrets-store-csi-driver/releases/tag/v0.0.21 https://secrets-store-csi-driver.sigs.k8s.io/load-tests.html https://github.com/Azure/secrets-store-csi-driver-provider-azure/releases/tag/0.0.14
Try it out when you get a chance and let us know if the issue still persists.