secrets-store-csi-driver-provider-azure: Volume not being mounted to pods

What steps did you take and what happened: [A clear and concise description of what the bug is.] I’m trying to deploy pods mounting Azure KeyVault secrets in an Azure AKS cluster. When developing the app on docker-desktop, everything was working fine. After deploying the secrets-store-csi-driver-provider-azure to my live production cluster, pods are no longer able to mount the csi-secrets volume from the driver.

As soon as pods try to mount a volume, the secrets store node registrar is informing 2020-11-26T14:51:53.862734037Z E1126 14:51:53.862480 1 connection.go:129] Lost connection to unix:///csi/csi.sock. and the pod gets a restart.

When using the exact same values.yaml files for both my application and the Azure secrets store on docker-desktop under WSL2, the pods start up containing the secrets.

What did you expect to happen: The volume would be mounted properly, without any secrets-store-csi-driver-XYZ restarts

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.] https://gist.github.com/Dyllaann/61864f59f500fd1e2a5a7e046171ce93 gist files containing setup. I was able to see the following CPU Throttling while this was happening, but even increasing the limits of the secrets-store to 1000m, it was still throttling 100%. image

Which access mode did you use to access the Azure Key Vault instance: [e.g. Service Principal, Pod Identity, User Assigned Managed Identity, System Assigned Managed Identity] Service Principal. Credentials are verified working by using local installation.

Environment:

- Secrets Store CSI Driver version: (use the image tag): v0.0.16 - Azure Key Vault provider version: (use the image tag): 0.0.9 **- Kubernetes version: (use kubectl version and kubectl get nodes -o wide): ** Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.15", GitCommit:"2a313bf59aa0e1c98bbee2aea6ae727d3f6da6fd", GitTreeState:"clean", BuildDate:"2020-09-03T05:20:07Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"} - Cluster type: (e.g. AKS, aks-engine, etc): AKS. Using a NodePool of Standard_DS2_v2 instances

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 1
  • Comments: 26 (13 by maintainers)

Most upvoted comments

We’ve made a number of optimizations based on load testing in our latest releases. Please checkout:

https://github.com/kubernetes-sigs/secrets-store-csi-driver/releases/tag/v0.0.21 https://secrets-store-csi-driver.sigs.k8s.io/load-tests.html https://github.com/Azure/secrets-store-csi-driver-provider-azure/releases/tag/0.0.14

Try it out when you get a chance and let us know if the issue still persists.