secrets-store-csi-driver-provider-azure: aks failed to get mounted files, err: failed to list all files in target path
What steps did you take and what happened: [A clear and concise description of what the bug is.] I am only seeing this error on k8s init windows jobs. I am unsure of linux jobs are impacted as I don’t run any linux init jobs.
13m Warning FailedToCreateSecret /mssql-init-sr859 failed to get mounted files, err: failed to list all files in target path c:\var\lib\kubelet\pods\a7529eb1-16eb-4b23-9674-d8fc2c7e8424\volumes\kubernetes.io~csi\secrets-store-inline\mount, err: open c:\var\lib\kubelet\pods\a7529eb1-16eb-4b23-9674-d8fc2c7e8424\volumes\kubernetes.io~csi\secrets-store-inline\mount: The system cannot find
the path specified.
10m Warning FailedToCreateSecret /solr-init-pgzvc failed to get mounted files, err: failed to list all files in target path c:\var\lib\kubelet\pods\bd78698c-b15e-4d71-bf6d-e58f4e125f0e\volumes\kubernetes.io~csi\secrets-store-inline\mount, err: open c:\var\lib\kubelet\pods\bd78698c-b15e-4d71-bf6d-e58f4e125f0e\volumes\kubernetes.io~csi\secrets-store-inline\mount: The system cannot find
the path specified.
This does not stop the k8s job from running and completing successfully.
I did not see this issue in 0.0.8, but I am on 0.0.9 aks 1.18.8 vmss with latest windows node image
What did you expect to happen: no warning error.
Which access mode did you use to access the Azure Key Vault instance: [e.g. Service Principal, Pod Identity, User Assigned Managed Identity, System Assigned Managed Identity] User Assigned Managed Identity
Environment:
- Secrets Store CSI Driver version: (use the image tag): 0.0.16
- Azure Key Vault provider version: (use the image tag):0.0.9
- Kubernetes version: (use
kubectl version
andkubectl get nodes -o wide
):
kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.0", GitCommit:"9e991415386e4cf155a24b1da15becaa390438d8", GitTreeState:"clean", BuildDate:"2020-03-25T14:58:59Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"73ec19bdfc6008cd3ce6de96c663f70a69e2b8fc", GitTreeState:"clean", BuildDate:"2020-09-17T04:17:08Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
aks-nodepool1-27591730-vmss000000 Ready agent 8h v1.18.8 10.200.12.4 <none> Ubuntu 18.04.5 LTS 5.4.0-1026-azure docker://19.3.12
aks-nodepool1-27591730-vmss000001 Ready agent 8h v1.18.8 10.200.12.20 <none> Ubuntu 18.04.5 LTS 5.4.0-1026-azure docker://19.3.12
aks-npsolr-27591730-vmss000000 Ready agent 8h v1.18.8 10.200.12.129 <none> Ubuntu 18.04.5 LTS 5.4.0-1026-azure docker://19.3.12
aks-npsolr-27591730-vmss000001 Ready agent 8h v1.18.8 10.200.12.140 <none> Ubuntu 18.04.5 LTS 5.4.0-1026-azure docker://19.3.12
aks-npsolr-27591730-vmss000002 Ready agent 8h v1.18.8 10.200.12.151 <none> Ubuntu 18.04.5 LTS 5.4.0-1026-azure docker://19.3.12
aks-npzk-27591730-vmss000000 Ready agent 8h v1.18.8 10.200.12.162 <none> Ubuntu 18.04.5 LTS 5.4.0-1026-azure docker://19.3.12
aks-npzk-27591730-vmss000001 Ready agent 8h v1.18.8 10.200.12.173 <none> Ubuntu 18.04.5 LTS 5.4.0-1026-azure docker://19.3.12
aks-npzk-27591730-vmss000002 Ready agent 8h v1.18.8 10.200.12.184 <none> Ubuntu 18.04.5 LTS 5.4.0-1026-azure docker://19.3.12
aksnpwin000000 Ready agent 8h v1.18.8 10.200.12.36 <none> Windows Server 2019 Datacenter 10.0.17763.1397 docker://19.3.11
aksnpwin000001 Ready agent 8h v1.18.8 10.200.12.67 <none> Windows Server 2019 Datacenter 10.0.17763.1397 docker://19.3.11
aksnpwin000002 Ready agent 8h v1.18.8 10.200.12.98 <none> Windows Server 2019 Datacenter 10.0.17763.1397 docker://19.3.11
- Cluster type: (e.g. AKS, aks-engine, etc): AKS
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 18 (7 by maintainers)
We’ve released driver
v0.0.17
and AKV provider0.0.10
- https://github.com/Azure/secrets-store-csi-driver-provider-azure/releases/tag/0.0.10. Withv0.0.17
we have added checks to skip rotation and reconcile for deleting pods.Closing this issue with the release and also the primary issue for pod stuck in
Terminating
was caused due to windows insights. Please feel free to open a new issue for any further queries.@aramase they are pushing the fix across all the regions and hope to have it done by end of week. css case # 120101424001816