csi-driver-smb: The smb volume can't be mounted to the windows pod after reboot
I have installed a k8s environment with two linux worker nodes and one windows worker node.
k8s version: 1.20
kubectl get nodes
NAME STATUS ROLES AGE VERSION
docp-k8s-master1 Ready control-plane,master 24d v1.20.1
docp-k8s-worker-11 Ready worker 24d v1.20.1
docp-k8s-worker-21 Ready worker 24d v1.20.1
docp-win191 Ready <none> 23d v1.20.0
csi-smb-driver version: 0.6.0 kubectl get pod -n kube-system | grep csi
csi-smb-controller-645cd7b449-6dm2d 3/3 Running 4 21d
csi-smb-controller-645cd7b449-qzr6m 3/3 Running 7 21d
csi-smb-node-4t5n7 3/3 Running 1 21d
csi-smb-node-9j5xm 3/3 Running 1 21d
csi-smb-node-kbtb6 3/3 Running 1 21d
csi-smb-node-win-mdrv7 3/3 Running 3 109m
csi-proxy version: 0.22
I0201 02:13:57.776512 6596 main.go:55] Starting CSI-Proxy Server ...
I0201 02:13:57.781666 6596 main.go:56] Version: v0.2.2-0-gffb169f
sc.yaml:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: smb
provisioner: smb.csi.k8s.io
parameters:
source: "//docp-smb1/smbservice"
csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
csi.storage.k8s.io/node-stage-secret-namespace: "default"
createSubDir: "true" # optional: create a sub dir for new volume
reclaimPolicy: Retain # only retain is supported
volumeBindingMode: Immediate
mountOptions:
- dir_mode=0777
- file_mode=0777
- uid=1001
- gid=1001
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox-smb-2
labels:
app: busybox-2
spec:
replicas: 1
template:
metadata:
name: busybox-2
labels:
app: busybox-2
spec:
tolerations:
- key: "OS"
operator: "Equal"
value: "Windows"
effect: "NoSchedule"
nodeSelector:
"kubernetes.io/os": windows
containers:
- name: busybox-2
image: e2eteam/busybox:1.29
command:
- "powershell.exe"
- "-Command"
- "while (1) { Add-Content -Encoding Ascii C:\\mnt\\smb\\data.txt $(Get-Date -Format u); sleep 1 }"
volumeMounts:
- name: smb
mountPath: "/mnt/smb"
volumes:
- name: smb
persistentVolumeClaim:
claimName: pvc-smb-test
selector:
matchLabels:
app: busybox-2
I got the error message like this after reboot the windows node.
MountVolume.MountDevice failed for volume “pvc-1efb71f1-ab8a-4bbf-8db7-84a8e58877b4” : rpc error: code = Internal desc = volume(pvc-1efb71f1-ab8a-4bbf-8db7-84a8e58877b4) mount “//docp-smb1/smbservice” on “\var\lib\kubelet\plugins\kubernetes.io\csi\pv\pvc-1efb71f1-ab8a-4bbf-8db7-84a8e58877b4\globalmount” failed with smb mapping failed with error: rpc error: code = Unknown desc = NewSmbGlobalMapping failed. output: “New-SmbGlobalMapping : Multiple connections to a server or shared resource by the same user, using more than one user \r\nname, are not allowed. Disconnect all previous connections to the server or shared resource and try again. \r\nAt line:1 char:190\r\n+ … ser, $PWord;New-SmbGlobalMapping -RemotePath $Env:smbremotepath -Cred …\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (MSFT_SmbGlobalMapping:ROOT/Microsoft/…mbGlobalMapping) [New-SmbGlobalMa \r\n pping], CimException\r\n + FullyQualifiedErrorId : Windows System Error 1219,New-SmbGlobalMapping\r\n \r\n”, err: exit status 1
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 19 (9 by maintainers)
to workaround, run following commands on the agent node:
Get-SmbGlobalMappingto get the SMB mappingRemove-SmbGlobalMappingto remove the existing mappingthere should be a log file, e.g. csi-proxy.log
could you provide the csi-proxy logs? thanks. csi-proxy code is here, need to check what went wrong: https://github.com/kubernetes-csi/csi-proxy/blob/v0.2.2/internal/server/smb/server.go#L43-L72