kubernetes: nfs flexvolume example not working

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see https://kubernetes.io/docs/tasks/debug-application-cluster/troubleshooting/.): No.

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): flexvolume nfs

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Kubernetes version (use kubectl version): Client Version: version.Info{Major:“1”, Minor:“6”, GitVersion:“v1.6.4”, GitCommit:“d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae”, GitTreeState:“clean”, BuildDate:“2017-05-19T18:44:27Z”, GoVersion:“go1.7.5”, Compiler:“gc”, Platform:“linux/amd64”} Server Version: version.Info{Major:“1”, Minor:“6”, GitVersion:“v1.6.2”, GitCommit:“477efc3cbe6a7effca06bd1452fa356e2201e1ee”, GitTreeState:“clean”, BuildDate:“2017-04-19T20:22:08Z”, GoVersion:“go1.7.5”, Compiler:“gc”, Platform:“linux/amd64”}

Environment:

• Cloud provider or hardware configuration: virtualbox
• OS (e.g. from /etc/os-release): CentOS 7.3
• Kernel (e.g. uname -a): Linux master 3.10.0-514.16.1.el7.x86_64
• Install tools: kubeadm
• Others:

What happened: The flexvolume NFS example doesn’t work. I’ve noticed a bug with getvolumename and hence removed the implementation.

kubelet gives the following log. By adding traces into the shell script (using logger) and I noticed that only init() was called.

Jun  7 14:15:17 node1 kubelet: W0607 14:15:17.482243    2267 plugin-defaults.go:32] flexVolume driver k8s/nfs: using default GetVolumeName for volume 0x5b0380
Jun  7 14:15:17 node1 kubelet: I0607 14:15:17.968999    2267 reconciler.go:231] VerifyControllerAttachedVolume operation started for volume "k8s/nfs/flex002" (spec.Name: "flex002") pod "ad2d1540-4b48-11e7-996e-080027632f89" (UID: "ad2d1540-4b48-11e7-996e-080027632f89")
Jun  7 14:15:17 node1 kubelet: E0607 14:15:17.969123    2267 nestedpendingoperations.go:262] Operation for "\"k8s/nfs/flex002\"" failed. No retries permitted until 2017-06-07 14:15:18.969101889 +0800 HKT (durationBeforeRetry 1s). Error: Volume "k8s/nfs/flex002" (spec.Name: "flex002") pod "ad2d1540-4b48-11e7-996e-080027632f89" (UID: "ad2d1540-4b48-11e7-996e-080027632f89") has not yet been added to the list of VolumesInUse in the node's volume status
Jun  7 14:15:18 node1 kubelet: I0607 14:15:18.982262    2267 reconciler.go:231] VerifyControllerAttachedVolume operation started for volume "k8s/nfs/flex002" (spec.Name: "flex002") pod "ad2d1540-4b48-11e7-996e-080027632f89" (UID: "ad2d1540-4b48-11e7-996e-080027632f89")
Jun  7 14:15:18 node1 kubelet: E0607 14:15:18.982313    2267 nestedpendingoperations.go:262] Operation for "\"k8s/nfs/flex002\"" failed. No retries permitted until 2017-06-07 14:15:20.982301785 +0800 HKT (durationBeforeRetry 2s). Error: Volume "k8s/nfs/flex002" (spec.Name: "flex002") pod "ad2d1540-4b48-11e7-996e-080027632f89" (UID: "ad2d1540-4b48-11e7-996e-080027632f89") has not yet been added to the list of VolumesInUse in the node's volume status
Jun  7 14:15:21 node1 kubelet: I0607 14:15:21.000292    2267 reconciler.go:231] VerifyControllerAttachedVolume operation started for volume "k8s/nfs/flex002" (spec.Name: "flex002") pod "ad2d1540-4b48-11e7-996e-080027632f89" (UID: "ad2d1540-4b48-11e7-996e-080027632f89")
Jun  7 14:15:21 node1 kubelet: E0607 14:15:21.002636    2267 nestedpendingoperations.go:262] Operation for "\"k8s/nfs/flex002\"" failed. No retries permitted until 2017-06-07 14:15:25.002617865 +0800 HKT (durationBeforeRetry 4s). Error: Volume "k8s/nfs/flex002" (spec.Name: "flex002") pod "ad2d1540-4b48-11e7-996e-080027632f89" (UID: "ad2d1540-4b48-11e7-996e-080027632f89") is not yet attached according to node status
Jun  7 14:15:25 node1 kubelet: I0607 14:15:25.035581    2267 reconciler.go:231] VerifyControllerAttachedVolume operation started for volume "k8s/nfs/flex002" (spec.Name: "flex002") pod "ad2d1540-4b48-11e7-996e-080027632f89" (UID: "ad2d1540-4b48-11e7-996e-080027632f89")
Jun  7 14:15:25 node1 kubelet: E0607 14:15:25.039820    2267 nestedpendingoperations.go:262] Operation for "\"k8s/nfs/flex002\"" failed. No retries permitted until 2017-06-07 14:15:33.039799967 +0800 HKT (durationBeforeRetry 8s). Error: Volume "k8s/nfs/flex002" (spec.Name: "flex002") pod "ad2d1540-4b48-11e7-996e-080027632f89" (UID: "ad2d1540-4b48-11e7-996e-080027632f89") is not yet attached according to node status

What you expected to happen: init() -> domount() are called and volume successfully mounted into the pod.

How to reproduce it (as minimally and precisely as possible): Install the plugin in /usr/libexec/kubernetes/kubelet-plugins/volume/exec/k8s~nfs/nfs and use the following spec to create a pod

apiVersion: v1
kind: Pod
metadata:
  name: nginxnfs
  namespace: default
spec:
  containers:
  - name: nginxnfs
    image: nginx
    volumeMounts:
    - name: flex002
      mountPath: /data2
    ports:
    - containerPort: 80
  volumes:
  - name: flex002
    flexVolume:
      driver: "k8s/nfs"
      fsType: "nfs"
      options:
        server: "172.20.0.1"
        share: "/data/nfs-k8s"

Anything else we need to know: