trident: msg="GRPC error: rpc error: code = NotFound desc = node XXXXX was not found"

Describe the bug Following the basic example in the documentation fails to attach the volume to the Pod.

Environment Provide accurate information about the environment to help us reproduce the issue.

[root@k3n trident-installer]# ./tridentctl -n trident get backend
+----------------------+----------------+--------------------------------------+--------+---------+
|         NAME         | STORAGE DRIVER |                 UUID                 | STATE  | VOLUMES |
+----------------------+----------------+--------------------------------------+--------+---------+
| ontapnas_10.11.5.186 | ontap-nas      | 57a270cb-051a-4107-8146-1111111e7a5 | online |       2 |
+----------------------+----------------+--------------------------------------+--------+---------+


[root@k3n trident-installer]# ./tridentctl -n trident  version
+----------------+----------------+
| SERVER VERSION | CLIENT VERSION |
+----------------+----------------+
| 19.10.0        | 19.10.0        |
+----------------+----------------+

Docker

Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.39 (downgraded from 1.40)
 Go version:        go1.12.12
 Git commit:        633a0ea
 Built:             Wed Nov 13 07:25:41 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.7
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       2d0083d
  Built:            Thu Jun 27 17:26:28 2019
  OS/Arch:          linux/amd64
  Experimental:     false

k8s version

[root@k3n trident-installer]# kubectl  version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.5", GitCommit:"20c265fef0741dd71a66480e35bd69f18351daea", GitTreeState:"clean", BuildDate:"2019-10-15T19:16:51Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.5", GitCommit:"20c265fef0741dd71a66480e35bd69f18351daea", GitTreeState:"clean", BuildDate:"2019-10-15T19:07:57Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
[root@k3n trident-installer]# kubectl get nodes
NAME          STATUS   ROLES    AGE   VERSION
k1m.domain.com   Ready    master   28d   v1.15.5
k3n.domain.com   Ready    <none>   28d   v1.15.5
k4n.domain.com   Ready    <none>   28d   v1.15.5

To Reproduce Follow the basic example

Expected behavior attach the created volume to the Pod

Additional context I also see in the logs errors related to iSCSI, which I believe we are not using.

time="2019-12-12T09:47:06Z" level=warning msg="Couldn't retrieve volume transaction logs: Unable to find key"
time="2019-12-12T09:47:06Z" level=info msg="Trident bootstrapped successfully."
time="2019-12-12T09:47:06Z" level=info msg="Activating plain CSI helper frontend."
time="2019-12-12T09:47:06Z" level=info msg="Activating CSI frontend."
time="2019-12-12T09:47:06Z" level=info msg="Listening for GRPC connections." name=/plugin/csi.sock net=unix
time="2019-12-12T09:47:06Z" level=error msg="Error gathering initiator names."
time="2019-12-12T09:47:06Z" level=error msg="Could not get iSCSI initiator name." error="exit status 1"

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 1
  • Comments: 19 (9 by maintainers)

Commits related to this issue

Most upvoted comments

For everyone that encountered this reported issue it was determined that either a DNS or a networking issue kept the Trident node DaemonSet from registering with the Trident controller. Commit 8e51987 improves the Info log message to help the Trident user resolve this registration issue.

@khatrig thanks for updating this issue.