trident: Docker plugin fails to start after upgrade to 20.10+
Describe the bug
After upgrading the docker netapp/trident-plugin to version 20.10, it fails to be re-enabled with an error of
Error response from daemon: dial unix /run/docker/plugins/2311ab6e7b3b1461539ddc3c783ac65ad9a179d44fffd76b53aff9dd844ae1c6/netapp.sock: connect: no such file or directory
Upon following the troubleshooting steps in https://netapp-trident.readthedocs.io/en/stable-v20.10/docker/troubleshooting.html and checking the logs with journalctl, the following messages are logged:
Jan 14 10:59:35 hostname dockerd[14554]: time="2021-01-14T10:59:35-07:00" level=error msg="time=\"2021-01-14T17:59:35Z\" level=info msg=\"Running Trident storage orchestrator.\" binary=/netapp/trident build_time=\"Fri Jan 8 19:10:34 UTC 2021\" version=20.10.1" plugin=2311ab6e7b3b1461539ddc3c783ac65ad9a179d44fffd76b53aff9dd844ae1c6
Jan 14 10:59:35 hostname dockerd[14554]: time="2021-01-14T10:59:35-07:00" level=error msg="time=\"2021-01-14T17:59:35Z\" level=fatal msg=\"Insufficient arguments provided for Trident to start. Specify k8sAPIServer (for Kubernetes) or configPath (for Docker) or csiEndpoint (for CSI).\"" plugin=2311ab6e7b3b1461539ddc3c783ac65ad9a179d44fffd76b53aff9dd844ae1c6
Jan 14 10:59:47 hostname dockerd[14554]: time="2021-01-14T10:59:47.360433094-07:00" level=error msg="Handler for POST /v1.40/plugins/netapp:latest/enable returned error: dial unix /run/docker/plugins/2311ab6e7b3b1461539ddc3c783ac65ad9a179d44fffd76b53aff9dd844ae1c6/netapp.sock: connect: no such file or directory"
Environment Provide accurate information about the environment to help us reproduce the issue.
- Trident version: 20.10
- Trident installation flags used:
docker plugin install --grant-all-permissions --alias netapp netapp/trident-plugin:19.10 config=/etc/netappdvp/config.jsonfrom the original install, though this is an upgrade - Container runtime: Docker 19.03.14-CE
- Kubernetes version: N/A
- Kubernetes orchestrator: N/A
- Kubernetes enabled feature gates: N/A
- OS: Ubuntu 18.04.5 LTS
- NetApp backend types: ONTAP-NAS
- Other: N/A
To Reproduce Steps to reproduce the behavior: With a previous version of the netapp/trident-plugin installed (I tested with 19.10, 20.04, and 20.07) run through the steps in https://netapp-trident.readthedocs.io/en/latest/docker/use/managing.html#updating-trident
- Disable plugin
$ sudo docker plugin disable -f netapp:latest
netapp:latest
- Upgrade plugin
$ sudo docker plugin upgrade --skip-remote-check --grant-all-permissions netapp:latest netapp/trident-plugin:20.10
Upgrading plugin netapp:latest from netapp/trident-plugin:20.07 to netapp/trident-plugin:20.10
20.10: Pulling from netapp/trident-plugin
09043c5b1668: Download complete
Digest: sha256:bb73c2da04224603ce60a88546143b5045b235234c937c07331b4d4a1236da59
Status: Downloaded newer image for netapp/trident-plugin:20.10
Upgraded plugin netapp:latest to netapp/trident-plugin:20.10
- Enable plugin
$ sudo docker plugin enable netapp:latest
Error response from daemon: dial unix /run/docker/plugins/2311ab6e7b3b1461539ddc3c783ac65ad9a179d44fffd76b53aff9dd844ae1c6/netapp.sock: connect: no such file or directory
Expected behavior Expected a response of
$ sudo docker plugin enable netapp:latest
netapp:latest
Additional context
In researching the error that is reported I found the corresponding code at https://github.com/NetApp/trident/blob/stable/v20.10/main.go#L129-L132 suggesting that the configPath may not be set. When the docker plugin was installed with docker plugin install the following parameter was passed though config=/etc/netappdvp/config.json and I can confirm that it is set by running docker plugin inspect netapp and seeing the output for Config.Env.config Value: “config.json” as well as this section of the output:
"Id": "2311ab6e7b3b1461539ddc3c783ac65ad9a179d44fffd76b53aff9dd844ae1c6",
"Name": "netapp:latest",
"PluginReference": "docker.io/netapp/trident-plugin:20.10",
"Settings": {
"Args": [],
"Devices": [],
"Env": [
"debug=true",
"rest=false",
"config=/etc/netappdvp/config.json"
],
I was able to downgrade back to 20.07 which functions as expected.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 17 (5 by maintainers)
This issue is fixed with commit ce346f3 and is included in the Trident 21.04 release.
Hi @xd999e,
There is a NetApp support case open on this issue and the team is investigating it. If you need immediate assistance you can also contact NetApp support. We will update this issue when we have more information.