tensorflow: Could not load dynamic library 'libcudnn.so.8'

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): CentOS 7.9
TensorFlow installed from (source or binary): binary
TensorFlow version: tf-nightly-gpu 2.5.0.dev20201125
Python version: 3.6
Installed using virtualenv? pip? conda?: pip
Bazel version (if compiling from source): —
GCC/Compiler version (if compiling from source): —
CUDA/cuDNN version: 11.1
GPU model and memory: 10 x 1080Ti 11GB

I want to create keras model using multiple GPUs.

I have prepared simple code to display visible devices:

import os
import tensorflow as tf

os.environ["CUDA_VISIBLE_DEVICES"]="7, 8, 9"
tf.config.experimental.list_logical_devices('GPU')

But my output is:

2020-11-26 10:23:19.183104: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-26 10:23:22.087731: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-26 10:23:22.089534: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2020-11-26 10:23:22.894765: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1727] Found device 0 with properties:
pciBusID: 0000:0d:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.582GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2020-11-26 10:23:22.901688: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1727] Found device 1 with properties:
pciBusID: 0000:0e:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.582GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2020-11-26 10:23:22.908844: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1727] Found device 2 with properties:
pciBusID: 0000:0f:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.582GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2020-11-26 10:23:22.908893: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-26 10:23:22.913327: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-26 10:23:22.913376: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2020-11-26 10:23:22.914897: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2020-11-26 10:23:22.915190: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2020-11-26 10:23:22.917940: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2020-11-26 10:23:22.918885: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2020-11-26 10:23:22.919011: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory
2020-11-26 10:23:22.919038: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1764] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 18
Comments: 21 (1 by maintainers)

Most upvoted comments

Solved using Anaconda.

+197

martinpolacek on Nov 26, 2020

Same issue with Ubuntu 20.04, tensorflow 2.4.1 and cuda 11.2.

EDIT: In my case the issue was, that I needed to install libcudnn8. I did so downloading and executing the debian package from here.

+99

jaunruh on Feb 26, 2021

Same issue with “tensorflow==2.4.0rc” on WSL 2 Ubuntu with Cuda 11

+23

akshayrana30 on Dec 7, 2020

I got same situation when using my school’s GPU farm, and I ran “conda install -c nvidia cudnn”, then the problem is gone(The machine had already install CUDA10.1 and cudnn8.0.4)

+21

HeliumPeng on May 2, 2021

Same issue with Ubuntu 20.04, tensorflow 2.4.1 and cuda 11.2.

EDIT: In my case the issue was, that I needed to install libcudnn8. I did so downloading and executing the debian package from here.

It helps me a lot! My env is Ubuntu 20.04 kernel 5.8.0-45-generic,nvidia-460.67,cuda 11.0,cudnn v8.0.5,tensorflow 2.4.1. I downloaded and installed “libcudnn8_8.0.5.39-1+cuda11.0_amd64.deb”, and the question was solved.

+19

lxysl on Apr 3, 2021

You can also use conda install -c conda-forge cudnn, if it would not start download after using conda install -c nvidia cudnn (as it was in my case). It helped me solve the problem.

Ubuntu 18.04; TensorFlow 2.5.0;

conda-forge cudnn

+18

MortenTabaka on Jun 10, 2021

This worked for me on Ubuntu 22.04.1 LTS sudo apt install nvidia-cudnn

It’s a large install but it worked best for me as several other library files were missing.

andriworld on Aug 20, 2022

see: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html

this work for me (centos7)

find / -name "*libcuda.so*"
cd /usr/lib64/
ls -al |grep cuda
 ln -s libcuda.so.515.43.04 libcudnn.so.8

zgfh on Jul 7, 2022

or simply sudo apt install libcudnn8 if you use the Nvidia Cuda docker image. I don’t know why it has libcudnn8.so missing.

matifali on Sep 1, 2022

Vanilla tensorflow doesn’t work with cuda out of the box. Just follow Nvidia’s instructions and use containerized special tensorflow-nvidia with pre-set up environment around.

ValleZ on Feb 26, 2021