faster-whisper: Could not load library libcudnn_ops_infer.so.8

Could not load library libcudnn_ops_infer.so.8. Error: libcudnn_ops_infer.so.8: cannot open shared object file: No such file or directory

I’m using “nvidia/cuda:12.2.0-base-ubuntu20.04” image on google cloud with nvidia t4 gpus.

The normal whisper package model works fine on cuda.

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 12
  • Comments: 46 (2 by maintainers)

Most upvoted comments

Sweet. I was able to get this working.

I installed the NVIDIA software in the README. That caused issues. Had the same libcudnn_ops_infer.so.8 error as the original poster.

Steps to fix:

  1. Went back to factory GPU driver sudo apt-get purge nvidia-* then sudo apt autoremove then sudo apt install system76-driver-nvidia
    • Your factory driver will likely be different than mine
  2. Ran the alternative instructions in the README (I use Linux)
    • pip install nvidia-cublas-cu11 nvidia-cudnn-cu11
    export LD_LIBRARY_PATH=`python3 -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`
    
    • pip install faster-whisper
  3. Tried the quickstart code in the README with the jfk.flak from the test/data directory
    • The first time it ran, it downloaded the model
    • Second time, it transcribed the data
    • The key was setting the LD_LIBRARY_PATH env var
me@me:~/projects/speech$ source .venv/bin/activate
(.venv) me@me:~/projects/speech$ python3 test3.py 
Could not load library libcudnn_ops_infer.so.8. Error: libcudnn_ops_infer.so.8: cannot open shared object file: No such file or directory
Aborted (core dumped)
(.venv) me@me:~/projects/speech$ export LD_LIBRARY_PATH=`python3 -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`
(.venv) me@me:~/projects/speech$ python3 test3.py 
Detected language 'en' with probability 0.929688
[0.00s -> 3.00s]  And so my fellow Americans,
[3.00s -> 8.00s]  ask not what your country can do for you,
[8.00s -> 11.00s]  ask what you can do for your country.

Hope this helps someone!

If you are installing CUDA via pip in a virtual environment (and the same goes for on host, VM, or in a container):

# point to VENV's local CUDA 11.8 python lib
export LD_LIBRARY_PATH=${PWD}/.venv/lib64/python3.11/site-packages/nvidia/cublas/lib:${PWD}/.venv/lib64/python3.11/site-packages/nvidia/cudnn/lib

My previous comment about needing to downgrade my host CUDA toolkit and drivers was wrong. You just need to have a host system with drivers that supports up to or past the CUDA version required by the library.

If you continue to have trouble, please provide the pip dependencies installed in your dev/prod environment, where those deps are located in the environment, and also post the outputs of the following:

nvidia-smi
nvcc --version

For everyone that has this issue, what fixed it for me was to include the path to torch too in LD_LIBRARY_PATH. The docker image I run is pytorch/pytorch:2.0.1-cuda11.7-cudnn8-runtime

The line below adds torch as well as cudnn and cublas to the path.

export LD_LIBRARY_PATH=`python3 -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; import torch; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__) + ":" + os.path.dirname(torch.__file__) +"/lib")'`

For me installing the cuDNN 8 libraries using sudo apt install libcudnn8 on Ubuntu 22.04 fixed the issue!

Solution: if you are on CUDA=12.x and have cuDNN=9, then install ctranslate==4.5.0

cudnn 8.x.x.x is very important. https://docs.nvidia.com/deeplearning/cudnn/latest/installation/linux.html

python3 -m pip install nvidia-cudnn-cu11==8.9.6.50

Thanks @seaBFH ! After that, find where they are.

find . -name 'libcublas.so*' -o -name 'libcudnn_ops*'

If you see like this, ./.venv/lib/python3.10/site-packages/nvidia/cudnn/lib/libcudnn_ops_train.so.8 ./.venv/lib/python3.10/site-packages/nvidia/cudnn/lib/libcudnn_ops_infer.so.8

Then, do like this. It worked for me.

export LD_LIBRARY_PATH=${PWD}/.venv/lib/python3.10/site-packages/nvidia/cudnn/lib

I used in Ubuntu 24.04 sudo apt install nvidia-cudnn

Problem: Could not load library libcudnn_ops_infer.so.8. Error: libcublas.so.11: cannot open shared object file: No such file or directory Aborted (core dumped)

Use python to check the path of the lib

import os
import nvidia.cublas.lib
import nvidia.cudnn.lib

print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))

Add the LD_LIBRARY_PATH variable in bashrc, the content is the data printed by python.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/usr/.local/lib/python3.10/site-packages/nvidia/cublas/lib:/home/usr/.local/lib/python3.10/site-packages/nvidia/ cudnn/lib

After modification, remember to close the current terminal and reopen a new terminal so that the above configuration will take effect.

Just a note: vast.ai machine was Ubuntu 22.04 x84_64 machine. And I tested it for this machine only.

Here is what helped me:

  1. Install NVCC driver
sudo apt install nvidia-cuda-toolkit
  1. Install cuDNN (Version 8!)

It is not trivial to install version 8! If you go to the official website to install cuDNN and enter your machine’s details, you will only get the latest version — version 9. I tried it, but it doesn’t work (even when you link from calls of version 8 to version 9).

Official Website

So, you would need to go to archive and submit your request for version 8.

After that, you will get a file called .deb. In my case, I got cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb

Here is how you install it:

sudo dpkg -i cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb
sudo cp /var/cudnn-local-repo-ubuntu2204-8.9.7/cudnn-local-*.gpg /usr/share/keyrings/

Maybe you would also need to do that:

sudo apt-get update
sudo apt-get install --reinstall libcudnn8 libcudnn8-dev libcudnn8-samples
  1. Verify installation
ls /usr/lib/x86_64-linux-gnu/libcudnn* | grep libcudnn_ops_infer.so.8
ls /usr/lib/x86_64-linux-gnu/libcudnn* | grep libcudnn_cnn_infer.so.8

Could not load library libcudnn_ops_infer.so.8

You can find cuBLAS and cuDNN libs for Linux there at Releases -> https://github.com/Purfview/whisper-standalone-win

Not tested, report if they work.

need cuda 11.8

As in “at least 11.8” or “exactly 11.8”? I have CUDA Version: 12.0 installed (in WSL2/Ubuntu) but get this error.

cudnn 8.x.x.x is very important. https://docs.nvidia.com/deeplearning/cudnn/latest/installation/linux.html

python3 -m pip install nvidia-cudnn-cu11==8.9.6.50

Hi everybody, and thank you for helping me in solving this issue!

Expanding on @justinthelaw comment I have used the following command instead:

path LD_LIBRARY=$LD_LIBRARY_PATH:$HOME/path/to/venv/lib64/python3.x/site-packages/nvidia/cublas/lib:$HOME/path/to/venv/lib64/python3.x/site-packages/nvidia/cudnn/lib

With this you append the paths to the $LD_LIBRARY_PATH rather than overwriting it. In the path, /path/to/venv must be substituted with the actual location (and name) of your virtual environment. The same applies to python3.x, where the x must be substituted with the Python version in use.

As a final comment, export applies only to the terminal in which it is issued. One may consider to append it to the $HOME/.bashrc script to make it persistent.

各位大佬,有没有解决方案阿,上边的方案都尝试了

Image

Image

Image

解决了 ,用的现成的docker镜像搞定了,可以参考下我写的博客 https://blog.csdn.net/CXY00000/article/details/147079214?spm=1001.2014.3001.5501

Just find the location of libcudnn_ops_infer.so.8, and add it to LD_LIBRARY_PATH:

$ find ~/ -name libcudnn_ops_infer.so.8
/home/alan/miniconda3/lib/python3.11/site-packages/nvidia/cudnn/lib/libcudnn_ops_infer.so.8
$ export LD_LIBRARY_PATH=/home/alan/miniconda3/lib/python3.11/site-packages/nvidia/cudnn/lib/

it work for me:

pip install torch --index-url https://download.pytorch.org/whl/cu121

It happened in the docker-compose / docker. To solve it I had to execute the next command inside the docker:

 export LD_LIBRARY_PATH=/usr/local/lib/python3.9/site-packages/torch/lib:$LD_LIBRARY_PATH

Check if your LD_LIBRARY_PATH is specified and points to your cuda location.

Another solution is to install nvidia-cudnn in your venv. You could simply run:

pip install nvidia-pyindex
pip install nvidia-cudnn

link: https://pypi.org/project/nvidia-cudnn/

That really worked for me!

For me pip install ctranslate2==4.5.0 worked (instead of pip install ctranslate==4.5.0 as https://github.com/SYSTRAN/faster-whisper/issues/516#issuecomment-2628799793 said), thanks for the fix @sheldonrong 🙌🏻✨

@SeaDude your solution is the only one that worked! thank you

For me installing the cuDNN 8 libraries using sudo apt install libcudnn8 on Ubuntu 22.04 fixed the issue!

Just doing this command fixed the issue for me. No need for export LD_LIBRARY_PATH=... in my case.

I have been using it but I was hoping to use faster-whisper for obvious reasons…

@justinthelaw try adding path to .../**/torch/lib to the LD_LIBRARY_PATH.