faster-whisper: Docker error "Could not load library libcudnn_ops_infer.so.8. Error: libcudnn_ops_infer.so.8: cannot open shared object file: No such file or directory"

Hi. I have a container which has CUDA and cudnn (nvidia/cuda:11.8.0-base-ubuntu22.04). But when I want to use it in inference mode, I have a problem: Could not load library libcudnn_ops_infer.so.8. Error: libcudnn_ops_infer.so.8: cannot open shared object file: No such file or directory

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 18

Commits related to this issue

Most upvoted comments

This issue was resolved by installing libcudnn8 and libcudnn8-dev on ubuntu 22.04 with cuda12

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb 
dpkg -i cuda-keyring_1.0-1_all.deb 
apt update && apt upgrade
apt install libcudnn8 libcudnn8-dev

@cod3r0k , ok. My CUDA version in host machine is 12.0. Therefor I used image nvidia/cuda:12.0.0-devel-ubuntu20.04. Below is a simple example for FW Dockerfile:

FROM nvidia/cuda:12.0.0-devel-ubuntu20.04

WORKDIR /root

RUN apt-get update -y && apt-get install -y python3 python3-pip libcudnn8 libcudnn8-dev
RUN python3 -m pip install pip --upgrade
COPY infer.py .
COPY jfk.flac .
RUN pip3 install faster-whisper

ENTRYPOINT ["python3", "infer.py"]

infer.py

import time
from faster_whisper import WhisperModel
 
jfk_path = 'jfk.flac'
model_path = 'tiny'
tic = time.time()
model = WhisperModel(model_path, device='cuda')
segments, info = model.transcribe(jfk_path, word_timestamps=True)
for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

print("Total time: ", time.time() - tic)

Running:

docker run --gpus all faster_whisper:latest
[0.00s -> 10.48s]  And so my fellow Americans ask not what your country can do for you, ask what you can do for your country.
Total time:  8.151259183883667

Hope it’s helpful for you.

This issue was resolved by installing libcudnn8 and libcudnn8-dev on ubuntu 22.04 with cuda12

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb 
dpkg -i cuda-keyring_1.0-1_all.deb 
apt update && apt upgrade
apt install libcudnn8 libcudnn8-dev

Thank you! It works for me!

This issue was resolved by installing libcudnn8 and libcudnn8-dev on ubuntu 22.04 with cuda12

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb 
dpkg -i cuda-keyring_1.0-1_all.deb 
apt update && apt upgrade
apt install libcudnn8 libcudnn8-dev

Many thanks.

@cod3r0k , hello. Can you try again with:

sudo apt-get install libcudnn8
sudo apt-get install libcudnn8-dev

I think you should use nvidia/cuda image version 12. Recently, ctranslate2 lib was upgraded to 4.0 to support cuda 12.

Currently, I am facing an issue that is not resolved yet. It’s working fine in non-docker, but in docker, it’s not working. Can you please share the docker file that you have used to run it successfully? @trungkienbkhn