whisper-asr-webservice: GPU not being utilized

Hello,

Great project! For the most part things are working but even when using the GPU docker images, I don’t see any GPU activity. Also doesn’t seem like the model is even getting loaded on the GPU. I’m starting a docker container like so:

docker run -d -p 9000:9000 --gpus all onerahmet/openai-whisper-asr-webservice:latest-gpu

Additionally, I know the GPUs are visible in the docker container:

root@0de2b16a94e1:/app# nvidia-smi
Wed May 17 02:25:33 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.182.03   Driver Version: 470.182.03   CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:26:00.0 Off |                  N/A |
| 28%   33C    P8    14W / 151W |      0MiB /  8117MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA TITAN RTX    Off  | 00000000:27:00.0 Off |                  N/A |
| 40%   29C    P8     3W / 280W |   1736MiB / 24220MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

Please let me know if I’m missing something.

About this issue

Original URL
State: open
Created a year ago
Comments: 15 (2 by maintainers)

Most upvoted comments

Thanks, but no luck.

@ahmetoner could you share which CUDA chips you’ve tested this build with? I’m wondering if I would need to build with a different cuda version. I’m on a machine with an NVIDIA Titan RTX card and a GeForce GTX 1070.

themantalope on Jun 4, 2023

Could you please test debug image to verify GPU issue is gone?

docker run -d --gpus all -p 9000:9000 -e ASR_MODEL=base onerahmet/openai-whisper-asr-webservice:debug-gpu

ahmetoner on May 19, 2023