tensorflow: TF-TRT Warning: Could not find TensorRT

Issue type

Bug

Have you reproduced the bug with TensorFlow Nightly?

Source

source

TensorFlow version

2.15.0

Custom code

OS platform and distribution

Ubuntu 22.04

Mobile device

No response

Python version

3.11

Bazel version

No response

GCC/compiler version

11.2

CUDA/cuDNN version

12.4

GPU model and memory

RTX 3050 TI

Current behavior?

Hey everybody,

I have tried installing tensorflow from my ubuntu 22.04 terminal with anaconda environment. I am getting Can not find TensorRT. I have a 4 monitor setup with Ubuntu and the 550 NVIDIA driver for the RTX 3050 TI does not work on my machine. I have to run the 535 driver, and I think 12.4 CUDA is supported. It installed the 550 driver automatically and I revert back to the 535 driver. But after all that I still ended up with the error below.

I have tried uninstalling, reinstalling, and can’t get it to work. I am a graduate student taking a Machine Learning course. I am spending more time debugging this issue then actually coding.

Could somebody help me resolve?

Standalone code to reproduce the issue

+---------------------------------------------------------------------------------------+                                                                                                                          
| NVIDIA-SMI 535.161.08             Driver Version: 535.161.08   CUDA Version: 12.2     |                                                                                                                          
|-----------------------------------------+----------------------+----------------------+                                                                                                                          
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |                                                                                                                          
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |                                                                                                                          
|                                         |                      |               MIG M. |                                                                                                                          
|=========================================+======================+======================|                                                                                                                          
|   0  NVIDIA GeForce RTX 3050 ...    Off | 00000000:01:00.0  On |                  N/A |                                                                                                                          
| N/A   66C    P8               8W /  35W |   1011MiB /  4096MiB |      0%      Default |                                                                                                                          
|                                         |                      |                  N/A |                                                                                                                          
+-----------------------------------------+----------------------+----------------------+                                                                                                                          
                                                                                                                                                                                                                   
+---------------------------------------------------------------------------------------+                                                                                                                          
| Processes:                                                                            |                                                                                                                          
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |                                                                                                                          
|        ID   ID                                                             Usage      |                                                                                                                          
|=======================================================================================|                                                                                                                          
|    0   N/A  N/A      2812      G   /usr/lib/xorg/Xorg                          648MiB |                                                                                                                          
|    0   N/A  N/A      3023      G   /usr/bin/gnome-shell                        148MiB |                                                                                                                          
|    0   N/A  N/A     78082      G   ...az/anaconda3/envs/skynet/bin/python        1MiB |                                                                                                                          
|    0   N/A  N/A     78475      G   ...seed-version=20240329-165146.919000       96MiB |                                                                                                                          
|    0   N/A  N/A     79970      G   ...gnu/webkit2gtk-4.0/WebKitWebProcess       68MiB |                                                                                                                          
+---------------------------------------------------------------------------------------+   
(base) thenaz@Skynet:~/Downloads$ lspci | grep -i nvidia
01:00.0 VGA compatible controller: NVIDIA Corporation GA107M [GeForce RTX 3050 Ti Mobile] (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 2291 (rev a1)
(base) thenaz@Skynet:~/Downloads$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver                                                                                                                                                                              
Copyright (c) 2005-2024 NVIDIA Corporation                                                                                                                                                                         
Built on Tue_Feb_27_16:19:38_PST_2024                                                                                                                                                                              
Cuda compilation tools, release 12.4, V12.4.99                                                                                                                                                                     
Build cuda_12.4.r12.4/compiler.33961263_0

Relevant log output

python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"                                                                                  
2024-03-30 14:29:58.668656: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.                                                                                                                                 
2024-03-30 14:29:58.691384: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 
2024-03-30 14:29:58.691405: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered  
2024-03-30 14:29:58.692006: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered                                                                                                                                                                                                                 
2024-03-30 14:29:58.695703: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.                         
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-30 14:29:59.119119: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-03-30 14:29:59.660586: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-03-30 14:29:59.675221: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-03-30 14:29:59.675340: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

About this issue

Original URL
State: open
Created 3 months ago
Reactions: 5
Comments: 18

Most upvoted comments

Wanted to add more onto this, I spent a day trying various WSL Tensorflow installation methods and still no luck.

For context, I’m on Windows 11, with an i7-13700 and an RTX A4000 Series GPU.

Here are my printouts:

nvidia-smi
Fri May  3 09:50:02 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.76.01              Driver Version: 552.22         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX A4000               On  |   00000000:01:00.0 Off |                  Off |
| 45%   37C    P0             35W /  140W |       0MiB /  16376MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

 which python
/home/bplaster/anaconda3/bin/python

interestingly, after using pip install tensorflow[and-cuda] within the WSL environment, I was getting this:

python3 -c "import tensorrt; print(tensorrt.__file__)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'tensorrt'

I attempted to resolve this with: pip install tensorrt

Collecting tensorrt
  Downloading tensorrt-10.0.1.tar.gz (16 kB)
  Preparing metadata (setup.py) ... done
Collecting tensorrt-cu12 (from tensorrt)
  Downloading tensorrt-cu12-10.0.1.tar.gz (18 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: tensorrt, tensorrt-cu12
  Building wheel for tensorrt (setup.py) ... done
  Created wheel for tensorrt: filename=tensorrt-10.0.1-py2.py3-none-any.whl size=16333 sha256=19d103fecc3bfb19836fe815d369e65171978f61bb476e89ae84b857953f64e7
  Stored in directory: /home/bplaster/.cache/pip/wheels/31/90/ef/53ad98d9a1bd660c0177aa1ea91bde288edfbe4d15621ca472
  Building wheel for tensorrt-cu12 (setup.py) ... done
  Created wheel for tensorrt-cu12: filename=tensorrt_cu12-10.0.1-py2.py3-none-any.whl size=17549 sha256=5877c62938e16384e9c969b00a94dea5bda1bea7de2db0249bcfd8520d5082fd
  Stored in directory: /home/bplaster/.cache/pip/wheels/c5/08/30/a058eaf14eeeabb3799e3abd3a2a534649123824dcd9695876
Successfully built tensorrt tensorrt-cu12
Installing collected packages: tensorrt-cu12, tensorrt
Successfully installed tensorrt-10.0.1 tensorrt-cu12-10.0.1

Checking the nvidia compatibility matrrix, this should be good to go:

Now, we see that TensorRT is there when called:

python3 -c "import tensorrt; print(tensorrt.__file__)"
/home/bplaster/anaconda3/lib/python3.11/site-packages/tensorrt/__init__.py

Yet we still get the following message when checking the Tensorflow version:

python -c "import tensorflow as tf; print(tf.__version__)"
2024-05-03 09:51:40.606740: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-03 09:51:40.632018: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-03 09:51:41.004536: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2.16.1

Finally, confirming that the GPU cannot be seen by Tensorflow for some reason:

python -c "import tensorflow as tf; print(tf.config.list_physical_devices())"
2024-05-03 10:01:27.648034: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-03 10:01:27.673662: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-03 10:01:28.043090: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-05-03 10:01:30.488133: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:984] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-05-03 10:01:30.501591: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2251] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]

This is all done on a completely fresh WSL2 Ubuntu environment, all done in the (base) conda environment. Hope this is helpful.

bplaster-neogen on May 3, 2024

I have same issue on python 3.8 image on docker

" tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT"

vahidsabet on Mar 31, 2024

Just commenting to say I am experiencing the exact same issue. Some follow up information.

which python3
>>> /home/bplaster/anaconda3/bin/python3

python3 -c "import tensorrt; print(tensorrt.__file__)"
>>> /home/bplaster/anaconda3/lib/python3.11/site-packages/tensorrt/__init__.py

python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices())"
>>> 2024-05-02 15:35:07.243819: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-02 15:35:07.269922: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-02 15:35:07.628229: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-05-02 15:35:08.153905: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:984] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-05-02 15:35:08.156012: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2251] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]

nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

 nvidia-smi
Thu May  2 15:46:11 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.76.01              Driver Version: 552.22         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX A4000               On  |   00000000:01:00.0 Off |                  Off |
| 41%   37C    P8              6W /  140W |      10MiB /  16376MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

bplaster-neogen on May 2, 2024

i think pip install tensorflow[cuda] will solve the issue but i am not quite sure

SigireddyBalasai on Apr 1, 2024