tensorflow: TF-TRT Warning: Could not find TensorRT
Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
No
Source
source
TensorFlow version
2.15.0
Custom code
No
OS platform and distribution
Ubuntu 22.04
Mobile device
No response
Python version
3.11
Bazel version
No response
GCC/compiler version
11.2
CUDA/cuDNN version
12.4
GPU model and memory
RTX 3050 TI
Current behavior?
Hey everybody,
I have tried installing tensorflow from my ubuntu 22.04 terminal with anaconda environment. I am getting Can not find TensorRT. I have a 4 monitor setup with Ubuntu and the 550 NVIDIA driver for the RTX 3050 TI does not work on my machine. I have to run the 535 driver, and I think 12.4 CUDA is supported. It installed the 550 driver automatically and I revert back to the 535 driver. But after all that I still ended up with the error below.
I have tried uninstalling, reinstalling, and can’t get it to work. I am a graduate student taking a Machine Learning course. I am spending more time debugging this issue then actually coding.
Could somebody help me resolve?
Standalone code to reproduce the issue
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.08 Driver Version: 535.161.08 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 3050 ... Off | 00000000:01:00.0 On | N/A |
| N/A 66C P8 8W / 35W | 1011MiB / 4096MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 2812 G /usr/lib/xorg/Xorg 648MiB |
| 0 N/A N/A 3023 G /usr/bin/gnome-shell 148MiB |
| 0 N/A N/A 78082 G ...az/anaconda3/envs/skynet/bin/python 1MiB |
| 0 N/A N/A 78475 G ...seed-version=20240329-165146.919000 96MiB |
| 0 N/A N/A 79970 G ...gnu/webkit2gtk-4.0/WebKitWebProcess 68MiB |
+---------------------------------------------------------------------------------------+
(base) thenaz@Skynet:~/Downloads$ lspci | grep -i nvidia
01:00.0 VGA compatible controller: NVIDIA Corporation GA107M [GeForce RTX 3050 Ti Mobile] (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 2291 (rev a1)
(base) thenaz@Skynet:~/Downloads$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Feb_27_16:19:38_PST_2024
Cuda compilation tools, release 12.4, V12.4.99
Build cuda_12.4.r12.4/compiler.33961263_0
Relevant log output
python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
2024-03-30 14:29:58.668656: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-03-30 14:29:58.691384: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-03-30 14:29:58.691405: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-03-30 14:29:58.692006: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-03-30 14:29:58.695703: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-30 14:29:59.119119: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-03-30 14:29:59.660586: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-03-30 14:29:59.675221: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-03-30 14:29:59.675340: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 8
- Comments: 49
i think pip install tensorflow[cuda] will solve the issue but i am not quite sure
Wanted to add more onto this, I spent a day trying various WSL Tensorflow installation methods and still no luck.
For context, I’m on Windows 11, with an i7-13700 and an RTX A4000 Series GPU.
Here are my printouts:
interestingly, after using
pip install tensorflow[and-cuda]within the WSL environment, I was getting this:I attempted to resolve this with:
pip install tensorrtChecking the nvidia compatibility matrrix, this should be good to go:
Now, we see that TensorRT is there when called:
Yet we still get the following message when checking the Tensorflow version:
Finally, confirming that the GPU cannot be seen by Tensorflow for some reason:
This is all done on a completely fresh WSL2 Ubuntu environment, all done in the (base) conda environment. Hope this is helpful.
I’m frustrated with tensorflow, 1 decade after launch and its still a nightmare to get it to work with gpu.
EDIT: hours later, I created fresh venv and installed tensorflow[and-cuda] alone, then installed tensorrt. The error in this gituhub issue vanished, but still no gpu. After that, I had to do path manipulation as indicated here and it worked.
Setting TF_CPP_MAX_VLOG_LEVEL=3 in my environment (WSL2 Ubuntu) gives the following specific error:
While I have TensorRT, and thus libnvinfer, installed, the actual libraries are named libnvinfer.so.10 and libnvinfer_plugin.so.10 Just as a test, I did the following:
And the warning about not finding TensorRT diapered. Of course, if the libraries are not binary compatible, this might lead to further issues down the line, but at least it hints at what the problem is.
I have same issue on python 3.8 image on docker
" tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT"
Hi, I was able to get it fixed. Here are the steps:
*now go to the extracted folder and install tensorrt from there
Hope this helps, i was using python 3.10 tensor rt folder has lib for other python versions to
Couldn’t resolve TF-TRT Warning: Could not find TensorRT
I’m using wsl2 in windows 11.
Previously, I wasn’t able to have GPU as the backend, I had tried all the methods in installing tensorflow in wsl.
But, trying with python3.11 and tensorflow 2.15.1, got me access to the GPU backend.
versions:
nvidia-smi:
I’ve tried all ways of installing tensorrt - pip, .deb, .tar; updated path variable, etc. Its importing in python shell
All the files are available (.tar installation method)
I’ve also used the files from .tar method to install using pip:
But still I get this error
Got exactly into the same problem with Python 3.11, CUDA 12.4 and TensorRT 10.1. Tried suggestion by @Zilla - it appears it’s a working solution / workaround.
What I did:
Then I did:
My application is not complaining about missing TensorRT library.
Yes. It works. I have the miniconda env “cuda12”, cuda 12, tensorrt 10 on Ubuntu 22, WSL2.
That is my commands:
So, after trying out pretty much everything else, I can confirm that this worked out and my GPU is now also detected. I am on Ubuntu 22.0.4.4 running Tensorflow 2.16.
Many thanks @BigKhalPablo, saved my day 😉
Cheers, Eric
Same issue:
My nvidia drivers and cuda version:
I’ve installed tensorflow and tensorrt through pip:
I have follow the guide: Install TensorFlow with pip and added path as in Install Tensorflow & Pytorch with CUDA [Linux | WSL2] but I’m still getting the same warning (TF-TRT Warning: Could not find TensorRT).
Does any anyone managed to find a solution?
Just commenting to say I am experiencing the exact same issue. Some follow up information.
Hi @MachinesOnTheRise ,
You might need to install compatible TensorRT version from Nvidia. Since this is a optimization library for Inferences and developed by Nvidia for their specific GPUs, it may not come bundled with TF package. You can use
tensorflow[and-cuda]package for installing TF with required cuda/cudnn packages as below.But they seems don’t have tensorrt bundled.https://github.com/tensorflow/tensorflow/blob/ced28148be8cec1f234886f21c7e528c374032b8/tensorflow/tools/pip_package/setup.py#L151-L166
Thank you all for your help. I was able to resolve it by adding the tensorrtlibs to LD_LIBRARY_PATH.
I got exactly the same question like you, any ideas?
I think that the only way is to find the compatible versions for python, tensorflow, nvidia driver, cuda, cudnn and tensorrt. I’ll try the tested build configurations from here in WSL and Ubuntu. Hope it works 🙏