tensorflow: tensorrt==8.5.3.1 from [and-cuda] not available in Python 3.11

As of the latest tensorflow release 2.15, I was able to successfully install it using:

python -m pip install "tensorflow[and-cuda]==2.15" --extra-index-url https://pypi.nvidia.com

I ran a quick test and so far, it seems like everything is working fine. I think that it would be nice to fix the package pinning so that the extra-index doesn’t have to be used, but this is farther than I was able to get with TF2.14.

EDIT: This was tested on Python 3.11 under Linux. It appears that tensorrt was bumped to 8.6.x so it appears to work as intended.

Hi @poulsbo , Could you please help triage this issue to the right person?

Using something similar to the following: $ strace python -c "import tensorflow" 2>&1 | grep libnvinfer, it appears that TF2.15 is searching for libnvinfer.so.8.6.1 and libnvinfer_plugin.so.8.6.1.

When I manually created the symbolic links libnvinfer.so.8.6.1 -> libnvinfer.so.8 and libnvinfer_plugin.so.8.6.1 -> libnvinfer_plugin.so.8, the warning went away. (LD_LIBRARY_PATH still needs to be set)

For TensorRT 8.6 Python packages, TF’s RUNPATH would need to have been updated; I’m not sure if that happened when updating the TRT version dependency in TF? This can be worked around with either symlinks or patchelf to update the runpath or just LD_LIBRARY_PATH – definitely easier than the issue with TF 2.14 and TRT 8.5.

Can we pick one or the other to go after here?

patchelf is unfortunately not available on my servers, and it would take sometime to get it through the chain to get it approved. LD_LIBRARY_PATH OTOH can probably updated with conda envs (or .bashrc) so I am learning towards that. symlinks are also an easy option and I don’t mind them either. @cliffwoolley Can you provide the some instructions for both of these? I am not fully familiar with TensorRT, so I don’t exactly what files need to be symlinked/LD’ed.

@stallam-unb this is great! Thanks! Do you know what CUDA/CuDNN versions does 2.15 use? I don’t see an entry for it on the webpage https://www.tensorflow.org/install/source

I confirmed with our TensorRT team that TRT 8.5 did not support Python 3.11. This was a matter of timing of release dates: TensorRT 8.5 was first released in early November 2022, and Python 3.11 had only been out for a few days by then, so it didn’t get onto that TensorRT release’s support matrix in time.

TRT 8.6 does support Python 3.11, and our TensorRT releases generally do maintain binary backward compatibility as per semantic versioning, though as it happens here the python packaging was redone between 8.5 and 8.6, so even if you could force pip to ignore the == dependency baked into TensorFlow (which I didn’t find any particularly straightforward way to do), the binaries for TRT are installed to a different path with 8.6 than with 8.5, and the RUNPATH settings baked into libtensorflow_cc.so would need updating to use the 8.6 pip package.

TensorFlow does actually act appropriately if the TRT library isn’t present at runtime, even though pip is treating it as a hard dependency. (There’s no such thing in pip as an ‘optional dependency’.) It’s possible to make a dummy package that tricks pip into moving forward without installing TRT, and then you can run TF (just without TF-TRT). Is that useful?

Cliff Woolley DL Frameworks Engineering, NVIDIA

Can we see anywhere that the “TensorFlow GPU team” has acknowledged this issue? Can we expect 2.15.0 to fix?

Doesn’t seem like it, even though there have been multiple issues that have been raised that are related to this. I am a bit disappointed to see that it has been over a month since the release of TF2.14 and it is not possible to even install the latest version. I am not sure how it hasn’t been caught in tests. This should be considered a high-priority issue, and we have yet to see any indication of an acknowledgement that the team is aware of the issue, or an estimated timeline for the issue to get fixed.

@SuryanarayanaY Please assign to the TensorFlow GPU team.