tensorflow: cuBLAS Error in 2.14.0

tensorflow is a disappointment

Same issue with Ubuntu 22.04 + Cuda 12.2 + cudnn 8.9 + build from source

alex@pc:~$ python
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
2023-10-09 22:31:24.914587: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2023-10-09 22:31:24.914620: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2023-10-09 22:31:24.914625: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-10-09 22:31:24.919056: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

@BinyanHu , Could you please try in the Tensorflow 2.15rc0

Tried to install tf2.15 on Unbuntu with python3.11 using:

pip install tensorflow[and-cuda]

and

conda install tensorflow[and-cuda]

Both did not install tf2.15 but tf2.13 or 2.14. I guess that is because cudatoolkit 12.2 (which tf2.15 depends on) is not yet available. (p.s. I am using an HPC with no admin, thus relying on conda to handle cuda dependencies.)

In case pip install tensorflow[and-cuda] may not find the proper source of cuda dependencies, I tried to install tf2.14 and cuda dependencies manually using:

conda install cudatoolkit=11.8
pip install nvidia-cudnn-cu11==8.7.0.84
conda install -c nvidia cuda-nvcc
pip install tensorflow==2.14

The three lines of errors Unable to register ... persist.

Same issue with v2.15

This problem occured in version 2.10 and 2.11, then repaired since 2.12, but come again in 2.14.

same here, using nvidia 535 on ubuntu 22.04, install tf using pip install tensorflow[and-cuda] and got the same errors.

i have the same problem win11/wsl2/tensorflow 2.14.0

Any luck on this ?

I seem to have exact problem on win10/wsl2/tensorflow2.14/

These errors are not impacting performance!!!

Tested it on ResNet50 google colab V100 vs local rtx 3090 yield almost the same performance.

https://github.com/tensorflow/tensorflow/issues/62075#issuecomment-1867470232

I think it’s a problem by design. TF has stopped supporting gpu under windows since 2.10…so don’t break your head anymore …

On Wed, 15 Nov 2023 at 9:05 AM, Sebastián Segovia @.***> wrote:

Same error happening with Windows 11, WSL2, and Tensorflow 2.14.1 with an RTX 2060

2023-11-14 21:31:07.995539: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2023-11-14 21:31:07.995626: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2023-11-14 21:31:07.995681: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered

Downgrading to another version of Tensorflow did not work

— Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/62002#issuecomment-1811756806, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPLTDRMG36MFVDGULUWMOTYEQZ6NAVCNFSM6AAAAAA5KZQL6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJRG42TMOBQGY . You are receiving this because you commented.Message ID: @.***>

Same error happening with Windows 11, WSL2, and Tensorflow 2.14.1 with an RTX 2060

2023-11-14 21:31:07.995539: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2023-11-14 21:31:07.995626: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2023-11-14 21:31:07.995681: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered

Downgrading to another version of Tensorflow did not work

I get the same error. rtx 3090 Driver Version: 536.99 cuda: 12.2