tensorflow: NotFoundError: No algorithm worked! when using Conv2D

Adding the following lines fixes the issue:

from tensorflow.compat.v1 import ConfigProto
from tensorflow.compat.v1 import InteractiveSession

config = ConfigProto()
config.gpu_options.allow_growth = True
session = InteractiveSession(config=config)

Odd, because I didnt need them before.

This is needed for the computer with the RTX 2070 Super. The one with GTX 1080 TI doesnt need them. Same libraries.

same issue here with docker image tensorflow/tensorflow:2.4.0-gpu-jupyter RTX 2070 Super. P.S. Restarting the kernel and adding the code below resolves the issue.

import tensorflow as tf
physical_devices = tf.config.list_physical_devices('GPU') 
tf.config.experimental.set_memory_growth(physical_devices[0], True)

Adding the following lines fixes the issue:

from tensorflow.compat.v1 import ConfigProto
from tensorflow.compat.v1 import InteractiveSession

config = ConfigProto()
config.gpu_options.allow_growth = True
session = InteractiveSession(config=config)

Odd, because I didnt need them before.

This is needed for the computer with the RTX 2070 Super. The one with GTX 1080 TI doesnt need them. Same libraries.

Yes!!! This solved my problem on RTX 2070 card!

I faced this very same error and turns out that when you read a Grayscale Image as an RGB Image (as I was using ImageDataGenerator and did not provide color_mode="grayscale" as an argument to the generator when flowing the data), it throws the “Algorithm did not work” sort of error.

Hopefully it helps someone!

I had the exact same error but for a much more stupid mistake than people mentioned above. Model was defined with input shape (height, width, 1) but images were RGB, not grayscale.

I see there is a “solution”, but this should never happen in a popular library in the first place. Why is this happening, and why users need to add this extra code to get tensorflow to work? I have another process that uses 6gb memory and for some reason tf cannot simply coexist with it, when I kill it the super helpful error message “UNIMPLEMENTED: DNN library is not found” disappears. I can understand that things can be more efficient if you map the whole gpu memory, but in case of issues at least it would be nice to fail with clear error messages. No issues with pytorch. Ehh, I can only hope Jax works out much better.

Btw, for me the following worked (limiting the memory, to 2gb, single gpu case):

gpus = tf.config.list_physical_devices('GPU')
tf.config.set_logical_device_configuration(
    gpus[0],
    [tf.config.LogicalDeviceConfiguration(memory_limit=2048)])

Same here today; the extra config lines for gpu_options.allow_growth = True fixed the “NotFoundError: No algorithm worked!” issue for me for code that was running fine on one PC but gave the above problem on another PC with AFAIK the same TensorFlow configuration, same libraries etc but a different GPU (both NVIDIA).

OS: Windows 10 64-bit TF: 2.4.1 GPU: NVIDIA Quadro T1000 with Max-Q Design (on the other Windows 10 64-bit system PC w/o problems it was NVIDIA Quadro P1000) Python: 3.7.6 CUDA: 11.2 nvcc --version: 11.2, V11.2.67 (on the PC w/o problems: 11.2, V11.2.142) RAM: 32GB (on the PC w/o problems 16GB)

Why is this fix needed only in some cases and why does it work?

BTW, the shorter “os.environ[‘TF_FORCE_GPU_ALLOW_GROWTH’] = ‘true’” also worked fine for me.

This might be too late but still be of help to post it here since I too faced the exact same issue. For me, I just restricted the amount of GPU that the model was allowed to use. Also, the way I did is an older way to do it which I did it in tf 1.15 but it for sure works on the latest tf 2.4 version as well. Below is the code:

def set_tensorflow_config(per_process_gpu_memory_fraction=0.8):
    config = tf.compat.v1.ConfigProto()
    # config = tf.ConfigProto()
    config.gpu_options.per_process_gpu_memory_fraction = per_process_gpu_memory_fraction
    config.gpu_options.allow_growth=True
    # sess = tf.Session(config=config)
    sess = tf.compat.v1.Session(config=config)
    
    print("== TensorFlow Config options set ==")
    print("\nThis process will now utilize {} GPU Memeory Fraction".format(per_process_gpu_memory_fraction))

set_tensorflow_config()

GPU: NVIDIA GeForce RTX 2070, 8GB

Also double check CUDA versions cause CUDA 11 Is on nightly. I don’t know what Arch Linux maintainers done but in that case It Is an Arch Linux issue. See https://github.com/tensorflow/tensorflow/issues/40227

gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
  # Restrict TensorFlow to only allocate 4GB of memory on the first GPU
  try:
    tf.config.experimental.set_virtual_device_configuration(
        gpus[0],
        [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=4096)])
    logical_gpus = tf.config.experimental.list_logical_devices('GPU')
    print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs")
  except RuntimeError as e:
    # Virtual devices must be set before GPUs have been initialized
    print(e)

This solution worked for me…Thanks for the help…

To check and will solve easily:

1. Check your image input, maybe you are wrong with dimensions (usually channels) and all
2. Out of GPU memory

Steve

In my case, this error appears because the memory is full. Try to check nvidia-smi via terminal. In my case, I use cloud server using jupyter. Shutdown all kernels (not only close the file, but shutdown), and restarting solve the issue.

Hope it helps anyone stumble upon this.

same issue here with docker image tensorflow/tensorflow:2.4.0-gpu-jupyter RTX 2070 Super. P.S. Restarting the kernel and adding the code below resolves the issue.

import tensorflow as tf
physical_devices = tf.config.list_physical_devices('GPU') 
tf.config.experimental.set_memory_growth(physical_devices[0], True)

Same issue on RTX 2060 Ti with TF 2.4.1 If I upgrade TF 2.4.1 to tf-nightly, the missing CUDA library error occurs.

The proposed above solution was best for me. But still, it has drawbacks that limit the GPU memories. It could be a temporary solution.

Worked for me: OS: Ubuntu 18.04 TensorFlow version: 2.4.1 Python version: 3.7.10 CUDA/cuDNN version: Cuda is 11.0, cuDNN is 8.0.4 GPU model and memory: GeForce GTX 1660 SUPER

Still doesn’t make any sense, thanks anyway 😃

same issue here with docker image tensorflow/tensorflow:2.4.0-gpu-jupyter RTX 2070 Super. P.S. Restarting the kernel and adding the code below resolves the issue.

import tensorflow as tf
physical_devices = tf.config.list_physical_devices('GPU') 
tf.config.experimental.set_memory_growth(physical_devices[0], True)

Same issue on RTX 2060 Ti with TF 2.4.1 If I upgrade TF 2.4.1 to tf-nightly, the missing CUDA library error occurs.

The proposed above solution was best for me. But still, it has drawbacks that limit the GPU memories. It could be a temporary solution.

same issue here with docker image tensorflow/tensorflow:2.4.0-gpu-jupyter RTX 2070 Super. P.S. Restarting the kernel and adding the code below resolves the issue.

import tensorflow as tf
physical_devices = tf.config.list_physical_devices('GPU') 
tf.config.experimental.set_memory_growth(physical_devices[0], True)

Same worked for me on 2070 Super tf 2.4.1 Thanks

Closing as stale. Please reopen if you’d like to work on this further.