tensorflow: GpuLaunchKernel errors with Internal: invalid device function

System information

  • Windows 10
  • TensorFlow GPU installed from PIP installer:
  • TensorFlow version: 2.0.0
  • Python version: 3.7.4
  • CUDA/cuDNN version: 10.1
  • GPU model and memory: Geforce GTX 1050 4GB

Describe the current behavior

I’m trying to run a facial recognition project and the message bellow appears

Using TensorFlow backend.

2019-10-24 14:32:44.154323: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
2019-10-24 14:32:51.960166: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2019-10-24 14:32:52.321115: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce GTX 1050 major: 6 minor: 1 memoryClockRate(GHz): 1.493
pciBusID: 0000:01:00.0
2019-10-24 14:32:52.325985: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check.
2019-10-24 14:32:52.331740: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-10-24 14:32:52.335877: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-10-24 14:32:52.342803: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce GTX 1050 major: 6 minor: 1 memoryClockRate(GHz): 1.493
pciBusID: 0000:01:00.0
2019-10-24 14:32:52.346290: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check.
2019-10-24 14:32:52.350583: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-10-24 14:32:52.458458: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-10-24 14:32:52.461435: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
2019-10-24 14:32:52.462925: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
2019-10-24 14:32:52.467353: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3083 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050, pci bus id: 0000:01:00.0, compute capability: 6.1)
2019-10-24 14:32:52.647343: F .\tensorflow/core/kernels/random_op_gpu.h:227] Non-OK-status: GpuLaunchKernel(FillPhiloxRandomKernelLaunch<Distribution>, num_blocks, block_size, 0, d.stream(), gen, data, size, dist) status: Internal: invalid device function

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 15

Most upvoted comments

Good news, I tested again now and it seems that this issue has been fixed in the latest tf-nightly-gpu. @glaucimarfaleiro , can you please check as well?