tensorflow: [Windows] Tensorflow GPU fails to find CUDA.
NOTE: Only file GitHub issues for bugs and feature requests. All other topics will be closed.
For bugs or installation issues, please provide the following information. The more information you provide, the more easily we will be able to offer help and advice.
What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?
NONE (Tensorflow for Windows is very new).
Environment info
Operating System: Windows 8.1 (Conda 4.2.9)
conda --version conda 4.2.9
Installed version of CUDA and cuDNN:
(please attach the output of ls -l /path/to/cuda/lib/libcud*
):
nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2016 NVIDIA Corporation Built on Sat_Sep__3_19:05:48_CDT_2016 Cuda compilation tools, release 8.0, V8.0.44
If installed from binary pip package, provide:
-
A link to the pip package you installed:
pip install --upgrade https://storage.googleapis.com/tensorflow/windows/gpu/tensorflow_gpu-0.12.0rc0-cp35-cp35m-win_amd64.whl
-
The output from
python -c "import tensorflow; print(tensorflow.__version__)"
.
python -c "import tensorflow; print(tensorflow.__version__)" 0.12.0-rc0 I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library cublas64_80.dll locally I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:119] Couldn't open CUDA library cudnn64_5.dll I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_dnn.cc:3459] Unable to load cuDNN DSO I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library cufft64_80.dll locally I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library nvcuda.dll locally I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library curand64_80.dll locally
If possible, provide a minimal reproducible example (We usually don’t have time to read hundreds of lines of your code)
What I did: I attempt to run some tensorflow code that works within ubuntu CPU instance.
Code: MNIST simple autoencoder (requires MNIST dataset); CODE
Error Message error_message:
In [10]: %run CNN_autoencoder.py
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stre
am_executor\dso_loader.cc:128] successfully opened CUDA library cublas64_80.dll
locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stre
am_executor\dso_loader.cc:119] Couldn't open CUDA library cudnn64_5.dll
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stre
am_executor\cuda\cuda_dnn.cc:3459] Unable to load cuDNN DSO
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stre
am_executor\dso_loader.cc:128] successfully opened CUDA library cufft64_80.dll l
ocally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stre
am_executor\dso_loader.cc:128] successfully opened CUDA library nvcuda.dll local
ly
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stre
am_executor\dso_loader.cc:128] successfully opened CUDA library curand64_80.dll
locally
Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core
\common_runtime\gpu\gpu_device.cc:885] Found device 0 with properties:
name: GeForce GTX 965M
major: 5 minor: 2 memoryClockRate (GHz) 0.9495
pciBusID 0000:01:00.0
Total memory: 2.00GiB
Free memory: 1.86GiB
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core
\common_runtime\gpu\gpu_device.cc:906] DMA: 0
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core
\common_runtime\gpu\gpu_device.cc:916] 0: Y
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core
\common_runtime\gpu\gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (d
evice: 0, name: GeForce GTX 965M, pci bus id: 0000:01:00.0)
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core
\common_runtime\gpu\gpu_device.cc:586] Could not identify NUMA node of /job:loca
lhost/replica:0/task:0/gpu:0, defaulting to 0. Your kernel may not have been bu
ilt with NUMA support.
WARNING:tensorflow:From C:\Users\Gregoty\Programming\cs231n\repo\project\tensorf
low\autoencoder\CNN_autoencoder.py:188 in <module>.: initialize_all_variables (f
rom tensorflow.python.ops.variables) is deprecated and will be removed after 201
7-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
number of test = 10000
number of train = 55000
number_of validation = 5000
Done splitting up test data set;
Starting training loop.
Epoch: 0
Shuffling the training data;
F c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stre
am_executor\cuda\cuda_dnn.cc:221] Check failed: s.ok() could not find cudnnCreat
e in cudnn DSO; dlerror: cudnnCreate not found
What other attempted solutions have you tried?
Tried running it through ipython
(conda), python
(initiated by msys2)
Logs or other output that would be helpful
(If logs are large, please upload as attachment or provide link).
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 30 (6 by maintainers)
Thanks. This was the problem (I hadn’t placed cudnn in the proper location) - I copied it into the nVidia CUDA install location. I downloaded Windows 7 Version cuDNN - incase other people are wondering.
I then copied the
cudnn64_5.dll
(cuda\bin\cudnn64_5.dll
) from that zip archive intoC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin\
;cuda\include\cudnn.h
toC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\include\
;and
cuda\lib\x64\cudnn.lib
toC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64\
WHERE
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0
is my install PATH for the CUDA toolkit.I had already added
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin\
to my PATH (you need to make sure this is done too).This should be added to the install instructions for running Tensorflow with Windows.
And dont forget to restart.
as of 14/4/2017 if you’re downloading the cuDNN
[Download cuDNN v6.0 (March 23, 2017), for CUDA 8.0]
pay attention that TF will still look for cudnn64_5.dll , while the cuDNN contains cudnn64_6.dll I was able to fix this by simply renaming the file cudnn64_6.dll -> cudnn64_5.dllA few quick questions to figure out the problem:
cudnn64_5.dll
.)%PATH%
environment variable?%PATH%
environment variable, does that fix the problem?renaming the file cudnn64_6.dll -> cudnn64_5.dll
And dont forget to restart. it is really important, which waste me 30 min…