tensorflow: ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory
I installed tf-nightly build and I get the following error on import of tensorflow.
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory
.
If I check for cuda 9, I get the following:
ldconfig -v
/usr/local/cuda-8.0/targets/x86_64-linux/lib:
libnvgraph.so.8.0 -> libnvgraph.so.8.0.61
libnppicom.so.8.0 -> libnppicom.so.8.0.61
libnppial.so.8.0 -> libnppial.so.8.0.61
libcufftw.so.8.0 -> libcufftw.so.8.0.61
libcufft.so.8.0 -> libcufft.so.8.0.61
libnppif.so.8.0 -> libnppif.so.8.0.61
libcublas.so.8.0 -> libcublas.so.8.0.88
libnvblas.so.8.0 -> libnvblas.so.8.0.88
libnppi.so.8.0 -> libnppi.so.8.0.61
libcusolver.so.8.0 -> libcusolver.so.8.0.61
libnppidei.so.8.0 -> libnppidei.so.8.0.61
libnvrtc-builtins.so.8.0 -> libnvrtc-builtins.so.8.0.61
libnvrtc.so.8.0 -> libnvrtc.so.8.0.61
libnpps.so.8.0 -> libnpps.so.8.0.61
libcuinj64.so.8.0 -> libcuinj64.so.8.0.61
libnppig.so.8.0 -> libnppig.so.8.0.61
libOpenCL.so.1 -> libOpenCL.so.1.0.0
libnppicc.so.8.0 -> libnppicc.so.8.0.61
libnppist.so.8.0 -> libnppist.so.8.0.61
libnppisu.so.8.0 -> libnppisu.so.8.0.61
libnppim.so.8.0 -> libnppim.so.8.0.61
libcurand.so.8.0 -> libcurand.so.8.0.61
libcudart.so.8.0 -> libcudart.so.8.0.61
libnvToolsExt.so.1 -> libnvToolsExt.so.1.0.0
libnppitc.so.8.0 -> libnppitc.so.8.0.61
libnppc.so.8.0 -> libnppc.so.8.0.61
libcusparse.so.8.0 -> libcusparse.so.8.0.61
/usr/local/cuda-9.1/targets/x86_64-linux/lib:
libnppicc.so.9.1 -> libnppicc.so.9.1.85
libnppisu.so.9.1 -> libnppisu.so.9.1.85
libcufftw.so.9.1 -> libcufftw.so.9.1.85
libcufft.so.9.1 -> libcufft.so.9.1.85
libnppial.so.9.1 -> libnppial.so.9.1.85
libnppist.so.9.1 -> libnppist.so.9.1.85
libcublas.so.9.1 -> libcublas.so.9.1.85
libnvblas.so.9.1 -> libnvblas.so.9.1.85
libnppitc.so.9.1 -> libnppitc.so.9.1.85
libcusolver.so.9.1 -> libcusolver.so.9.1.85
libnvrtc.so.9.1 -> libnvrtc.so.9.1.85
libnvrtc-builtins.so.9.1 -> libnvrtc-builtins.so.9.1.85
libnppidei.so.9.1 -> libnppidei.so.9.1.85
libOpenCL.so.1 -> libOpenCL.so.1.0.0
libnppig.so.9.1 -> libnppig.so.9.1.85
libnppc.so.9.1 -> libnppc.so.9.1.85
libcudart.so.9.1 -> libcudart.so.9.1.85
libnvToolsExt.so.1 -> libnvToolsExt.so.1.0.0
libnvgraph.so.9.1 -> libnvgraph.so.9.1.85
libnppif.so.9.1 -> libnppif.so.9.1.85
libcusparse.so.9.1 -> libcusparse.so.9.1.85
libaccinj64.so.9.1 -> libaccinj64.so.9.1.85
libcuinj64.so.9.1 -> libcuinj64.so.9.1.85
libnppim.so.9.1 -> libnppim.so.9.1.85
libnppicom.so.9.1 -> libnppicom.so.9.1.85
libnpps.so.9.1 -> libnpps.so.9.1.85
libcurand.so.9.1 -> libcurand.so.9.1.85
I that due to a name mismatch. libcublas.so.9.0 =! libcublas.so.9.1
? And if so how can we overcome this?
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 80
- Comments: 142 (4 by maintainers)
Have you solved it ? This problem is caused tensorflow-gpu-1.5 required cuda 9.0 ,so you should install tensorflow-gpu-1.4. And rember uninstall tensorflow-gpu-1.5. Please use this"pip install --upgrade tensorflow-gpu==1.4"
Even better solution - remove tensorflow, install pytorch Why tensorflow can brake so easily? Why can’t I just install it and run? Why is it so easy with pytorch and unpredictable with tf? Wtf is wrong with it?
Someone needs to fix the https://www.tensorflow.org/install/install_linux page if this is true, I just followed its instructions exactly, and tells you to install CUDA 8.0 (specifically, not “latest CUDA”).
Then as soon as you’re done, you get this error (it is looking for cublas 9.0, which, from what I can read here, would not have worked either, as CUDA 9.1 is the default you get from NVIDIA).
Either the webpage instructions should work with the default latest of everything, or it should tell you explicitly to install tensorflow-gpu-1.4 (for example) and not tensorflow-gpu…
Why not just install cuda-9-0?
@aipeteryao This fixed it, thanks! I ended up uninstalling the latest version and installing 1.4, in my virtualenv.
The install page for Ubuntu should be updated: https://www.tensorflow.org/install/install_linux Since TensorFlow 1.5 is expecting Cuda 9.0 ( NOT 9.1 ), as well as cuDNN 7
@asimshankar Would like to know that in your above comment you mean that we should downgrade cuda to 9.0 and tensorflow 1.5 doesn’t work with cuda 9.1 ?
You have closed this issue but its not clear what is the correct action that we should take!
Note: I also have cuda 9.1 installed instead of cuda 9.0.
I think this is due to the fact that you have CUDA 9.1 and not 9.0, I am facing exactly the same issue.
@Timonzimm I know and I think the whole issue is this f** naming libcublas.so.xxx that nvidia puts. This inherently is mismatch on linux systems whenever that number changes, so since it can not find the exact matches then it thinks the file doesn’t exist and throws the error.
Seems like this is resolved (and the root cause was a version mismatch of CUDA - 9.0 vs 9.1)? Closing this out since I understand it to be resolved, but please let me know if I’m mistaken.
FYI @gunan @av8ramit (who are working on the upcoming 1.5 release)
I’m getting this issue (Cuda 9.1.85, cuDNN 7.05)
Tried with tensorflow 1.5, it broke. Uninstalled, installed 1.4 with
pip3 install --upgrade tensorflow-gpu==1.4
, still broke.This is still an issue with Cuda 9.1 and Tensorflow 1.11
@burui11087 I completely forgot about symlinking. Thanks for reminding me.
cuda 9.1 is the current version. I want TF to use it. How to?
I encounter this problem as well when I use PyCharm to debug my code at remote server(aws deep learning ami). The cuda version is 9.0 and the cudnn is 7.1.4 and tensorflow is 1.10.0(for gpu). I can import tensorflow normally by terminal connected to my server but fail to import tensorflow in the python console of PyCharm. The problem occurs at this time. Therefore, I cannot debug with PyCharm… My solution is: input this line in terminal after connecting to your server
And I referenced thisLink
@abrahamrhoffman That’s easy for anyone who has sudo privileges but what about people on a shared system like a cluster environment with simple user privileges. In those cases even if you ask from the sys admin to install any libraries most probably the answer is gonna be NO! Since they are afraid that might interfere with other users’ settings and environments.
@abrahamrhoffman Would you also mind providing a justification on the down vote?
I got Tensorflow 1.11.0 working by running the following commands:
Please refer combinations of CUDA, CuDNN and Tensorflow.
This error happens majorly due to incorrect version combinations of Nvidia-driver, CUDA, CuDNN and Tensorflow-gpu
This works for me (tensorflow-gpu==1.8.0 and cuda version is 9.0, install in anaconda)
suggestion from: https://stackoverflow.com/questions/48428415/importerror-libcublas-so-9-0-cannot-open-shared-object-file
This worked for me:
Download CUDA Toolkit 9.0 from NVidia previous releases section. Then:
Notice 9.0 at the last line above.
export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64
My setup Tensorflow 1.7 cuDNN 7.1.2 Ubuntu 16.04
I was handling with this issue as well. What worked for me with tensorflow-gpu 1.6:
sudo apt-get install cuda-libraries-9-0
as suggested at the end of the installation instructions.I think you should use symbol link from ‘‘cuda/’’ to ''cuda/9.1",or your cuda version is too new to tensorflow master branch
the man has got a point. TF CAN be compiled to work with more modern CUDA versions… why not just offer that to most people with a
tensorflow-gpu-cuda110
package or something temporary until the library can figure out the underlying cuda version dynamically?Stupid solution
I have find the reason is ldconf, ldconfig is a dynamic link library management command whose purpose is to allow the dynamic link library to be usedby the system.
The default ldconf only search /lib and /usr/lib, as well as the library file under the directory listed in the configuration file /etc/ld. so. conf.
so all of this is caused by the dynamic library of CUDA in the installed CUDA path such as : /path/cuda-9.0/lib64 or /path/cuda-9.0/lib. (for eample my CUDA is installed in /usr/local/cuda-9.0)
if you install the CUDA manual, then after install, you should add the path of cuda/lib64 to /etc/ld.so.conf file
sudo echo "/usr/local/cuda-9.0/lib64/" >> /etc/ld.so.conf
thensudo ldconfig
of course , you can add the path manual, like:
vim /etc/ld.so.conf
then add the path ‘/usr/local/cuda-9.0/lib64’ at the end.sudo ldconfig
after the operation, reopen the ipython or pycharm , import tensorflow as tf wow, you will enjoy it!if you install the CUDA by command such as ‘dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb’ or others, it may add the cuda lib path to the /etc/ld.so.conf automatically . but to be on the safe side, check the /etc/ld.so.conf and see if the path add to it .
Somehow, I solved this by installing: cuda 9.1 (from package manager), cudnn 7.1 for 9.1 and from anaconda: by using this default command ‘conda install -c anaconda tensorflow-gpu’ cudatoolkit 9.0, tensorflow 1.7, tensorflow-gpu 1.7
I used Antergos linux, GTX 1060 in my PC. It worked as well in my notebook (Xubuntu 18.04, GT 840m). In my notebook i used : cuda 9.1 (from nvidia ppa), cudnn 7.1 for 9.1 (from nvidia web), and the rest was the same
I am sick of it!!!
I have find the reason is ldconf, ldconfig is a dynamic link library management command whose purpose is to allow the dynamic link library to be usedby the system.
The default ldconf only search /lib and /usr/lib, as well as the library file under the directory listed in the configuration file /etc/ld. so. conf.
so all of this is caused by the dynamic library of CUDA in the installed CUDA path such as : /path/cuda-9.0/lib64 or /path/cuda-9.0/lib. (for example my CUDA is installed in /usr/local/cuda-9.0)
1.if you install the CUDA manual, then after install, you should add the path of cuda/lib64 to /etc/ld.so.conf file
sudo echo "/usr/local/cuda-9.0/lib64/" >> /etc/ld.so.conf
thensudo ldconfig
of course , you can add the path manual, like: vim /etc/ld.so.conf then add the path ‘/usr/local/cuda-9.0’ at the end. then update itsudo ldconfig
after the operation, reopen the ipython or pycharm ,import tensorflow as tf
wow, you will enjoy it!if you install the CUDA by command such as ‘dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb’ or others, it may add the cuda lib path to the /etc/ld.so.conf automatically . but to be on the safe side, check the /etc/ld.so.conf and see if the path add to it .
Please make sure to set your PATH variable appropriately, such as described here: https://stackoverflow.com/questions/39287744/ubuntu-16-04-nvidia-toolkit-8-0-rc-darknet-compilation-error-expected-a/41290056#41290056
I fix him for now by:
Download deb (network) from: https://developer.nvidia.com/cuda-90-download-archive?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604&target_type=debnetwork
Then:
dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
Then: aptitude update
Then: aptitude install cuda-9-0
With cuda 8 and 9.0 installed, setting
LD_LIBRARY_PATH
in.bashrc
and.profile
not work. So, I setLD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64
in pycharm Environment variable field and it works.I installed cuda-9.0 and still it does not work. This is really irritating.
@Timonzimm I am facing the same issue. Have you figured it out?
I guess the problem has cropped up again with
cuda 9.2
andtensorflow-gpu 1.8
. In a virtualenv, I get:My
/usr/local/cuda-9.2/lib64
folder has the following libcublas:Versions:
We’ll need an update to tensorflow-gpu to use cuda 9.2.
Also, if I were to downgrade to cuda 9.0, would I have to first remove cuda 9.2 or just install 9.0 straight away? Would I have conflicting installations?
export PATH=${PATH}:/usr/local/cuda-9.0/bin export CUDA_HOME=${CUDA_HOME}:/usr/local/cuda:/usr/local/cuda-9.0 export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda-9.0/lib64 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64
if use pycharm - add it to interpreter
I think this issue should still be open. @bwesen’s comment is correct. The docs tell you to install Cuda 8.0 and use
pip install --upgrade tensorflow-gpu
. Right now that gives you tensorflow 1.5 which does not work with Cuda 8.0pinging @asimshankar
when I run my code on the linux environment directly, everything is OK. But when I run on the local pycharm through the remote interpreter, I encounter the problem:
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory. "Failed to load the native TensorFlow runtime."
Softlink seems not solve this issue:
Still got:
So I just added sudo apt-get -y install cuda-toolkit-9.0 and I’m up and running with TF1.6
I have the same issue (with cuda 9.1 + tensorflow 1.5). I think to resolve it, one option is that to downgrade cuda to 9.0. The other option would be to downgrade both cuda to 8.0 and tensorflow to 1.4. If you have already installed cuda 8.0, you only need to modify
LD_LIBRARY_PATH
(andCUDA_HOME
) environment variable to point to cuda 8.0 directory (i.e./usr/local/cuda-8.0
).conda install -c anaconda cudatoolkit=9.0 cudnn=7
solved it for me!This comment does not add anything to the issue. Also no one forces anyone to use TF, but If you really need to and can’t get pip packages working, why not to build it yourself? That’s what I ended up doing at some point. I saw there are also community supported builds here https://github.com/tensorflow/tensorflow just scroll down. Maybe build for your GPU and contribute?
what about f** I just only want to use tensorflow1.8 and cuda9.1?
the above worked for installing on Ubuntu Server 17. namely,
much easier than compiling.
Just to clarify a few things for anyone who might stumble on this post. I have in my system installed cuda-8.0, cuda-9.0, cuda-9.1. You don’t have to remove anything to make it work with tensorflow. Instead if you are missing cuda-9.0 from your system, as other have already pointed then you’ll need to install it that is a prerequisite for tensorflow to work properly.
If you have cuda-9.0 installed on you system and tensorflow is complaining about
libcublas.so.9.0
again as others have said expose that during runtime through yourLD_LIBRARY_PATH
environment variable in your.bashrc
make it point to/usr/local/cuda-9.0/lib64
. This should be working even for tensorflow 1.7.What I have tried and failed to accomplish is build from source. For some reason bazel always exits with an error. If you try to build with cuda-9.0/cuda-9.1 and cudnn7 it complains about gcc7. Using gcc5 compilation seems to be working fine but then at the end I always get an error and the build is unsuccessful.
My question is if anyone has managed to compile from source with cuda-9.1/cuda-9.0 without problems?
As @pascalwhoop mentioned, I followed the instructions in here http://www.python36.com/install-tensorflow141-gpu/ to build Tensorflow from source.
Whenever it said cuda 9.0 I changed to 9.1, and cudnn 7.0 I put 7.1.2.
Worked fine, so far!
@abrahamrhoffman that’s rude.
I just changed my batchrc from cuda-9.1 to just cuda. Then my tensorflow is able of finding the libcublas.so.9.0
This is an issue page. Please, state your issues so they can be fixed, or, even better, help fixing them.
I have installed Cuda 10 and the latest version of Tensor Flow, but I have received
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory
, I have made the downgrade and I still receiving this message…@Abduoit your Ubuntu version doesn’t really matter. The thing is, TensorFlow 1.6 expects CUDA to be version 9.0, and cuDNN to be version 7.0.4 (yes, the 0.4 does matter)
I tried symbolic links from all the 9.0 filenames to all the 9.1 filenames and it didn’t work. In the end, TF knows the true version. The repo doesn’t even have 9.0 anymore so I’m afraid I’ll break my nvidia stuff if I remove 9.1 and then manually install 9.0.
Good solution @dodler ! but I think you are missing the commands to make this great advice easy to follow through xD.
Happy coding!
This worked for me: Because Tensorflow > 1.4 requires CUDA 9, I uninstalled all CUDA versions :
Install excatly CUDA 9.0 and CuDNN 7.0.x and reboot.
I am trying this (which builds tensorflow manually)link
Might take a while longer but you can define the minor versions this way.
@dartdog
First I’ve installed tensorflow 1.5, it broke, and I get the following error:
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory
then I uninstalled, installed 1.4 with pip install --upgrade tensorflow-gpu==1.4, it did’t work, and I get the following error:ImportError: libcublas.so.8.0: cannot open shared object file: No such file or directory
Even tf-nightly-gpu is not looking for cuda 9.1. Meh
In fact, we should view the official document of tensorflow ,it give tensorflow‘s envirment(include python,gcc,cuda,cudnn,an so on).
Could you tell us the OS version and the exact command you issued to “downgrade”?
@dhruvhacks Good. I think people should install the right version of tensorflow-gpu with the right version cuda.
Thank u!
| | 杨康 | | 邮箱:18375320027@163.com |
签名由 网易邮箱大师 定制
On 08/18/2018 08:00, Dhruv Srivastava wrote:
Please refer combinations of CUDA and CuDNN.
This error happens majorly due to incorrect version combinations of Nvidia-driver, CUDA, CuDNN and Tensorflow-gpu
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
@Suananda Thanks, it works for me in a tensorflow conda environment. Suggest the official guys to modify the installation guide “https://www.tensorflow.org/install/install_linux#InstallingAnaconda”, step 4 of Anaconda installing, from “pip install --ignore-installed --upgrade tfBinaryURL” to “conda install -c anaconda tensorflow-gpu”
from https://gist.github.com/zhanwenchen/e520767a409325d9961072f666815bb8
Thanks @Suananda! It works like magic.
be careful conda users. i hit the same problem and was scratching my head for two days, until finally i discovered that local copy of libcudnn.so was used by conda, under: /miniconda3/lib/libcudnn.so which pointed to libcudnn.so.7 which pointed to libcudnn.so.7.0.5 i don’t remember who and how placed it there but pretty much it overloaded the system default libcudnn.so.7.1.2 !!!
once removed, everything works like a charm: tensorflow 1.7 or 1.8-nightly, cuda-9.1, cudnn-7.1.2 on ubuntu 16.04
I had teh same situation. I had cuda 9.1, and tensorflow would not find libraries for cuda 9.0.
I have installed cuda 9.0 with command: sudo apt-get install cuda-libraries-9-0
That solved my problem.
I had same issue, I think I solved by some changing, this combination works for me
Ubuntu 16, cuda 9.0, cudnn 7.0, python 3,5, tensorflow 1.6
If you install new cuda while you still have previous version please make sure to specify the path like this
@magick93 and all that turn up here!
LISTEN! Anything you need is downgrade your cuda 9.1 -> cuda 9.0. That’s it! Just do it (if you downloaded cuda 9.1 before that you can execute following command in your terminal):
sudo apt-get install cuda-9-0
and remove cuda 9.1 by rm -rf.Btw, don’t forget to change $PATH in your
~/.bashrc
(9.1 -> 9.0).Im facing the same issue, but I am trying to run tensorflow using nvidia-docker. I have cuda-9-0 installed on the host, but when I try to run my docker container I get
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory
@DanlanChen but then, I guess it is preffered to use the latest version. So, if you ever want to upgrade, you now know what to do! 😄
This is definitely supposed to be included in the tensorflow documentation, as said by @bwesen. It should also be included in the errors list. Is it possible for us do this in anyway? I think I am going to open a new issue, requesting to either add support for cuda 9.1, or mention that you need cuda 9.0 specifically in the docs, and also include this in the errors list
@mkaze Thank you very much
@xiezhongzhao Install Cuda 9.0 and you should be fine. Tensorflow 1.5 does not work with Cuda 9.1.