tensorflow: ipython ImportError: Library not loaded: @rpath/libcudart.8.0.dylib

I just installed 0.11rc1 from pip (TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow-0.11.0rc1-py2-none-any.whl), with a brand-new CUDA 8.0 installation, and got this error:

In [1]: import tensorflow as tf
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-41389fad42b5> in <module>()
----> 1 import tensorflow as tf

/Users/me/anaconda/lib/python2.7/site-packages/tensorflow/__init__.py in <module>()
     21 from __future__ import print_function
     22
---> 23 from tensorflow.python import *
     24
     25

/Users/me/anaconda/lib/python2.7/site-packages/tensorflow/python/__init__.py in <module>()
     47 _default_dlopen_flags = sys.getdlopenflags()
     48 sys.setdlopenflags(_default_dlopen_flags | ctypes.RTLD_GLOBAL)
---> 49 from tensorflow.python import pywrap_tensorflow
     50 sys.setdlopenflags(_default_dlopen_flags)
     51

/Users/me/anaconda/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py in <module>()
     26                 fp.close()
     27             return _mod
---> 28     _pywrap_tensorflow = swig_import_helper()
     29     del swig_import_helper
     30 else:

/Users/me/anaconda/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py in swig_import_helper()
     22         if fp is not None:
     23             try:
---> 24                 _mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
     25             finally:
     26                 fp.close()

ImportError: dlopen(/Users/me/anaconda/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so, 10): Library not loaded: @rpath/libcudart.8.0.dylib
  Referenced from: /Users/me/anaconda/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so
  Reason: image not found

(Similar to https://github.com/tensorflow/tensorflow/issues/4187 but different error.) Looks like a path issue but I don’t see a problem:

➜  ~ echo $PATH
/Users/me/anaconda/bin:/usr/local/cuda/bin:/Developer/NVIDIA/CUDA-8.0/bin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
➜  ~ echo $DYLD_LIBRARY_PATH
/usr/local/cuda/lib:/Developer/NVIDIA/CUDA-8.0/lib:
➜  ~ ll /Developer/NVIDIA/CUDA-8.0/lib/libcudart*
-rwxr-xr-x 1 root wheel 285K Sep 27 00:59 /Developer/NVIDIA/CUDA-8.0/lib/libcudart.8.0.dylib
lrwxr-xr-x 1 root wheel   19 Sep 27 00:59 /Developer/NVIDIA/CUDA-8.0/lib/libcudart.dylib -> libcudart.8.0.dylib
-rw-r--r-- 1 root wheel 599K Sep 27 00:59 /Developer/NVIDIA/CUDA-8.0/lib/libcudart_static.a

I also get a segfault when importing in python:

➜  ~ python -c "import tensorflow;"
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.dylib locally
[1]    60723 segmentation fault  python -c "import tensorflow;"

Tried the suggestion from the docs, didn’t work:

ln -sf /usr/local/cuda/lib/libcuda.dylib /usr/local/cuda/lib/libcuda.1.dylib

On OSX 10.11.6, CUDA 8.0.46, no CUDNN installed yet.

Is this a SIP issue? Should the install procedure be updated? I don’t know if I can turn off SIP, this is a work computer.

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 1
Comments: 43 (13 by maintainers)

Most upvoted comments

The solution presented in https://github.com/tensorflow/tensorflow/issues/6729 resolved my issue. Namely, setting the following environment variables:

export CUDA_HOME=/usr/local/cuda
export DYLD_LIBRARY_PATH=${CUDA_HOME}/lib:${CUDA_HOME}/extras/CUPTI/lib:/Developer/NVIDIA/CUDA-8.0/lib
export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
export PATH=$DYLD_LIBRARY_PATH:$PATH

plus symlinking libcuda.1 (cd /usr/local/cuda/lib && sudo ln -s libcuda.dylib libcuda.1.dylib)
and ensuring that pyenv is not active (remove eval "$(pyenv init -)" from ~/.bash_profile if you have it).

This now yields:

Socrates:~ bRad$ python -c "import tensorflow as tf"
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.8.0.dylib locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.5.dylib locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.8.0.dylib locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.1.dylib locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.8.0.dylib locally```

+26

U007D on Feb 23, 2017

UPDATE: I have to update the path variables each time I activate/deactivate the tensorflow environment before launching jupyter notebook using:

export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/usr/local/cuda/lib export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/Developer/NVIDIA/CUDA-8.0/lib export PATH=$PATH:/Developer/NVIDIA/CUDA-8.0/lib

After doing this, tensorflow imports fine inside jupyter.

vaibhavravi on Feb 27, 2017

Exactly the same issue. libcudart.8.0.dylib is available on the DYLD_LIBRARY_PATH:

$ python -c "import tensorflow"
dyld: warning, LC_RPATH $ORIGIN/../../_solib_darwin/_U@local_Uconfig_Ucuda_S_Scuda_Ccudart___Uexternal_Slocal_Uconfig_    Ucuda_Scuda_Slib in /usr/local/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so being ignored in restricted program because it is a relative path
dyld: warning, LC_RPATH ../local_config_cuda/cuda/lib in /usr/local/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so being ignored in restricted program because it is a relative path
dyld: warning, LC_RPATH ../local_config_cuda/cuda/extras/CUPTI/lib in /usr/local/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so being ignored in restricted program because it is a relative path
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/tensorflow/__init__.py", line 23, in <module>
from tensorflow.python import *
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 28, in <module>
    _pywrap_tensorflow = swig_import_helper()
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
ImportError: dlopen(/usr/local/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so, 10): Library not loaded: @rpath/libcudart.8.0.dylib
  Referenced from: /usr/local/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so
  Reason: image not found


$ echo $DYLD_LIBRARY_PATH
:/usr/local/cuda/lib

$ ls -all /usr/local/cuda/lib/libcudart.8.0.dylib
lrwxr-xr-x@ 1 root  wheel  50 Sep 27 00:00 /usr/local/cuda/lib/libcudart.8.0.dylib -> /Developer/NVIDIA/CUDA-8.0/lib/libcudart.8.0.dylib

$ ls -all /Developer/NVIDIA/CUDA-8.0/lib/libcudart.8.0.dylib
-rwxr-xr-x@ 1 root  wheel  291712 Sep 26 23:59 /Developer/NVIDIA/CUDA-8.0/lib/libcudart.8.0.dylib

Any input is appreciated.

visionscaper on Nov 9, 2016

Add the following to your bash profile. This will provide Tensorflow the path needed to get to this dylib.

export DYLD_LIBRARY_PATH=“/Developer/NVIDIA/CUDA-8.0/lib:$DYLD_LIBRARY_PATH”

harpreet1010 on Nov 7, 2016

Apparently SIP prevents you from overriding DYLD_LIBRARY_PATH which would explain why the library is not found – https://groups.google.com/forum/#!topic/caffe-users/waugt62RQMU

yaroslavvb on Oct 22, 2016

@bradleygibson Worked! Thank you so much. I think setting the environment variables helped me because I had done everything else beforehand and it still was not working for me.

EDIT: See comment update below

vaibhavravi on Feb 27, 2017

FYI, it works with ipython after the applying the hack in issue #2910 (ln -s libcuda.dylib libcuda.1.dylib). How could this be?

visionscaper on Nov 9, 2016

Here’s a similar-looking crash that only happens in IPython and not in Python. The solution is to run python <my anaconda virtual environment path>/bin/ipython instead of ipython

PS, you can also get stack traces on MacOS from diagnostics reports folder, as shown here: https://github.com/tensorflow/tensorflow/issues/3494#issuecomment-239081963

“gdb ipython” doesn’t work because “ipython” is a shell script, an alternative way is to start ipython, then “gdb -p <process id>”, then do import tensorflow.

yaroslavvb on Oct 24, 2016