spack: [BUG] No RPATH in Python extensions
See the dialog below for details. Since Spack does not use hierarchical modules, working around this bug requires that one do spack load on an arbitrary and growing set of modules.
[me@ankeli spack2]$ spack purge
[me@ankeli spack2]$ spack load py-numpy
[me@ankeli spack2]$ python3 -c 'import numpy'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/me/spack2/opt/spack/linux-x86_64/gcc-4.9.3/py-numpy-1.11.0-mulodttw5pcyjufva4htsktwty4qd52r/lib/python3.5/site-packages/numpy-1.11.0-py3.5-linux-x86_64.egg/numpy/__init__.py", line 180, in <module>
from . import add_newdocs
File "/home/me/spack2/opt/spack/linux-x86_64/gcc-4.9.3/py-numpy-1.11.0-mulodttw5pcyjufva4htsktwty4qd52r/lib/python3.5/site-packages/numpy-1.11.0-py3.5-linux-x86_64.egg/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/home/me/spack2/opt/spack/linux-x86_64/gcc-4.9.3/py-numpy-1.11.0-mulodttw5pcyjufva4htsktwty4qd52r/lib/python3.5/site-packages/numpy-1.11.0-py3.5-linux-x86_64.egg/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/home/me/spack2/opt/spack/linux-x86_64/gcc-4.9.3/py-numpy-1.11.0-mulodttw5pcyjufva4htsktwty4qd52r/lib/python3.5/site-packages/numpy-1.11.0-py3.5-linux-x86_64.egg/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/home/me/spack2/opt/spack/linux-x86_64/gcc-4.9.3/py-numpy-1.11.0-mulodttw5pcyjufva4htsktwty4qd52r/lib/python3.5/site-packages/numpy-1.11.0-py3.5-linux-x86_64.egg/numpy/core/__init__.py", line 14, in <module>
from . import multiarray
ImportError: libopenblas.so.0: cannot open shared object file: No such file or directory
Uh-oh, what’s wrong… let’s debug this with ldd:
[me@ankeli spack2]$ ldd `spack location -i py-numpy`/lib/python3.5/site-packages/numpy-1.11.0-py3.5-linux-x86_64.egg/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so
linux-vdso.so.1 => (0x00007ffc4ef1f000)
libopenblas.so.0 => not found
libm.so.6 => /lib64/libm.so.6 (0x00007f6ccbd1d000)
libpython3.5m.so.1.0 => /home/me/spack2/opt/spack/linux-x86_64/gcc-4.9.3/python-3.5.1-5q5rsrtjld4u6jiicuvtnx52m7tfhegi/lib/libpython3.5m.so.1.0 (0x00007f6ccb7f7000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6ccb5db000)
libc.so.6 => /lib64/libc.so.6 (0x00007f6ccb219000)
/lib64/ld-linux-x86-64.so.2 (0x00007f6ccc440000)
libdl.so.2 => /usr/lib64/libdl.so.2 (0x00007f6ccb014000)
libutil.so.1 => /usr/lib64/libutil.so.1 (0x00007f6ccae11000)
Same problem, it cannot load libopenblas.so. So let’s tell Spack to load it explicitly and try again:
[me@ankeli spack2]$ spack load openblas
[me@ankeli spack2]$ ldd `spack location -i py-numpy`/lib/python3.5/site-packages/numpy-1.11.0-py3.5-linux-x86_64.egg/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so
linux-vdso.so.1 => (0x00007ffc78d2c000)
libopenblas.so.0 => /home/me/spack2/opt/spack/linux-x86_64/gcc-4.9.3/openblas-0.2.17-pw6rmlom7apfsnjtzfttyayzc7nx5e7y/lib/libopenblas.so.0 (0x00007fb61f48e000)
libm.so.6 => /lib64/libm.so.6 (0x00007fb61f175000)
libpython3.5m.so.1.0 => /home/me/spack2/opt/spack/linux-x86_64/gcc-4.9.3/python-3.5.1-5q5rsrtjld4u6jiicuvtnx52m7tfhegi/lib/libpython3.5m.so.1.0 (0x00007fb61ec4f000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb61ea32000)
libc.so.6 => /lib64/libc.so.6 (0x00007fb61e671000)
/lib64/ld-linux-x86-64.so.2 (0x00007fb620878000)
libgfortran.so.3 => /home/me/spack/opt/spack/linux-x86_64/gcc-4.8.5/gcc-4.9.3-layphctulnk3omsbjpzftqv6dlxpfe3d/lib64/libgfortran.so.3 (0x00007fb61e352000)
libdl.so.2 => /usr/lib64/libdl.so.2 (0x00007fb61e14d000)
libutil.so.1 => /usr/lib64/libutil.so.1 (0x00007fb61df4a000)
libquadmath.so.0 => /home/me/spack/opt/spack/linux-x86_64/gcc-4.8.5/gcc-4.9.3-layphctulnk3omsbjpzftqv6dlxpfe3d/lib64/../lib64/libquadmath.so.0 (0x00007fb61dd0c000)
libgcc_s.so.1 => /home/me/spack/opt/spack/linux-x86_64/gcc-4.8.5/gcc-4.9.3-layphctulnk3omsbjpzftqv6dlxpfe3d/lib64/../lib64/libgcc_s.so.1 (0x00007fb61daf6000)
OK, now it works with ldd. And try with Python:
[me@ankeli spack2]$ python3 -c 'import numpy'
Remember… LD_LIBRARY_PATH is EVIL!!!
About this issue
- Original URL
- State: closed
- Created 11 years ago
- Comments: 15 (15 by maintainers)
Commits related to this issue
- fixes #719 — committed to epfl-scitas/spack by nrichart 10 years ago
- Update BGLibPy version to 4.2.14 (#719) * updated bglibpy version * remove pinned BluePy version on BGLibPy * fixed style: [W293] blank line contains whitespace — committed to matz-e/spack by anilbey 7 years ago
@adamjstewart
I can replicate what you are seeing with spack compiled gcc-6.1.0 and gcc-5.3.0. Building with the system gcc does work though.