scipy: BUG: undefined references to libmvec functions in some compiled modules
Describe your issue.
On arch linux with
$ python --version
Python 3.10.1
$ gcc --version
gcc (GCC) 11.1.0
$ gfortran --version
GNU Fortran (GCC) 11.1.0
$ pacman -Q openblas
openblas 0.3.19-1
I’m unable to build a working version of v1.7.3 because at least the scipy/linalg/_interpolative.so
compiled module has undefined symbols. This appears to be because the various object files are compiled with enough optimisations that a mangled call to the vectorised GLIBC cosine function is emitted, but the link line that produces the module doesn’t link explicitly against libm.so
and is therefore relying on transitive link dependencies to pull in libmvec.so
(which on this system doesn’t happen).
The link line for _interpolative.so
in the output from the build script is:
INFO: gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.10/build/src.linux-x86_64-3.10/scipy/linalg/_interpolativemodule.o build/temp.linux-x86_64-3.10/build/src.linux-x86_64-3.10/build/src.linux-x86_64-3.10/scipy/linalg/fortranobject.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/dfft.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/id_rand.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/id_rtrans.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idd_frm.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idd_house.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idd_id.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idd_id2svd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idd_qrpiv.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idd_sfft.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idd_snorm.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idd_svd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/iddp_aid.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/iddp_asvd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/iddp_rid.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/iddp_rsvd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/iddr_aid.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/iddr_asvd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/iddr_rid.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/iddr_rsvd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idz_frm.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idz_house.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idz_id.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idz_id2svd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idz_qrpiv.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idz_sfft.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idz_snorm.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idz_svd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idzp_aid.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idzp_asvd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idzp_rid.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idzp_rsvd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idzr_aid.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idzr_asvd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idzr_rid.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/idzr_rsvd.o build/temp.linux-x86_64-3.10/scipy/linalg/src/id_dist/src/prini.o -L/usr/lib64 -L/usr/lib -Lbuild/temp.linux-x86_64-3.10 -lopenblas -llapack -lopenblas -llapack -o build/lib.linux-x86_64-3.10/scipy/linalg/_interpolative.cpython-310-x86_64-linux-gnu.so -Wl,--version-script=build/temp.linux-x86_64-3.10/link-version-scipy.linalg._interpolative.map
I think the right fix is to add -lm
to the link lines, but I have no idea how to do so.
Reproducing Code Example
# Get a build environment up [I can provide exact details if necessary]
# Then approximately this
git clone https://github.com/scipy/scipy.git
cd scipy
git checkout v1.7.3
git submodule update --init
python3 -m venv foo
. foo/bin/activate
pip install numpy Cython pythran pybind11
python3 setup.py build
[... wait]
cd build/lib.linux-x86_64-3.10/scipy/linalg
python3 -c "import _interpolative"
Error message
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: .../scipy/build/lib.linux-x86_64-3.10/scipy/linalg/_interpolative.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZGVbN2v_cos
SciPy/NumPy/Python version information
scipy: 1.7.3, numpy: 1.22.0, Python:3.10.1
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 39 (34 by maintainers)
Commits related to this issue
- BLD: meson: add `-lm` to add C extensions in a portable manner See https://github.com/scipy/scipy/issues/15414, where it turned out we were missing `-lm` in the distutils based build, and under some ... — committed to rgommers/scipy by rgommers 2 years ago
- BLD: meson: add `-lm` to add C extensions in a portable manner See https://github.com/scipy/scipy/issues/15414, where it turned out we were missing `-lm` in the distutils based build, and under some ... — committed to rgommers/scipy by rgommers 2 years ago
- BLD: meson: add `-lm` to add C extensions in a portable manner (#126) See https://github.com/scipy/scipy/issues/15414, where it turned out we were missing `-lm` in the distutils based build, and und... — committed to rgommers/scipy by rgommers 2 years ago
- BLD: meson: add `-lm` to add C extensions in a portable manner (#126) See https://github.com/scipy/scipy/issues/15414, where it turned out we were missing `-lm` in the distutils based build, and und... — committed to rgommers/scipy by rgommers 2 years ago
This is fixed in NumPy
main
now (by https://github.com/numpy/numpy/pull/20886) and that fix will land in NumPy1.22.x
. So I will close the issue. Thanks for the input and testing everyone!I’ll check it this morning; I was planning to continue bisecting so probably saves me quite some bit of time anyway.
Okay, I found the problem I think - PR for
numpy.distutils
should be ready later today.cat /etc/redhat-release
lscpu
For the record, this allows SciPy to import/work properly for my Linux workflow with
spack
, adjustingcompilers.yaml
ldflags
for the compiler I’m using with global application of-lmvec -lm
:That’s useful. Obviously better might be:
py-scipy
spack
packageAt least for me with gcc 9.4.0 toolchain and
spack
, the answer is “yes.” About to try the patch.