scipy: BUG: Missing symbol when used with reference LAPACK: cblas_cdotc_sub
I’ve been testing scipy against all the BLAS/LAPACK variants available in conda-forge for a while, and though there are small issues here and there, generally things work everywhere.
Since scipy 1.10.1 (or some other change in the last 3 months), there’s however now a kind of error I haven’t seen before - a missing symbol (`cblas_cdotc_sub), but only for one BLAS/LAPACK flavour, namely the “netlib” one from https://github.com/Reference-LAPACK/lapack/.
This happens for linux on all arches (x64, aarch64, ppc64le), but not on osx/win.
Since this is causing an error directly upon import (rather than a small handful of failed tests), this has a much bigger blast radius.
import: 'scipy.cluster'
Traceback (most recent call last):
[...]
from . import _iterative
ImportError: $PREFIX/lib/python3.10/site-packages/scipy/sparse/linalg/_isolve/_iterative.cpython-310-x86_64-linux-gnu.so: undefined symbol: cblas_cdotc_sub
Apparently this is encountered in the wild already: https://github.com/python-control/Slycot/pull/194
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 20 (15 by maintainers)
I can reproduce this in an environment with the following changes:
And then running
python dev.py build -C-Dblas=blas -C-Dlapack=lapack -C-Duse-g77-abi=true
(after fixing another regression).The issue is that Netlib BLAS has separate
libblas
andlibcblas
libraries, and we’re only linking againstlibblas
. We are using CBLAS explicitly - but only when building with-Duse-g77-abi=true
, that’s why this doesn’t show up in CI. So the combination of building against Netlib BLAS and using those g77 ABI wrappers is where things go wrong:A fix is on the way. Our CI situation for BLAS/LAPACK libraries is very bad, that’s why this snuck in. I’ll see if I can address that as well.
This
cdotc
usage will be gone end of this week when #18391. It doesn’t solve the issue but will eliminate its impact radiusReporting back here, the distutils based builds against netlibs on aarch/ppc were successful in https://github.com/conda-forge/scipy-feedstock/pull/224, so it appears to be some consequence of switching to meson.
It’s late for me so I won’t be able to report the result right away, but I’ve restarted a test against all blas variants in https://github.com/conda-forge/scipy-feedstock/pull/224 when built with distutils rather than meson (if nothing else, then to rule out that this might have something to do with it).
If the linux + netlib jobs pass the import tests and run the test suite, then the feedstock changes in https://github.com/conda-forge/scipy-feedstock/pull/231 are at fault (somehow).
It looks like 3 weeks ago our CI was running with 1.10.1 binaries from conda forge. So maybe this change which was 2 weeks ago has some connection: https://github.com/conda-forge/scipy-feedstock/pull/231