scipy: Segmentation fault on import of scipy.integrate on Apple M1 ARM silicon

SciPy can be installed under native Python 3.9.1 on an Apple M1 system running macOS Big Sur 11.1 using these instructions. However, importing scipy.integrate gives a segmentation fault.

Reproducing code example:

$ brew install python@3.9
$ brew install openblas
$ OPENBLAS="$(brew --prefix openblas)" MACOSX_DEPLOYMENT_TARGET=11.1 python3 -m pip install cython --no-use-pep517
$ OPENBLAS="$(brew --prefix openblas)" MACOSX_DEPLOYMENT_TARGET=11.1 python3 -m pip install numpy --no-use-pep517
$ OPENBLAS="$(brew --prefix openblas)" MACOSX_DEPLOYMENT_TARGET=11.1 python3 -m pip install pandas --no-use-pep517
$ OPENBLAS="$(brew --prefix openblas)" MACOSX_DEPLOYMENT_TARGET=11.1 python3 -m pip install pybind11 --no-use-pep517
$ OPENBLAS="$(brew --prefix openblas)" MACOSX_DEPLOYMENT_TARGET=11.1 python3 -m pip install scipy --no-use-pep517
$ python3
Python 3.9.1 (default, Dec 29 2020, 02:24:05) 
[Clang 12.0.0 (clang-1200.0.32.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import scipy.integrate

Error message:

I’m not sure how to install gdb on M1 – I’m not sure it’s possible yet. If there is another tool I can use to get a stack trace, please advise. Here is the detail on where in the import process the error occurs, via the last bit of verbose output achieved via python3 -vv:

...
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/quadpack.cpython-39-darwin.so
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/quadpack.abi3.so
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/quadpack.so
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/quadpack.py
# /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/__pycache__/quadpack.cpython-39.pyc matches /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/quadpack.py
# code object from '/opt/homebrew/lib/python3.9/site-packages/scipy/integrate/__pycache__/quadpack.cpython-39.pyc'
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/_quadpack.cpython-39-darwin.so
# extension module 'scipy.integrate._quadpack' loaded from '/opt/homebrew/lib/python3.9/site-packages/scipy/integrate/_quadpack.cpython-39-darwin.so'
# extension module 'scipy.integrate._quadpack' executed from '/opt/homebrew/lib/python3.9/site-packages/scipy/integrate/_quadpack.cpython-39-darwin.so'
import 'scipy.integrate._quadpack' # <_frozen_importlib_external.ExtensionFileLoader object at 0x116030880>
import 'scipy.integrate.quadpack' # <_frozen_importlib_external.SourceFileLoader object at 0x1151608e0>
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/_ode.cpython-39-darwin.so
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/_ode.abi3.so
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/_ode.so
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/_ode.py
# /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/__pycache__/_ode.cpython-39.pyc matches /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/_ode.py
# code object from '/opt/homebrew/lib/python3.9/site-packages/scipy/integrate/__pycache__/_ode.cpython-39.pyc'
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/vode.cpython-39-darwin.so
# extension module 'scipy.integrate.vode' loaded from '/opt/homebrew/lib/python3.9/site-packages/scipy/integrate/vode.cpython-39-darwin.so'
# extension module 'scipy.integrate.vode' executed from '/opt/homebrew/lib/python3.9/site-packages/scipy/integrate/vode.cpython-39-darwin.so'
import 'scipy.integrate.vode' # <_frozen_importlib_external.ExtensionFileLoader object at 0x116041580>
# trying /opt/homebrew/lib/python3.9/site-packages/scipy/integrate/_dop.cpython-39-darwin.so
zsh: segmentation fault  /opt/homebrew/bin/python3 -vv -c "import scipy.integrate"

Scipy/Numpy/Python version information:

>> import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
1.6.0 1.19.5 sys.version_info(major=3, minor=9, micro=1, releaselevel='final', serial=0)

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 2
Comments: 22 (16 by maintainers)

Links to this issue

Most upvoted comments

I believe these were the same changes as made in numpy-wheels: https://github.com/MacPython/scipy-wheels/pull/123

judahrand on Jun 28, 2021

There was one test in numpy that was marked as xfail which is the reason for the RuntimeWarnings here. https://github.com/numpy/numpy/issues/18555

isuruf on Jun 25, 2021

@hjmr Thanks

Pull request for homebrew here: https://github.com/Homebrew/homebrew-core/pull/70096.

Won’t fix the actual issue.

xght on Jan 31, 2021

Caused by https://github.com/iains/gcc-darwin-arm64/issues/32

Edit: proof

crash in _f2pyinittypes_, the assembly is:

    00000000000069a0 _f2pyinittypes_:
    69a0: 02 01 00 f0                   adrp    x2, #143360
    69a4: e1 03 00 aa                   mov     x1, x0
    69a8: 40 c0 28 91                   add     x0, x2, #2608
    69ac: 61 00 10 37                   tbnz    w1, #2, #12 <_f2pyinittypes_+0x18>
    69b0: f1 03 01 aa                   mov     x17, x1
    69b4: 20 02 1f d6                   br      x17
    69b8: 30 c0 5f f8                   ldur    x16, [x1, #-4]
    69bc: 21 40 40 f8                   ldur    x1, [x1, #4]
    69c0: fc ff ff 17                   b       #-16 <_f2pyinittypes_+0x10>

We can maybe find a workaround with a -falign-functions=8somewhere.

xght on Jan 29, 2021

Okay, we’ve now had enough report of successful builds that I think we can close this. The crash in the initial report seems to be due to using Homebrew compilers that had an issue. That issue could be worked around with export CFLAGS="-falign-functions=8 and has been fixed quite a while ago.

Thanks for the input everyone!

rgommers on Nov 14, 2021

@judahrand if you’d wanted to make a PR to the scipy-wheels repo, copying the approach in the numpy-wheels PRs that added support, that’d be super helpful.

rgommers on Jun 28, 2021

This successfully seems to build Scipy and run the tests without segfaults, however, there are 42 failing tests.

Thanks for the update @judahrand! All of these are RuntimeWarning’s coming from which the test suite turns into errors on the master branch. On a release branch that’s not done, so it looks like all tests would pass.

The warnings are incorrect it looks like, but not too surprising to see those on new hardware. They should be fixed in NumPy, if they are still happening with NumPy 1.21.0 and on master. If they’re fixed in master, identifying the fix and asking for it to be backported for 1.21.1 would be quite useful. If you’d want to look into that, that would be great.

It seems like the other thing we have to do here is incorporate -falign-functions=8 into the build setup, and then we should be able to have M1 wheels for SciPy 1.7.1.

rgommers on Jun 25, 2021

@xght Great!! I have compiled with a-falign-functions=8 option and got it working! The procedure is simply copy the source tree from GitHub and create the wheel with the option as:

 $ git clone https://github.com/scipy/scipy
 $ CFLAGS="-falign-functions=8 ${CFLAGS}" pip wheel ./scipy

I have not yet used it for serious tasks but, so far, it seems working well.

hjmr on Jan 31, 2021