scipy: BUG: Library not loaded: @rpath/libgfortran.5.dylib for scipy v1.14.0rc1 x86 macos wheels

Describe your issue.

For SciPy release candidate v1.14.0rc1 the x86 macOS wheels will error out with

ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/sparse/linalg/_eigen/arpack/_arpack.cpython-312-darwin.so, 0x0002): Library not loaded: @rpath/libgfortran.5.dylib

on import of scipy.special.

image

Reproducing Code Example

Here is the workflow YAML for a full reproducer in GitHub Actions:

name: Reproducer

on:
  push:
    branches:
    - main
  workflow_dispatch:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

permissions:
  contents: read

jobs:
  example:

    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, macos-13, windows-latest]
        python-version: ['3.10', '3.11', '3.12']
      fail-fast: false

    steps:
    - uses: actions/checkout@v4

    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install --upgrade 'scipy==1.14.0rc1'
        python -m pip install --upgrade pyyaml  # given reporting information
        python -m pip list

    - name: Reporting information
      run: |
        python -c 'import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info); scipy.show_config()'

    - name: Reproducer
      run: |
        python -c 'from scipy import special'

but on an x86 macOS just

python -m pip install --upgrade 'scipy==1.14.0rc1'
python -c 'from scipy import special'

is enough.

Error message

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/__init__.py", line 134, in __getattr__
    return _importlib.import_module(f'scipy.{name}')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/special/__init__.py", line 820, in <module>
    from . import _orthogonal
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/special/_orthogonal.py", line 81, in <module>
    from scipy import linalg
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/__init__.py", line 134, in __getattr__
    return _importlib.import_module(f'scipy.{name}')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/linalg/__init__.py", line 222, in <module>
    from ._sketches import *
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/linalg/_sketches.py", line 9, in <module>
    from scipy.sparse import csc_matrix
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/sparse/__init__.py", line 307, in <module>
    from . import csgraph
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/sparse/csgraph/__init__.py", line 187, in <module>
    from ._laplacian import laplacian
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/sparse/csgraph/_laplacian.py", line 7, in <module>
    from scipy.sparse.linalg import LinearOperator
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/sparse/linalg/__init__.py", line 132, in <module>
    from ._eigen import *
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/sparse/linalg/_eigen/__init__.py", line 9, in <module>
    from .arpack import *
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/sparse/linalg/_eigen/arpack/__init__.py", line 20, in <module>
    from .arpack import *
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/sparse/linalg/_eigen/arpack/arpack.py", line 48, in <module>
    from . import _arpack
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/sparse/linalg/_eigen/arpack/_arpack.cpython-312-darwin.so, 0x0002): Library not loaded: @rpath/libgfortran.5.dylib
  Referenced from: <7C2A9D44-8556-309A-BF2C-0E48CBA31E6C> /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/scipy/sparse/linalg/_eigen/arpack/_arpack.cpython-312-darwin.so
  Reason: tried: '/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib/libgfortran.5.dylib' (no such file), '/usr/local/lib/libgfortran.5.dylib' (no such file), '/usr/lib/libgfortran.5.dylib' (no such file, not in dyld cache)

SciPy/NumPy/Python version and system information

1.14.0rc1 1.26.4 sys.version_info(major=3, minor=12, micro=3, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/include
    lib directory: /private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib
    name: scipy-openblas
    openblas configuration: OpenBLAS 0.3.27.dev DYNAMIC_ARCH NO_AFFINITY Sandybridge
      MAX_THREADS=64
    pc file directory: /Users/runner/work/scipy/scipy
    version: 0.3.27.dev
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/include
    lib directory: /private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/lib/python3.12/site-packages/scipy_openblas32/lib
    name: scipy-openblas
    openblas configuration: OpenBLAS 0.3.27.dev DYNAMIC_ARCH NO_AFFINITY Sandybridge
      MAX_THREADS=64
    pc file directory: /Users/runner/work/scipy/scipy
    version: 0.3.27.dev
  pybind11:
    detection method: config-tool
    include directory: unknown
    name: pybind11
    version: 2.12.0
Compilers:
  c:
    commands: cc
    linker: ld64
    name: clang
    version: 13.1.6
  c++:
    commands: c++
    linker: ld64
    name: clang
    version: 13.1.6
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.0.10
  fortran:
    commands: gfortran
    linker: ld64
    name: gcc
    version: 11.3.0
  pythran:
    include directory: ../../../../../../private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/pip-build-env-usrnp1ch/overlay/lib/python3.12/site-packages/pythran
    version: 0.16.1
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: darwin
  cross-compiled: false
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: darwin
Python Information:
  path: /private/var/folders/n8/w901w1rn4ldg6yskstndsj200000gn/T/cibw-run-dk691tbh/cp312-macosx_x86_64/build/venv/bin/python
  version: '3.12'

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 21 (20 by maintainers)

Commits related to this issue

Most upvoted comments

I have yanked the release to avoid more users running into broken wheels:

image

Sorry for the mess.

It’s just the cost of pushing back the frontiers, don’t worry about it.

In https://github.com/andyfaff/scipy/pull/69 I amend the wheel building process to show that removing the libgfortran dylibs that are used for linking (rm -rf /usr/local/gfortran/lib) makes the post-build tests fail, see https://github.com/andyfaff/scipy/actions/runs/9342115595/job/25709721944?pr=69 for what happens.

  E   ImportError: dlopen(/private/var/folders/lr/439_fwvd3m76p9vy50d57kcc0000gn/T/cibw-run-bb9i5_xw/cp312-macosx_arm64/venv-test-arm64/lib/python3.12/site-packages/scipy/sparse/linalg/_eigen/arpack/_arpack.cpython-312-darwin.so, 0x0002): Library not loaded: @rpath/libgfortran.5.dylib

i.e. the install name for libgfortran is not correct for any fortran SO.