scipy: BUG: segfault in `scipy.linalg.lu` on x86_64 windows and macos with scipy 1.11.0

Describe your issue.

Discovered in last night’s scikit-learn build against scipy 1.11.0 released yesterday:

• https://github.com/scikit-learn/scikit-learn/issues/26698#issuecomment-1607107902

Reproducing Code Example

I don’t have x86_64 windows or macos machine handy by I suspect that the following should make it possible to reproduce the problem based on our CI logs:

pip install --upgrade scipy scikit-learn pytest
pytest -v -k test_randomized_eigsh_compared_to_others --pyargs sklearn.utils.tests.test_extmath

Note that I cannot reproduce it on macos M1 nor on any linux system.

Error message

Fatal Python error: Segmentation fault
  
  Thread 0x0000700003708000 (most recent call first):
    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 320 in wait
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/joblib/externals/loky/backend/queues.py", line 113 in _feed
    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 975 in run
    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 995 in _bootstrap
  
  Thread 0x0000700002705000 (most recent call first):
    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/selectors.py", line 415 in select
    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 930 in wait
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 611 in wait_result_broken_or_wakeup
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 557 in run
    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 995 in _bootstrap
  
  Current thread 0x000000010e5ff600 (most recent call first):
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/scipy/linalg/_decomp_lu.py", line 313 in lu
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/sklearn/utils/extmath.py", line 278 in randomized_range_finder
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/sklearn/utils/extmath.py", line 450 in randomized_svd
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/sklearn/utils/extmath.py", line 608 in _randomized_eigsh
     File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/sklearn/utils/tests/test_extmath.py", line 226 in test_randomized_eigsh_compared_to_others
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_callers.py", line 80 in _multicall
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_manager.py", line 112 in _hookexec
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_hooks.py", line 433 in __call__
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/python.py", line 1788 in runtest
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_callers.py", line 80 in _multicall
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_manager.py", line 112 in _hookexec
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_hooks.py", line 433 in __call__
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/runner.py", line 262 in <lambda>
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/runner.py", line 341 in from_call
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/runner.py", line 261 in call_runtest_hook
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/runner.py", line 222 in call_and_report
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/runner.py", line 133 in runtestprotocol
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_callers.py", line 80 in _multicall
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_manager.py", line 112 in _hookexec
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_hooks.py", line 433 in __call__
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/main.py", line 349 in pytest_runtestloop
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_callers.py", line 80 in _multicall
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_manager.py", line 112 in _hookexec
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_hooks.py", line 433 in __call__
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/main.py", line 324 in _main
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/main.py", line 270 in wrap_session
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/main.py", line 317 in pytest_cmdline_main
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_callers.py", line 80 in _multicall
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_manager.py", line 112 in _hookexec
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/pluggy/_hooks.py", line 433 in __call__
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/config/__init__.py", line 166 in main
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/_pytest/config/__init__.py", line 189 in console_main
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/bin/pytest", line 8 in <module>

SciPy/NumPy/Python version and system information

System:
      python: 3.11.4 (v3.11.4:d2340ef257, Jun  6 2023, 19:15:51) [Clang 13.0.0 (clang-1300.0.29.30)]
  executable: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/bin/python
     machine: macOS-12.6.6-x86_64-i386-64bit
  
  Python dependencies:
        sklearn: 1.4.dev0
            pip: 23.1.2
     setuptools: 67.7.2
          numpy: 1.25.0
          scipy: 1.11.0
         Cython: None
         pandas: 2.0.2
     matplotlib: None
         joblib: 1.2.0
  threadpoolctl: 3.1.0
  
  Built with OpenMP: True
  
  threadpoolctl info:
         user_api: openmp
     internal_api: openmp
           prefix: libomp
         filepath: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/sklearn/.dylibs/libomp.dylib
          version: None
      num_threads: 3
  
         user_api: blas
     internal_api: openblas
           prefix: libopenblas
         filepath: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
          version: 0.3.23
  threading_layer: pthreads
     architecture: Sandybridge
      num_threads: 3
  
         user_api: blas
     internal_api: openblas
           prefix: libopenblas
         filepath: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-0tc4opqq/cp311-macosx_x86_64/venv-test/lib/python3.11/site-packages/scipy/.dylibs/libopenblas.0.dylib
          version: 0.3.21.dev
  threading_layer: pthreads
     architecture: Sandybridge
      num_threads: 3