scipy: Python 3.8 not supported

Python 3.8.0 released on 14th October, but Scipy does not support it. I tried updating pyproject.toml to the following:

[build-system]
requires = [
    "wheel",
    "setuptools",
    "Cython>=0.29.2",
    "numpy==1.13.3; python_version=='3.5' and platform_system!='AIX'",
    "numpy==1.13.3; python_version=='3.6' and platform_system!='AIX'",
    "numpy==1.14.5; python_version>='3.7' and platform_system!='AIX'",
    "numpy==1.16.0; python_version=='3.5' and platform_system=='AIX'",
    "numpy==1.16.0; python_version=='3.6' and platform_system=='AIX'",
    "numpy==1.16.0; python_version>='3.7' and platform_system=='AIX'",
    "numpy==1.17.2; python_version>='3.8' and platform_system=='AIX'",
    "pybind11>=2.2.4",
]

(My commit is here.)

I then installed scipy from source and ran python3.8 -c "import scipy; scipy.test(). The tests freeze at 79%.

.local/lib/python3.8/site-packages/scipy/sparse/tests/test_construct.py ........................                               [ 78%]
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_csc.py ..                                                           [ 78%]
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_csr.py ...                                                          [ 78%]
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_extract.py ..                                                       [ 78%]
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_matrix_io.py ......                                                 [ 79%]
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_sparsetools.py ...

After using Ctrl+C to quit that test, it continues to 100% with the following final results:

========================================================== warnings summary ==========================================================
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
  /home/sam/.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    warnings.warn(

.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
  /home/sam/.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321: PytestUnknownMarkWarning: Unknown pytest.mark.xslow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    warnings.warn(

.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_explicit
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_explicit
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_implicit
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_implicit
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_multi
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_multi
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_pearson
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_pearson
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_pearson
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_pearson
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_lorentz
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_lorentz
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ticket_1253
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ticket_1253
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ifixx
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ifixx
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ifixx
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ifixx
  /home/sam/.local/lib/python3.8/site-packages/scipy/odr/odrpack.py:1103: DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
    self.output = Output(odr(*args, **kwds))

-- Docs: https://docs.pytest.org/en/latest/warnings.html
================ 14520 passed, 1280 skipped, 1225 deselected, 77 xfailed, 8 xpassed, 27 warnings in 614.71s (0:10:14) ================

   Normal return from subroutine COBYLA

   NFVALS =   50   F = 2.485185E+01    MAXCV = 1.999965E-10
   X = 4.955358E+00   6.666553E-01

 NNLS quitting on iteration count.

Scipy/Numpy/Python version information:

Note: scipy was installed from source after modifying pyproject.toml (see code). 1.3.1 1.17.2 sys.version_info(major=3, minor=8, micro=0, releaselevel='final', serial=0)

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 33 (22 by maintainers)

Most upvoted comments

Have some patience please we are already trying to figure out what changed on python 3.8 that we have to fix. And the old ones should keep working with 3.7 so it is not that trivial.

They’ll be added for SciPy 1.3.2, it’s on the way.

NumPy 1.17.3 is out so I’ll try amending #10930 to run 3.8

ERROR: Command errored out with exit status 1: command: ‘c:\users\fablab\appdata\local\programs\python\python38-32\python.exe’ ‘c:\users\fablab\appdata\local\programs\python\python38-32\lib\site-packages\pip’ install --ignore-installed --no-user --prefix ‘C:\Users\FabLab\AppData\Local\Temp\pip-build-env-grh495uw\overlay’ --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple – wheel setuptools ‘Cython>=0.29.2’ ‘numpy==1.13.3; python_version==’“'”‘3.5’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.13.3; python_version==’“'”‘3.6’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.14.5; python_version>=’“'”‘3.7’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version==’“'”‘3.5’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version==’“'”‘3.6’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version>=’“'”‘3.7’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ cwd: None Complete output (344 lines): Ignoring numpy: markers ‘python_version == “3.5” and platform_system != “AIX”’ don’t match your environment Ignoring numpy: markers ‘python_version == “3.6” and platform_system != “AIX”’ don’t match your environment Ignoring numpy: markers ‘python_version == “3.5” and platform_system == “AIX”’ don’t match your environment Ignoring numpy: markers ‘python_version == “3.6” and platform_system == “AIX”’ don’t match your environment Ignoring numpy: markers ‘python_version >= “3.7” and platform_system == “AIX”’ don’t match your environment Collecting wheel Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl Collecting setuptools Using cached https://files.pythonhosted.org/packages/d9/de/554b6310ac87c5b921bc45634b07b11394fe63bc4cb5176f5240addf18ab/setuptools-41.6.0-py2.py3-none-any.whl Collecting Cython>=0.29.2 Using cached https://files.pythonhosted.org/packages/22/03/510503cfbf20f62810a9548c9be13ab86181f00cca9a3a56717c4595d952/Cython-0.29.14-cp38-cp38-win32.whl Collecting numpy==1.14.5 Using cached https://files.pythonhosted.org/packages/d5/6e/f00492653d0fdf6497a181a1c1d46bbea5a2383e7faf4c8ca6d6f3d2581d/numpy-1.14.5.zip Installing collected packages: wheel, setuptools, Cython, numpy Running setup.py install for numpy: started Running setup.py install for numpy: finished with status ‘error’ ERROR: Command errored out with exit status 1: command: ‘c:\users\fablab\appdata\local\programs\python\python38-32\python.exe’ -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’“‘C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\setup.py’”’“‘; file=’”‘“‘C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\setup.py’”’“';f=getattr(tokenize, '”‘“‘open’”’“‘, open)(file);code=f.read().replace(’”‘"’\r\n’“'”‘, ‘"’"’\n’“'”‘);f.close();exec(compile(code, file, ‘"’“‘exec’”’"‘))’ install --record ‘C:\Users\FabLab\AppData\Local\Temp\pip-record-v6v16vfn\install-record.txt’ --single-version-externally-managed --prefix ‘C:\Users\FabLab\AppData\Local\Temp\pip-build-env-grh495uw\overlay’ --compile cwd: C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy
Complete output (322 lines): Running from numpy source directory.

  Note: if you need reliable uninstall behavior, then install
  with pip instead of using `setup.py install`:

    - `pip install .`       (from a git repo or downloaded source
                             release)
    - `pip install numpy`   (last NumPy release on PyPi)


  C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\misc_util.py:464: SyntaxWarning: "is" with a literal. Did you mean "=="?
    return is_string(s) and ('*' in s or '?' is s)
  blas_opt_info:
  blas_mkl_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries mkl_rt not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  blis_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries blis not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  openblas_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries openblas not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
  get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
  customize GnuFCompiler
  Could not locate executable g77
  Could not locate executable f77
  customize IntelVisualFCompiler
  Could not locate executable ifort
  Could not locate executable ifl
  customize AbsoftFCompiler
  Could not locate executable f90
  customize CompaqVisualFCompiler
  Could not locate executable DF
  customize IntelItaniumVisualFCompiler
  Could not locate executable efl
  customize Gnu95FCompiler
  Could not locate executable gfortran
  Could not locate executable f95
  customize G95FCompiler
  Could not locate executable g95
  customize IntelEM64VisualFCompiler
  customize IntelEM64TFCompiler
  Could not locate executable efort
  Could not locate executable efc
  customize PGroupFlangCompiler
  Could not locate executable flang
  don't know how to compile Fortran code on platform 'nt'
    NOT AVAILABLE

  atlas_3_10_blas_threads_info:
  Setting PTATLAS=ATLAS
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries tatlas not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  atlas_3_10_blas_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries satlas not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  atlas_blas_threads_info:
  Setting PTATLAS=ATLAS
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries ptf77blas,ptcblas,atlas not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  atlas_blas_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries f77blas,cblas,atlas not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\system_info.py:624: UserWarning:
      Atlas (http://math-atlas.sourceforge.net/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [atlas]) or by setting
      the ATLAS environment variable.
    self.calc_info()
  blas_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries blas not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\system_info.py:624: UserWarning:
      Blas (http://www.netlib.org/blas/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [blas]) or by setting
      the BLAS environment variable.
    self.calc_info()
  blas_src_info:
    NOT AVAILABLE

  C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\system_info.py:624: UserWarning:
      Blas (http://www.netlib.org/blas/) sources not found.
      Directories to search for the sources can be specified in the
      numpy/distutils/site.cfg file (section [blas_src]) or by setting
      the BLAS_SRC environment variable.
    self.calc_info()
    NOT AVAILABLE

  'svnversion' is not recognized as an internal or external command,
  operable program or batch file.
  'svnversion' is not recognized as an internal or external command,
  operable program or batch file.
  non-existing path in 'numpy\\distutils': 'site.cfg'
  F2PY Version 2
  lapack_opt_info:
  lapack_mkl_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries mkl_rt not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  openblas_lapack_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries openblas not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  openblas_clapack_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries openblas,lapack not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  atlas_3_10_threads_info:
  Setting PTATLAS=ATLAS
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries tatlas,tatlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\lib
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\lib
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries tatlas,tatlas not found in C:\
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in C:\
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries tatlas,tatlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\libs
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\libs
  <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
    NOT AVAILABLE

  atlas_3_10_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries satlas,satlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\lib
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\lib
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries satlas,satlas not found in C:\
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in C:\
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries satlas,satlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\libs
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\libs
  <class 'numpy.distutils.system_info.atlas_3_10_info'>
    NOT AVAILABLE

  atlas_threads_info:
  Setting PTATLAS=ATLAS
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries ptf77blas,ptcblas,atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\lib
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\lib
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries ptf77blas,ptcblas,atlas not found in C:\
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in C:\
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries ptf77blas,ptcblas,atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\libs
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\libs
  <class 'numpy.distutils.system_info.atlas_threads_info'>
    NOT AVAILABLE

  atlas_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries f77blas,cblas,atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\lib
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\lib
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries f77blas,cblas,atlas not found in C:\
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in C:\
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries f77blas,cblas,atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\libs
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack_atlas not found in c:\users\fablab\appdata\local\programs\python\python38-32\libs
  <class 'numpy.distutils.system_info.atlas_info'>
    NOT AVAILABLE

  lapack_info:
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  customize MSVCCompiler
    libraries lapack not found in ['c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\lib', 'C:\\', 'c:\\users\\fablab\\appdata\\local\\programs\\python\\python38-32\\libs']
    NOT AVAILABLE

  C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\system_info.py:624: UserWarning:
      Lapack (http://www.netlib.org/lapack/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [lapack]) or by setting
      the LAPACK environment variable.
    self.calc_info()
  lapack_src_info:
    NOT AVAILABLE

  C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\system_info.py:624: UserWarning:
      Lapack (http://www.netlib.org/lapack/) sources not found.
      Directories to search for the sources can be specified in the
      numpy/distutils/site.cfg file (section [lapack_src]) or by setting
      the LAPACK_SRC environment variable.
    self.calc_info()
    NOT AVAILABLE

  c:\users\fablab\appdata\local\programs\python\python38-32\lib\distutils\dist.py:274: UserWarning: Unknown distribution option: 'define_macros'
    warnings.warn(msg)
  running install
  running build
  running config_cc
  unifing config_cc, config, build_clib, build_ext, build commands --compiler options
  running config_fc
  unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
  running build_src
  build_src
  building py_modules sources
  creating build
  creating build\src.win32-3.8
  creating build\src.win32-3.8\numpy
  creating build\src.win32-3.8\numpy\distutils
  building library "npymath" sources
  No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\bin\HostX86\x86\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Inumpy\core\src\private -Inumpy\core\src -Inumpy\core -Inumpy\core\src\npymath -Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\src\npysort -Ic:\users\fablab\appdata\local\programs\python\python38-32\include -Ic:\users\fablab\appdata\local\programs\python\python38-32\include -I"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\include" /Tc_configtest.c /Fo_configtest.obj
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\bin\HostX86\x86\link.exe /nologo /INCREMENTAL:NO /LTCG /MANIFEST:EMBED,ID=1 /LIBPATH:"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\lib\x86" _configtest.obj /OUT:_configtest.exe
  LINK : fatal error LNK1104: cannot open file 'kernel32.lib'
  failure.
  removing: _configtest.c _configtest.obj
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\setup.py", line 394, in <module>
      setup_package()
    File "C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\setup.py", line 386, in setup_package
      setup(**metadata)
    File "C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\core.py", line 169, in setup
      return old_setup(**new_attr)
    File "c:\users\fablab\appdata\local\programs\python\python38-32\lib\site-packages\setuptools\__init__.py", line 145, in setup
      return distutils.core.setup(**attrs)
    File "c:\users\fablab\appdata\local\programs\python\python38-32\lib\distutils\core.py", line 148, in setup
      dist.run_commands()
    File "c:\users\fablab\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "c:\users\fablab\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 985, in run_command
      cmd_obj.run()
    File "C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\command\install.py", line 62, in run
      r = self.setuptools_run()
    File "C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\command\install.py", line 36, in setuptools_run
      return distutils_install.run(self)
    File "c:\users\fablab\appdata\local\programs\python\python38-32\lib\distutils\command\install.py", line 545, in run
      self.run_command('build')
    File "c:\users\fablab\appdata\local\programs\python\python38-32\lib\distutils\cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "c:\users\fablab\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 985, in run_command
      cmd_obj.run()
    File "C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\command\build.py", line 47, in run
      old_build.run(self)
    File "c:\users\fablab\appdata\local\programs\python\python38-32\lib\distutils\command\build.py", line 135, in run
      self.run_command(cmd_name)
    File "c:\users\fablab\appdata\local\programs\python\python38-32\lib\distutils\cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "c:\users\fablab\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 985, in run_command
      cmd_obj.run()
    File "C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\command\build_src.py", line 148, in run
      self.build_sources()
    File "C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\command\build_src.py", line 159, in build_sources
      self.build_library_sources(*libname_info)
    File "C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\command\build_src.py", line 294, in build_library_sources
      sources = self.generate_sources(sources, (lib_name, build_info))
    File "C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\numpy\distutils\command\build_src.py", line 377, in generate_sources
      source = func(extension, build_dir)
    File "numpy\core\setup.py", line 675, in get_mathlib_info
      raise RuntimeError("Broken toolchain: cannot link a simple C program")
  RuntimeError: Broken toolchain: cannot link a simple C program
  ----------------------------------------

ERROR: Command errored out with exit status 1: ‘c:\users\fablab\appdata\local\programs\python\python38-32\python.exe’ -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’“‘C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\setup.py’”’“‘; file=’”‘“‘C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\setup.py’”’“';f=getattr(tokenize, '”‘“‘open’”’“‘, open)(file);code=f.read().replace(’”‘"’\r\n’“'”‘, ‘"’"’\n’“'”‘);f.close();exec(compile(code, file, ‘"’“‘exec’”’"‘))’ install --record ‘C:\Users\FabLab\AppData\Local\Temp\pip-record-v6v16vfn\install-record.txt’ --single-version-externally-managed --prefix ‘C:\Users\FabLab\AppData\Local\Temp\pip-build-env-grh495uw\overlay’ --compile Check the logs for full command output.

ERROR: Command errored out with exit status 1: ‘c:\users\fablab\appdata\local\programs\python\python38-32\python.exe’ ‘c:\users\fablab\appdata\local\programs\python\python38-32\lib\site-packages\pip’ install --ignore-installed --no-user --prefix ‘C:\Users\FabLab\AppData\Local\Temp\pip-build-env-grh495uw\overlay’ --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple – wheel setuptools ‘Cython>=0.29.2’ ‘numpy==1.13.3; python_version==’“'”‘3.5’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.13.3; python_version==’“'”‘3.6’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.14.5; python_version>=’“'”‘3.7’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version==’“'”‘3.5’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version==’“'”‘3.6’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version>=’“'”‘3.7’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ Check the logs for full command output.

We’ll need NumPy 1.17.3 first; 1.17.2 doesn’t support Python 3.8.

After building NumPy and SciPy master locally on 3.8 I see the same behavior (plus what’s fixed by #10930) – with scipy.test('full') it gets stuck at 81% and ctrl-C allows it to finish. Running:

scipy.test('full', extra_argv=['-ra', '--full-trace', '-vv'])

and then ctrl-Cing I can see that the problem happens here:

python/scipy/scipy/sparse/tests/test_sparsetools.py::test_nnz_overflow ✓                                                                                             81% ████████▏ 
^C
python/scipy/scipy/sparse/tests/test_sparsetools.py::TestInt32Overflow.test_coo_todense ✓                                                                            81% ████████▏ 

But running pytest -ra -vv scipy/sparse/tests/ directly, I don’t get this problem. [EDIT: Removed some wrong/useless information about memory limitations.] However, thanks to snakeviz I was able to do

$ python -m cProfile -o ~/Desktop/prof.prof ~/Desktop/sci.py  # sci.py just does "import scipy; scipy.test('full')"
$ snakeviz ~/Desktop/prof.prof

And see that the problematic stack was (probably!):

test_sparsetools.py:112(test_coo_todense) 1.07e+3 s
~:0(<built-in method gc.collect>) 1.07e+3 s
util.py:170(__call__) 1.07e+3 s
pool.py:679(_terminate_pool) 1.07e+3 s
pool.py:670(_help_stuff_finish) 1.07e+3 s
~:0(<method 'acquire' of '_multiprocessing.SemLock' objects>) 1.07e+3 s

So the problem is actually with gc.collect making multiprocessing pool wait.

In #10932 I turn this into a pytest fixture, and on my 3.8 distribution it stops here:

python/scipy/scipy/optimize/tests/test__differential_evolution.py::TestDifferentialEvolutionSolver.test_parallel ✓

@andyfaff this thus looks like a probable _differential_evolution pool problem. Any ideas?

Has this been fixed? I’m still getting this error with Python 3.8.6 with scipy 1.5

How about wheels for 3.8?