scipy: Python 3.8 not supported
Python 3.8.0 released on 14th October, but Scipy does not support it. I tried updating pyproject.toml
to the following:
[build-system]
requires = [
"wheel",
"setuptools",
"Cython>=0.29.2",
"numpy==1.13.3; python_version=='3.5' and platform_system!='AIX'",
"numpy==1.13.3; python_version=='3.6' and platform_system!='AIX'",
"numpy==1.14.5; python_version>='3.7' and platform_system!='AIX'",
"numpy==1.16.0; python_version=='3.5' and platform_system=='AIX'",
"numpy==1.16.0; python_version=='3.6' and platform_system=='AIX'",
"numpy==1.16.0; python_version>='3.7' and platform_system=='AIX'",
"numpy==1.17.2; python_version>='3.8' and platform_system=='AIX'",
"pybind11>=2.2.4",
]
(My commit is here.)
I then installed scipy from source and ran python3.8 -c "import scipy; scipy.test()
. The tests freeze at 79%.
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_construct.py ........................ [ 78%]
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_csc.py .. [ 78%]
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_csr.py ... [ 78%]
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_extract.py .. [ 78%]
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_matrix_io.py ...... [ 79%]
.local/lib/python3.8/site-packages/scipy/sparse/tests/test_sparsetools.py ...
After using Ctrl
+C
to quit that test, it continues to 100% with the following final results:
========================================================== warnings summary ==========================================================
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
/home/sam/.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
warnings.warn(
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321
/home/sam/.local/lib/python3.8/site-packages/_pytest/mark/structures.py:321: PytestUnknownMarkWarning: Unknown pytest.mark.xslow - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
warnings.warn(
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_explicit
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_explicit
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_implicit
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_implicit
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_multi
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_multi
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_pearson
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_pearson
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_pearson
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_pearson
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_lorentz
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_lorentz
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ticket_1253
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ticket_1253
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ifixx
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ifixx
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ifixx
.local/lib/python3.8/site-packages/scipy/odr/tests/test_odr.py::TestODR::test_ifixx
/home/sam/.local/lib/python3.8/site-packages/scipy/odr/odrpack.py:1103: DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
self.output = Output(odr(*args, **kwds))
-- Docs: https://docs.pytest.org/en/latest/warnings.html
================ 14520 passed, 1280 skipped, 1225 deselected, 77 xfailed, 8 xpassed, 27 warnings in 614.71s (0:10:14) ================
Normal return from subroutine COBYLA
NFVALS = 50 F = 2.485185E+01 MAXCV = 1.999965E-10
X = 4.955358E+00 6.666553E-01
NNLS quitting on iteration count.
Scipy/Numpy/Python version information:
Note: scipy was installed from source after modifying pyproject.toml
(see code).
1.3.1 1.17.2 sys.version_info(major=3, minor=8, micro=0, releaselevel='final', serial=0)
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 33 (22 by maintainers)
Have some patience please we are already trying to figure out what changed on python 3.8 that we have to fix. And the old ones should keep working with 3.7 so it is not that trivial.
They’ll be added for SciPy 1.3.2, it’s on the way.
NumPy 1.17.3 is out so I’ll try amending #10930 to run 3.8
ERROR: Command errored out with exit status 1: command: ‘c:\users\fablab\appdata\local\programs\python\python38-32\python.exe’ ‘c:\users\fablab\appdata\local\programs\python\python38-32\lib\site-packages\pip’ install --ignore-installed --no-user --prefix ‘C:\Users\FabLab\AppData\Local\Temp\pip-build-env-grh495uw\overlay’ --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple – wheel setuptools ‘Cython>=0.29.2’ ‘numpy==1.13.3; python_version==’“'”‘3.5’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.13.3; python_version==’“'”‘3.6’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.14.5; python_version>=’“'”‘3.7’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version==’“'”‘3.5’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version==’“'”‘3.6’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version>=’“'”‘3.7’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ cwd: None Complete output (344 lines): Ignoring numpy: markers ‘python_version == “3.5” and platform_system != “AIX”’ don’t match your environment Ignoring numpy: markers ‘python_version == “3.6” and platform_system != “AIX”’ don’t match your environment Ignoring numpy: markers ‘python_version == “3.5” and platform_system == “AIX”’ don’t match your environment Ignoring numpy: markers ‘python_version == “3.6” and platform_system == “AIX”’ don’t match your environment Ignoring numpy: markers ‘python_version >= “3.7” and platform_system == “AIX”’ don’t match your environment Collecting wheel Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl Collecting setuptools Using cached https://files.pythonhosted.org/packages/d9/de/554b6310ac87c5b921bc45634b07b11394fe63bc4cb5176f5240addf18ab/setuptools-41.6.0-py2.py3-none-any.whl Collecting Cython>=0.29.2 Using cached https://files.pythonhosted.org/packages/22/03/510503cfbf20f62810a9548c9be13ab86181f00cca9a3a56717c4595d952/Cython-0.29.14-cp38-cp38-win32.whl Collecting numpy==1.14.5 Using cached https://files.pythonhosted.org/packages/d5/6e/f00492653d0fdf6497a181a1c1d46bbea5a2383e7faf4c8ca6d6f3d2581d/numpy-1.14.5.zip Installing collected packages: wheel, setuptools, Cython, numpy Running setup.py install for numpy: started Running setup.py install for numpy: finished with status ‘error’ ERROR: Command errored out with exit status 1: command: ‘c:\users\fablab\appdata\local\programs\python\python38-32\python.exe’ -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’“‘C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\setup.py’”’“‘; file=’”‘“‘C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\setup.py’”’“';f=getattr(tokenize, '”‘“‘open’”’“‘, open)(file);code=f.read().replace(’”‘"’\r\n’“'”‘, ‘"’"’\n’“'”‘);f.close();exec(compile(code, file, ‘"’“‘exec’”’"‘))’ install --record ‘C:\Users\FabLab\AppData\Local\Temp\pip-record-v6v16vfn\install-record.txt’ --single-version-externally-managed --prefix ‘C:\Users\FabLab\AppData\Local\Temp\pip-build-env-grh495uw\overlay’ --compile cwd: C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy
Complete output (322 lines): Running from numpy source directory.
ERROR: Command errored out with exit status 1: ‘c:\users\fablab\appdata\local\programs\python\python38-32\python.exe’ -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’“‘C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\setup.py’”’“‘; file=’”‘“‘C:\Users\FabLab\AppData\Local\Temp\pip-install-rs48fhf4\numpy\setup.py’”’“';f=getattr(tokenize, '”‘“‘open’”’“‘, open)(file);code=f.read().replace(’”‘"’\r\n’“'”‘, ‘"’"’\n’“'”‘);f.close();exec(compile(code, file, ‘"’“‘exec’”’"‘))’ install --record ‘C:\Users\FabLab\AppData\Local\Temp\pip-record-v6v16vfn\install-record.txt’ --single-version-externally-managed --prefix ‘C:\Users\FabLab\AppData\Local\Temp\pip-build-env-grh495uw\overlay’ --compile Check the logs for full command output.
ERROR: Command errored out with exit status 1: ‘c:\users\fablab\appdata\local\programs\python\python38-32\python.exe’ ‘c:\users\fablab\appdata\local\programs\python\python38-32\lib\site-packages\pip’ install --ignore-installed --no-user --prefix ‘C:\Users\FabLab\AppData\Local\Temp\pip-build-env-grh495uw\overlay’ --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple – wheel setuptools ‘Cython>=0.29.2’ ‘numpy==1.13.3; python_version==’“'”‘3.5’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.13.3; python_version==’“'”‘3.6’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.14.5; python_version>=’“'”‘3.7’“'”’ and platform_system!=‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version==’“'”‘3.5’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version==’“'”‘3.6’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ ‘numpy==1.16.0; python_version>=’“'”‘3.7’“'”’ and platform_system==‘"’“‘AIX’”‘"’’ Check the logs for full command output.
We’ll need NumPy 1.17.3 first; 1.17.2 doesn’t support Python 3.8.
After building NumPy and SciPy
master
locally on 3.8 I see the same behavior (plus what’s fixed by #10930) – withscipy.test('full')
it gets stuck at 81% and ctrl-C allows it to finish. Running:and then
ctrl-C
ing I can see that the problem happens here:But running
pytest -ra -vv scipy/sparse/tests/
directly, I don’t get this problem. [EDIT: Removed some wrong/useless information about memory limitations.] However, thanks tosnakeviz
I was able to doAnd see that the problematic stack was (probably!):
So the problem is actually with
gc.collect
makingmultiprocessing
pool wait.In #10932 I turn this into a pytest fixture, and on my 3.8 distribution it stops here:
@andyfaff this thus looks like a probable
_differential_evolution
pool problem. Any ideas?Has this been fixed? I’m still getting this error with Python 3.8.6 with scipy 1.5
How about wheels for 3.8?