scikit-learn: `pip install` fails (compile error) with Python 3.7rc1

Description

Unable to pip install(compile error: PyThreadState has no member …) with Python 3.7rc1.

My apologies if this has been reported, but I couldn’t find it in the issues (closed/open) nor any pull request for it.

There have been similar issues with pyyaml (yaml/pyyaml#126), numpy (numpy/numpy#10500), cython (cython/cython#1978), and pygame (pygame/pygame#382).

Steps/Code to Reproduce

Install Python 3.7rc1. pip install scikit-learn

Expected Results

pip succeeds.

Actual Results

Compilation errors:

...
sklearn/cluster/_dbscan_inner.cpp: In function ‘int
__Pyx__GetException(PyThreadState*, PyObject**, PyObject**, PyObject**)’:

sklearn/cluster/_dbscan_inner.cpp:6029:24: error: ‘PyThreadState’ has no member named ‘exc_type’
     tmp_type = tstate->exc_type;
                        ^
sklearn/cluster/_dbscan_inner.cpp:6030:25: error: ‘PyThreadState’ has no member named ‘exc_value’
     tmp_value = tstate->exc_value;
                         ^
sklearn/cluster/_dbscan_inner.cpp:6031:22: error: ‘PyThreadState’ has no member named ‘exc_traceback’
     tmp_tb = tstate->exc_traceback;
                      ^
sklearn/cluster/_dbscan_inner.cpp:6032:13: error: ‘PyThreadState’ has no member named ‘exc_type’
     tstate->exc_type = local_type;
             ^
sklearn/cluster/_dbscan_inner.cpp:6033:13: error: ‘PyThreadState’ has no member named ‘exc_value’
     tstate->exc_value = local_value;
             ^
sklearn/cluster/_dbscan_inner.cpp:6034:13: error: ‘PyThreadState’ has no member named ‘exc_traceback’
     tstate->exc_traceback = local_tb;
             ^
error: Command "g++ -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g
-fwrapv -O3 -Wall -fPIC
-I/data/install/lib/python3.7/site-packages/numpy/core/include
-I/data/install/lib/python3.7/site-packages/numpy/core/include
-I/data/install/include/python3.7m -c sklearn/cluster/_dbscan_inner.cpp -o
build/temp.linux-x86_64-3.7/sklearn/cluster/_dbscan_inner.o -MMD -MF
build/temp.linux-x86_64-3.7/sklearn/cluster/_dbscan_inner.o.d" failed with exit
status 1

Versions

Linux-3.10.0-862.el7.x86_64-x86_64-with-redhat-7.5-Maipo
Python 3.7.0rc1 (default, Jun 19 2018, 10:54:58) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
NumPy 1.14.5
SciPy 1.1.0

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 10
Comments: 29 (21 by maintainers)

Most upvoted comments

I think the point is that the average Homebrew user will have Python 3.7 by default.

A minor 0.19.2 release that will support Python 3.7 is in progress.

+38

rth on Jul 10, 2018

We could consider releasing 0.19.2 in a hurry just for Py3.7 support??

+19

jnothman on Jul 3, 2018

Closing as fixed. Please use 0.19.2 with Python 3.7.

rth on Jul 18, 2018

Thanks for reporting this!

Does installation of the latest dev version works?

pip install https://github.com/scikit-learn/scikit-learn/archive/master.zip

Note that you would need Cython >=0.27.3 to compile it…

(Linking to https://github.com/scikit-learn/scikit-learn/issues/11224 for future reference)

rth on Jul 3, 2018

On Ubuntu 18.04 the recommended workaround works: pip install https://github.com/scikit-learn/scikit-learn/archive/master.zip

coulanuk on Jul 6, 2018

If we end up doing a 0.19.2 release, here are two things to consider adding because they missed 0.19.1 for no particular good reason.

nose hard dependency removal. https://github.com/scikit-learn/scikit-learn/pull/9670
fix race condition in get_params. https://github.com/scikit-learn/scikit-learn/pull/9569#issuecomment-344081597

lesteve on Jul 4, 2018

@bgyarfas If you don’t have 3.6 cached in homebrew you can use pyenv. The following worked for me…

brew install pyenv
pyenv install 3.6.5
eval "$(pyenv init -)"
pyenv shell 3.6.5
pip install virtualenv
virtualenv venv-3.6
source venv-3.6/bin/activate

You can do this without virtualenv, but my typical setup relies on it.

ledona on Jul 10, 2018

@rth

A month very roughly for the next release. Meanwhile you can either install the development version #11320 (comment) or downgrade to python 3.6.

Homebrew has updated their default python installation to 3.7. Since homebrew doesn’t allow for installing specific versions downgrading to 3.6 is not an easy prospect. Just an FYI

bgyarfas on Jul 9, 2018

@sam-s If you need it now, you can try installing the master branch using the latest cython version:

pip install -U cython
pip install -e git+https://github.com/scikit-learn/scikit-learn@master#egg=scikit-learn

This worked for me on Python 3.7 using pipenv. Of course, when the release is available, you should swap from the (possibly) unstable master branch to the correct release.

shoeffner on Jul 12, 2018

I think the point is that the average Homebrew user will have Python 3.7 by default.

cbrnr on Jul 10, 2018

This works only if the previous version is still cached on the local machine.

cbrnr on Jul 10, 2018

@bgyarfas it should be possible to downgrade from python 3.7 back to 3.6.5 in brew

brew info python3
brew switch python 3.6.5

shusso on Jul 9, 2018

when do you think I will be able to do pip3 install scikit-learn? a week? a month? a year?

A month very roughly for the next release. Meanwhile you can either install the development version https://github.com/scikit-learn/scikit-learn/issues/11320#issuecomment-398556438 or downgrade to python 3.6.

rth on Jul 3, 2018