auto-sklearn: Can't install auto-sklearn due to pyrfr dependency

@sfalkner haha wow… can’t believe I didn’t catch that O____O

Here’s the recipe, I got it to work:

conda install swig --yes
pip install  pyrfr # probably don't need this b/c it reinstalls in auto-sklearn
pip install auto-sklearn

Output:

Collecting auto-sklearn
Requirement already satisfied: Cython in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Collecting pyrfr<0.8,>=0.6.1 (from auto-sklearn)
  Using cached https://files.pythonhosted.org/packages/c3/c6/c555cfa3c7d0078dded091d4901ed52344bbb925077aa70b871faf35fd58/pyrfr-0.7.4.tar.gz
Requirement already satisfied: scipy>=0.14.1 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Collecting smac<0.9,>=0.8 (from auto-sklearn)
Requirement already satisfied: nose in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Requirement already satisfied: pandas in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Requirement already satisfied: six in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Requirement already satisfied: joblib in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Requirement already satisfied: psutil in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Requirement already satisfied: scikit-learn<0.20,>=0.19 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Collecting liac-arff (from auto-sklearn)
Collecting pynisher<0.5,>=0.4 (from auto-sklearn)
Requirement already satisfied: lockfile in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Requirement already satisfied: setuptools in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Requirement already satisfied: pyyaml in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Collecting ConfigSpace<0.5,>=0.4.0 (from auto-sklearn)
Requirement already satisfied: numpy>=1.9.0 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from auto-sklearn)
Collecting sphinx (from smac<0.9,>=0.8->auto-sklearn)
  Using cached https://files.pythonhosted.org/packages/89/44/73cd04b856fa35a69e9e2e790aa283e2eaefb684e116f1d46d01a5e7986f/Sphinx-1.7.4-py2.py3-none-any.whl
Collecting typing (from smac<0.9,>=0.8->auto-sklearn)
  Using cached https://files.pythonhosted.org/packages/05/2b/2b05bf1d5a9dd450447c9a5df3e118a465e5d3cb12b73b7220a5064a403f/typing-3.6.4-py3-none-any.whl
Collecting sphinx-rtd-theme (from smac<0.9,>=0.8->auto-sklearn)
  Downloading https://files.pythonhosted.org/packages/47/33/e3a1cc08acf0c50418c505d1b954adc950141ace691e3b4a22ba91950a32/sphinx_rtd_theme-0.3.1-py2.py3-none-any.whl (2.2MB)
    100% |████████████████████████████████| 2.2MB 609kB/s
Requirement already satisfied: python-dateutil>=2 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from pandas->auto-sklearn)
Requirement already satisfied: pytz>=2011k in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from pandas->auto-sklearn)
Collecting docutils>=0.3 (from pynisher<0.5,>=0.4->auto-sklearn)
  Using cached https://files.pythonhosted.org/packages/36/fa/08e9e6e0e3cbd1d362c3bbee8d01d0aedb2155c4ac112b19ef3cae8eed8d/docutils-0.14-py3-none-any.whl
Requirement already satisfied: pyparsing in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from ConfigSpace<0.5,>=0.4.0->auto-sklearn)
Requirement already satisfied: packaging in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from sphinx->smac<0.9,>=0.8->auto-sklearn)
Collecting alabaster<0.8,>=0.7 (from sphinx->smac<0.9,>=0.8->auto-sklearn)
  Using cached https://files.pythonhosted.org/packages/2e/c3/9b7dcd8548cf2c00531763ba154e524af575e8f36701bacfe5bcadc67440/alabaster-0.7.10-py2.py3-none-any.whl
Collecting snowballstemmer>=1.1 (from sphinx->smac<0.9,>=0.8->auto-sklearn)
  Using cached https://files.pythonhosted.org/packages/d4/6c/8a935e2c7b54a37714656d753e4187ee0631988184ed50c0cf6476858566/snowballstemmer-1.2.1-py2.py3-none-any.whl
Collecting imagesize (from sphinx->smac<0.9,>=0.8->auto-sklearn)
  Using cached https://files.pythonhosted.org/packages/e9/79/31cc1c2e0daf575f8fd2b581e2975e6a6938bd439581f766b79c50479521/imagesize-1.0.0-py2.py3-none-any.whl
Collecting sphinxcontrib-websupport (from sphinx->smac<0.9,>=0.8->auto-sklearn)
  Using cached https://files.pythonhosted.org/packages/56/0f/3ee19ca5e5a1d9751cf4bbeb372d40a46421c4321fe55a4703ba66d0bafb/sphinxcontrib_websupport-1.0.1-py2.py3-none-any.whl
Collecting babel!=2.0,>=1.3 (from sphinx->smac<0.9,>=0.8->auto-sklearn)
  Using cached https://files.pythonhosted.org/packages/94/03/14e68ad12e771a79cf96792f7158d68a7b3d8c7b2badf39e9ef1f65b57d6/Babel-2.5.3-py2.py3-none-any.whl
Requirement already satisfied: Jinja2>=2.3 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from sphinx->smac<0.9,>=0.8->auto-sklearn)
Requirement already satisfied: Pygments>=2.0 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from sphinx->smac<0.9,>=0.8->auto-sklearn)
Requirement already satisfied: requests>=2.0.0 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from sphinx->smac<0.9,>=0.8->auto-sklearn)
Requirement already satisfied: MarkupSafe>=0.23 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from Jinja2>=2.3->sphinx->smac<0.9,>=0.8->auto-sklearn)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from requests>=2.0.0->sphinx->smac<0.9,>=0.8->auto-sklearn)
Requirement already satisfied: idna<2.7,>=2.5 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from requests>=2.0.0->sphinx->smac<0.9,>=0.8->auto-sklearn)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from requests>=2.0.0->sphinx->smac<0.9,>=0.8->auto-sklearn)
Requirement already satisfied: certifi>=2017.4.17 in ./anaconda/envs/py3_clone/lib/python3.6/site-packages (from requests>=2.0.0->sphinx->smac<0.9,>=0.8->auto-sklearn)
Building wheels for collected packages: pyrfr
  Running setup.py bdist_wheel for pyrfr ... done
  Stored in directory: /Users/jespinoz/Library/Caches/pip/wheels/fb/98/fd/b1d53cab6d5ed836980777d9733d7e549d82a727650eed6f6d
Successfully built pyrfr
Installing collected packages: pyrfr, alabaster, snowballstemmer, imagesize, sphinxcontrib-websupport, babel, docutils, sphinx, pynisher, typing, ConfigSpace, sphinx-rtd-theme, smac, liac-arff, auto-sklearn
  Found existing installation: pyrfr 0.8.0
    Uninstalling pyrfr-0.8.0:
      Successfully uninstalled pyrfr-0.8.0
Successfully installed ConfigSpace-0.4.6 alabaster-0.7.10 auto-sklearn-0.3.0 babel-2.5.3 docutils-0.14 imagesize-1.0.0 liac-arff-2.2.1 pynisher-0.4.2 pyrfr-0.7.4 smac-0.8.0 snowballstemmer-1.2.1 sphinx-1.7.4 sphinx-rtd-theme-0.3.1 sphinxcontrib-websupport-1.0.1 typing-3.6.4

This worked for me:

sudo apt-get remove swig
sudo apt-get install swig3.0
sudo ln -s /usr/bin/swig3.0 /usr/bin/swig
pip install --user pyrfr
pip install --user auto-sklearn 

macOS install

FYI, it seems you can replace conda install swig with brew install swig from @jolespin 's instructions, if you happen not to use conda

This worked for me to install:

brew install swig
pip install  pyrfr # probably don't need this b/c it reinstalls in auto-sklearn
pip install auto-sklearn

(However, while installation completes, models still can’t seem to train due to something like #600 which crashes the kernel in jupyterlab. Looks like #679 may fix)

As I expected, your include path contains different python libraries, specifically the option:

-I/opt/software/Python/2.7.2–GCC-4.4.5/include … -I/mnt/home/olsonran/anaconda3/lib/python3.5/site-packages/numpy/core/include -I/mnt/home/olsonran/anaconda3/include/python3.5m

tells the compiler to look first for python 2.7 header files before looking into your anaconda files. This is what causes the runtime problems you experience: The API changed from python 2 to 3 such that the old header files mess things up. You also get a lot of warnings as a result.

You will have to find a way to add your anaconda path before the system’s python include directory. How that is done on CentOS, I don’t know, but maybe you know (because the anaconda path is already in there somehow). If not, I am sure your local sysadmin or Google knows. Let me know, if that still does not solve you problem. Hope that helps…

!apt-get install swig -y !pip install Cython numpy !pip install auto-sklearn

This will work !!

It Works, thanx sudo apt-get remove swig sudo apt-get install swig3.0 sudo ln -s /usr/bin/swig3.0 /usr/bin/swig pip install --user pyrfr pip install --user auto-sklearn

This worked for me:

sudo apt-get remove swig
sudo apt-get install swig3.0
sudo ln -s /usr/bin/swig3.0 /usr/bin/swig
pip install --user pyrfr
pip install --user auto-sklearn 

This worked for me, many thanks…

@jolespin your solution works like a charm.

Thanks @jolespin …that solved my issue too…

Thanks @jolespin it solves my problem too.

Within Anaconda, auto-sklearn and its dependencies are fairly easy to install, if you know that you need swig and a somewhat recent GCC.

Huzzah, that was it! The sysadmins snuck a module load Python/2.7 in on me. Nice debugging and thank you for the help, @sfalkner.

@mfeurer: Do you have ~30 minutes this week to discuss auto-sklearn to make sure I run this benchmark properly?