scikit-learn: ImportError: dlopen: cannot load any more object with static TLS with torch built with gcc 5.5

I am not sure if this is a PyTorch bug, a scikit-learn bug or a numba, but this used to work in scikit-learn 0.20.3 and stopped working in the 0.21.0 series, so for now I am going to venture a guess that it is a regression in scikit learn.

When I do the following series of imports (minimized from the original import, which was import librosa), loading the following program fails:

import torch
import soundfile
import scipy.signal
import numba
import sklearn

with

Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/sklearn/__check_build/__init__.py", line 44, in <module>
    from ._check_build import check_build  # noqa
ImportError: dlopen: cannot load any more object with static TLS

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test_torch.py", line 5, in <module>
    import sklearn
  File "/opt/conda/lib/python3.6/site-packages/sklearn/__init__.py", line 75, in <module>
    from . import __check_build
  File "/opt/conda/lib/python3.6/site-packages/sklearn/__check_build/__init__.py", line 46, in <module>
    raise_build_error(e)
  File "/opt/conda/lib/python3.6/site-packages/sklearn/__check_build/__init__.py", line 41, in raise_build_error
    %s""" % (e, local_dir, ''.join(dir_content).strip(), msg))
ImportError: dlopen: cannot load any more object with static TLS
___________________________________________________________________________
Contents of /opt/conda/lib/python3.6/site-packages/sklearn/__check_build:
_check_build.cpython-36m-x86_64-linux-gnu.so__pycache__               __init__.py
setup.py
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.

Downgrading to scikit-learn 0.20.3 makes the problem go away.

Versions

jenkins@260bf77532d0:~/workspace/test$ python
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sklearn; sklearn.show_versions()

System:
    python: 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34)  [GCC 7.3.0]
executable: /opt/conda/bin/python
   machine: Linux-4.15.0-29-generic-x86_64-with-debian-jessie-sid

BLAS:
    macros: SCIPY_MKL_H=None, HAVE_CBLAS=None
  lib_dirs: /opt/conda/lib
cblas_libs: mkl_rt, pthread

Python deps:
       pip: 19.1.1
setuptools: 41.0.1
   sklearn: 0.21.2
     numpy: 1.16.4
     scipy: 1.1.0
    Cython: None
    pandas: None

Also, you may be interested in:

jenkins@260bf77532d0:~/workspace/test$ pip list | grep numba
numba                  0.43.1         
jenkins@260bf77532d0:~/workspace/test$ pip list | grep torch
torch                  1.2.0a0+ab800ad

The build of torch must be done with gcc 5.5.0 to cause this problem; other versions of gcc are known not to cause this problem.

For ease of reproduction, you can use the following docker image ezyang/scikit-learn-tls-repro:1 https://cloud.docker.com/repository/registry-1.docker.io/ezyang/scikit-learn-tls-repro Once in, follow the reproduction instructions as described above. (EDIT At time of writing, the Docker image is still uploading. Should be done soon.)