scikit-learn: Parallalism in gridsearcCV is ending up with a permission error

Interestingly,

import numpy as np
import pandas as pd
from sklearn.model_selection import RandomizedSearchCV
from sklearn.ensemble import RandomForestClassifier


for _ in range(10):
    X_train = np.random.rand(int(2e6)).reshape((int(1e6), 2))
    y_train = np.random.randint(0, 2, int(1e6))
    X_train = pd.DataFrame(X_train)

    clf = RandomForestClassifier()
    gs = RandomizedSearchCV(
        clf,
        param_distributions={"n_estimators": np.array([1]),
                                           "max_depth": np.array([2])},
        n_iter=1,
        cv=2,
        scoring="accuracy",
        verbose=1,
        n_jobs=2
    )

    gs.fit(X_train, y_train)

always fails (never at the first iteration of the for loop). Note the use of a pandas dataframe for X_train.

However when X_train is a numpy array

import numpy as np
import pandas as pd
from sklearn.model_selection import RandomizedSearchCV
from sklearn.ensemble import RandomForestClassifier


for _ in range(10):
    X_train = np.random.rand(int(2e6)).reshape((int(1e6), 2))
    y_train = np.random.randint(0, 2, int(1e6))

    clf = RandomForestClassifier()
    gs = RandomizedSearchCV(
        clf,
        param_distributions={"n_estimators": np.array([1]),
                                           "max_depth": np.array([2])},
        n_iter=1,
        cv=2,
        scoring="accuracy",
        verbose=1,
        n_jobs=2
    )

    gs.fit(X_train, y_train)

does not fail.

The fact that I cannot reproduce with a VM might be caused by the fact that memory mapped files might behave differently in a VM.

I will try to reproduce with a CI worker in this PR: https://github.com/joblib/joblib/pull/942

Please offer your comment at joblib.

Yes, in my last tests it happened the same as you mentioned. Numpy arrays work (at least on the implementation I am working ) and if I try a dataframe, it tends to raise an error after the fifth iteraction.

Actually, when in the same ipython session I first run the snippet with X_train as a pandas dataframe and then the snippet with X_train as a numpy array I get the error for both. But if quit the ipython session, restart a new one and run the snippet with X_train as a numpy array I don’t have an error.

@albertcthomas Thanks for providing the snippets. By me both pieces of code, with numpy arrays and dataframes I get an error.

With dataframe as X_train:

Fitting 2 folds for each of 1 candidates, totalling 2 fits
[Parallel(n_jobs=2)]: Using backend LokyBackend with 2 concurrent workers.
[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.5s remaining:    0.0s
[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.5s finished
C:\Users\lucas\Anaconda3\lib\site-packages\sklearn\externals\joblib\disk.py:122: UserWarning: Unable to delete folder C:\Users\lucas\AppData\Local\Temp\joblib_memmapping_folder_113764_4112142399 after 5 tentatives.
  .format(folder_path, RM_SUBDIRS_N_RETRY))

The run with numpy arrays:

Fitting 2 folds for each of 1 candidates, totalling 2 fits
[Parallel(n_jobs=2)]: Using backend LokyBackend with 2 concurrent workers.
[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.4s remaining:    0.0s
[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    0.4s finished
C:\Users\lucas\Anaconda3\lib\site-packages\sklearn\externals\joblib\disk.py:122: UserWarning: Unable to delete folder C:\Users\lucas\AppData\Local\Temp\joblib_memmapping_folder_113764_4112142399 after 5 tentatives.
  .format(folder_path, RM_SUBDIRS_N_RETRY))

In scikit-learn 0.19.2, I tried running the code you provided, so:

import numpy as np
import pandas as pd
from sklearn.model_selection import RandomizedSearchCV
from sklearn.ensemble import RandomForestClassifier
import sklearn

print(sklearn.__version__)
for _ in range(10):
    X_train = np.random.rand(int(2e6)).reshape((int(1e6), 2))
    y_train = np.random.randint(0, 2, int(1e6))
    X_train = pd.DataFrame(X_train)

    clf = RandomForestClassifier()
    gs = RandomizedSearchCV(
        clf,
        param_distributions={"n_estimators": np.array([1]),
                                           "max_depth": np.array([2])},
        n_iter=1,
        cv=2,
        scoring="accuracy",
        verbose=1,
        n_jobs=-1
    )

    gs.fit(X_train, y_train)

It runs the random search without error.

Have to check the library dependancies, as in another computer it run smoothly with 0.20.

If you have any idea what I can still try, I am open to suggestions. In the meanwhile, I will continue with 0.19

Thanks! Instead of sending pictures, readability and reusability of your code can be greatly improved if you format your code snippets and complete error messages appropriately. For example:

```python
print(something)
```

generates:

print(something)

And:

```pytb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'hello'
```

generates:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'hello'

Also, if the code is very long you can link to it like this.

You can edit your comments at any time to improve readability. This helps maintainers a lot.