scikit-learn: Verbosity option is not working in GridSearchCV (Jupyter notebook)

@lesteve I’m also having the problem on SLURM as well, even after updating scikit-learn to 1.3.0. Only works properly when n_jobs=1.

I also tried your suggestion in the comment you referenced above, but it didn’t help. I’m not entirely sure if I did it right. I used the argument --signal=B:TERM@n_duration to send the SIGTERM signal n_duration seconds before the job’s time limit is reached. Then I ran sys.stdout.flush() and sys.stderr.flush() in a function that I passed to signal.signal(signal.SIGTERM, flush_function). It didn’t resolve the issue.

@lesteve should we open another issue for this behavior outside of Jupyter then? I have a similar issue outside of Jupyter Notebook as @princyok and @Jose-Verdu-Diaz. Fresh sklearn install at v1.3.0 from conda-forge. Occurs for me with slurm jobs as @Jose-Verdu-Diaz describes but also regular command line execution on Linux. The behavior occurs so long as n_jobs != 1( as far as I can tell). Only the first line regarding the total number of combinations and validation setsis printed.

When I use:

from joblib import parallel_backend

grid_obj = GridSearchCV(... n_jobs=-1, verbose=3)

with parallel_backend('multiprocessing'):
    grid_obj.fit(x_text, y_test)

I can see verbose in terminal instead of notebook. I do not know why, but it works for me.

Yes, please start by opening a separate issue for the regular command line execution on Linux, which definitely seems unexpected. Please add a standalone snippet reproducing the problem, which will make it a lot more likely it get looked at and eventually fixed.

On the SLURM one, it may be that there is some buffering going on in different places (SLURM side, OS side, …), see https://github.com/dask/dask-jobqueue/issues/299#issuecomment-609002077 for some quick experiments I made a while ago on a SGE cluster.

I have same problem. Linux, scikit-learn 0.23.2, not running in a notebook. I’m running from command line and printing to a file. It prints properly when n_jobs=None (same as n_jobs=1), but no progress is printed when n_jobs=-1.

Regardless of what n_jobs is set to, it prints the first line, “Fitting n folds for each of nn candidates, totalling nnn fits”, and then stops there if n_jobs=-1.

Are there any updates on this? I have the same problem on Apple M1. I’m also teaching machine learning and I see that most of my students also have this problem. We set n_jobs=4 for them. No verbosity setting gives any output while running, but the output appears after the job is completed (often even in a different cell. It’s very annoying since it’s demotivating for the students if they don’t know how far along their gridsearch is

Reading the top post in more details:

• it seems like you need verbose > 1 if you want more details about the cross-validation progress. Maybe you did not need it in scikit-learn 0.22.1 but you definitely need it now. You can double-check with n_jobs=1 and verbose=10, you should definitely see some output about the cross-validation progress
• there is something fishy with subprocess output capture (maybe only for loky backend not 100% sure). The first time I execute the cell, I don’t get any output about CV progress, the second time I do …

A screenshot with a slight variation (first .fit does not show any progress, second .fit does):

When I use the multiprocessing backend I do get the output consistently so maybe a weird interaction between loky and ipykernel sub-process output capturing?

from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForestClassifier
from sklearn import datasets

from joblib import parallel_backend

iris = datasets.load_iris()
params = {'n_estimators':[10,20,30,40,50,60],
          'max_depth':[20,50,60,70,80]}

grid_obj = GridSearchCV(estimator=RandomForestClassifier(), param_grid=params, n_jobs=-1, verbose=10, cv=5)

with parallel_backend('multiprocessing'):
    grid_obj.fit(iris.data, iris.target)