sktime: [BUG] `ForecastByLevel` with native parallelization is not faster than without

_Originally posted by @marrov in https://github.com/sktime/sktime/discussions/6216#discussioncomment-8929469_


Versions:

sktime==0.27.0
skbase==0.7.5

Minimal example:

from sktime.forecasting.ets import AutoETS
from sktime.forecasting.naive import NaiveForecaster
from sktime.forecasting.base import ForecastingHorizon
from sktime.utils._testing.hierarchical import _make_hierarchical
from sktime.forecasting.compose import ForecastByLevel, FallbackForecaster

# Similar df to my example: 5 levels and 100k rows with period index with ~100 timepoints
df = _make_hierarchical(
    hierarchy_levels = (2, 2, 5, 5, 10),
    max_timepoints = 100,
    min_timepoints = 100,
    same_cutoff = True,
    n_columns = 1,
    all_positive = True,
    index_type = "period",
    random_state = 0,
    add_nan = False,
)

fh =  ForecastingHorizon((range(1, 13)))

model = ForecastByLevel(
    FallbackForecaster(
        forecasters=[
            ("autoets", AutoETS(auto=True, sp=12, n_jobs=-1)),
            ("naive", NaiveForecaster()),
        ],
    )
)

model_config = {
    "backend:parallel": "joblib",
    "backend:parallel:params": {"backend": "loky", "n_jobs": -1},
}

model.set_config(**model_config)

model.fit(y=df, fh=fh)

In my local laptop this takes ~6 minutes without the config (commenting out the second to last line) and ~5 minutes with the parallel config). Am I doing something wrong here?

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 18 (6 by maintainers)

Most upvoted comments

@marrov, @ninedigits, what happens if you nest joblib Parallel calls?

Will it do the sensible thing, or something problematic? E.g., if the two n_jobs multiply to a number that’s less than your cores, will it do the sensible thing?

I’m not 100% certain this is the issue, but I suspect that having both parallel options enabled may cause the performance to behave unexpectedly. I’ve run into this issue before. A quick check would be to run htop to monitor core performance while executing the job. It should look like this:

image

Cores should max out at 100%. Anything over means performance degradation.