sktime: [BUG] "update_params=False" for statsmodels adapter should call apply
The current logic of update(update_params=False) in the statsmodels interfaces is not correct, the model simply produces predictions it would have made before the update, instead of applying the fitted model to new data.
Instead, the apply interface from statsmodels should be used appropriately, see the discussion below. Previously, this was available only for a few models, but seems to be a general interface point now.
Describe the bug
After estimated a model, I would like to make forecast after my test set, but without updating parameters (changing the model parameters). This is specially important for comparison, moreover to forecast with new data without changing the model.
I have tried to set update_params=False in “update()” and in “update_predict_single()”, but it seems that is not working. Dates (timestamp) are updated, but forecast is the same for the previous period. I do not know If I am doing something wrong, or it is a bug. I have an example with auto.arima (code bellow and pdf attached).
Kind regards
To Reproduce
import pandas as pd
import numpy as np
from sktime.datasets import load_airline
from sktime.forecasting.arima import AutoARIMA
from sktime import show_versions
show_versions()
## data split
y=load_airline() ## total observations
y1 = y.head(y.shape[0]-6) #train observations
y2 = y.tail(6) # test observations
forecaster = AutoARIMA(sp=12, suppress_warnings=True,random_state=123)
z=forecaster.fit(y1)
AutoARIMA(...)
y_pred = forecaster.predict(fh=[1,2,3,4,5,6])
### model configuration and parameters for train
z.summary()
y_pred_semfit = forecaster.predict(fh=[1,2,3,4,5,6]) ## o cutoff 1960-06
y_pred_semfit
## using a function from stkime for not update_params, but to forecast using the test portion of the data set (to append to train y1)
y_pred = forecaster.update_predict_single(y_new=y2, update_params=False)
y_pred
### dates changed, but forecasted values are the same as above
### just changing the input of the function
y_pred = forecaster.update_predict_single(y=y, update_params=False)
y_pred
### another function from stkime, without update/retrain model parameters using update_params=False
updated_model = forecaster.update(y=y, update_params=False)
updated_model.predict()
### however, if we set "update_params=True", time for command excution increase and forecasted values change
updated_model = forecaster.update(y=y2, update_params=True)
updated_model.predict()
## but model parameters/configuration are different
updated_model.summary()
### using another function from Sktime, .fit_predict, the same results as above
y_pred = forecaster.fit_predict(y,fh=[1,2,3,4,5,6])
y_pred
forecaster.summary()
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 28 (5 by maintainers)
FYI @raphaccf, I´ve updated the header of this issue, so it´s now an instruction what to do.
I haven’t opened them yet. I will feel more comfortable (avoid any mistakes in the process) if you do it, sorry.