sktime: [BUG] Very different behaviour depending on whether make_pipeline is from sktime or sklearn
Describe the bug
It seems odd that make_pipeline should produce very different score results. I found this exploring one of the Examples files:
https://www.sktime.net/en/latest/examples/classification/channel_selection.html .
To Reproduce
Υou can see the difference in results by commenting/uncommenting the appropriate make_pipeline lines
below. Results are 0.25 (sklearn) and 0.975 (sktime), suggesting there is some important differences here.
import warnings
warnings.filterwarnings("ignore")
from sklearn.linear_model import RidgeClassifierCV
# Which version of make_pipeline should we use when results differ so much!!?!?!
# from sklearn.pipeline import make_pipeline
from sktime.pipeline import make_pipeline
from sktime.datasets import load_UCR_UEA_dataset
from sktime.transformations.panel import channel_selection
from sktime.transformations.panel.rocket import Rocket
cs = channel_selection.ElbowClassPairwise() # ECP
rocket_pipeline = make_pipeline(cs, Rocket(), RidgeClassifierCV())
data = "BasicMotions"
X_train, y_train = load_UCR_UEA_dataset(data, split="train", return_X_y=True)
X_test, y_test = load_UCR_UEA_dataset(data, split="test", return_X_y=True)
rocket_pipeline.fit(X_train, y_train)
rocket_pipeline.score(X_test, y_test)
Expected behavior I didn’t expect such a difference in results.
Additional context
What happens if you’ve loaded both forms of make_pipeline in one session? Is only the first form active?
Versions
System: python: 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0] executable: /opt/conda/bin/python3.8 machine: Linux-5.15.0-72-generic-x86_64-with-glibc2.10
Python dependencies: pip: 23.3.2 sktime: 0.25.0 sklearn: 0.24.2 skbase: 0.6.2 numpy: 1.22.4 scipy: 1.6.3 pandas: 1.2.4 matplotlib: 3.4.1 joblib: 1.0.1 numba: 0.53.1 statsmodels: 0.12.2 pmdarima: None statsforecast: None tsfresh: None tslearn: None torch: None tensorflow: None tensorflow_probability: None
About this issue
- Original URL
- State: open
- Created 5 months ago
- Comments: 27
This might be possible in a few days …