sktime: [BUG] `LabelEncoder` does not fit in a nested pipeline
Important: this is a discussion assuming the current state of #6458.
This error is not related to it but for reproducing this error, code changed in that pr is necessary. LabelEncoder returns a not fitted error when used in a nested pipeline after calling predict. To Reproduce
import numpy
import pandas
from sklearn.preprocessing import LabelEncoder, StandardScaler
from sktime.forecasting.arima import ARIMA
from sktime.transformations.series.subset import ColumnSelect
different_series = ["A", "B"]
different_dates = pandas.date_range(start="2000-01-01", end="2000-03-31", freq="D")
number_of_series = len(different_series)
number_of_timesteps = len(different_dates)
random_generator = numpy.random.default_rng(seed=0)
different_values = [1, 2, 3]
sample_data = {
"series": numpy.repeat(different_series, number_of_timesteps),
"dates": numpy.tile(different_dates, number_of_series),
"P": random_generator.standard_normal(size=number_of_series * number_of_timesteps),
"Q": random_generator.choice(
different_values, size=number_of_series * number_of_timesteps, replace=True
),
"R": random_generator.standard_exponential(size=number_of_series * number_of_timesteps),
}
sample_dataset = pandas.DataFrame(data=sample_data)
sample_dataset = sample_dataset.set_index(["series", "dates"])
pipeline = (ColumnSelect(columns=["Q"]) * LabelEncoder()) ** (StandardScaler() * ARIMA())
pipeline.fit(sample_dataset[["P"]], X=sample_dataset[["Q", "R"]])
z=pandas.DataFrame({"Q":[1,2,3], "R":[2.3,1.2,1.8]})
pipeline.predict(fh=[1,2,3],X=z)
Expected behavior
Should run without error. Additional context
Though the above pipeline doesnt work, the following does:
pipeline1 = (ColumnSelect(columns=["Q"]) * LabelEncoder())
pipeline1.fit(sample_dataset[["Q", "R"]])
pipeline1.transform(sample_dataset[["Q", "R"]])
Versions
System:
python: 3.10.12 (main, Jul 5 2023, 18:54:27) [GCC 11.2.0]
machine: Linux-6.5.0-35-generic-x86_64-with-glibc2.35
Python dependencies: pip: 24.0 sktime: 0.29.0 sklearn: 1.3.2 skbase: 0.5.2 numpy: 1.24.3 scipy: 1.11.3 pandas: 2.0.2 matplotlib: 3.7.2 joblib: 1.3.2 numba: 0.58.1 statsmodels: 0.14.0 pmdarima: 2.0.3 statsforecast: 1.5.0 tsfresh: 0.20.1 tslearn: 0.5.3.2 torch: 2.2.1+cu121 tensorflow: 2.13.0 tensorflow_probability: None
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 24 (12 by maintainers)
Could you check after
fit, and just before the error, the identity of:transformer_attributetransformer_'sclasses_attribute?Are these the same objects? I.e., identical (as via
isor - probably easier to check via -id), or just instances of the same class but non-identical?Also, could you check the adaptor’s
fit_is_emptytag viaget_tags()just before the error?@fkiraly should pr #6458 be added to the project board? Though all tests pass and it fixes #5867 , since I will be making commits for this issue in the same pr as well, it is still ‘under progress’ .
in the above statement, can you kindly explain, which of the objects are you referring to precisely? There are three: the pipeline, the adaptor, the
LabelEncoder. Which one is trying to access an attribute that it does not have, when? Which one does have the attribute, when?#6458 fixes the scitype error reported in issue #5867 which prevented the encoder from working at all. This error(of not getting fit) occurs after solving the scitype error and only in nested pipelines. So I just thought since its a different error, I’d open a new issue.
Sure, I’ll continue debugging and make commits in the same pr then.