statsmodels: BUG: ExponentialSmoothing returns NaNs from forecast method without any error
Describe the bug
ExponentialSmoothing is returning NaNs from the forecast method.
Expected output
Values being in the result of forecast/predict method or exception raised in case model should return NaNs (ideally already in fit). Hint to the user how to fix it would also be nice.
Code Sample, a copy-pastable example if possible
import pandas as pd
from statsmodels.tsa.holtwinters import ExponentialSmoothing
y = [2342., 2226., 2487., 3082., 2735., 2178., 1717., 2182., 2215., 1663., 1067., 1261.,
1073., 986., 559., 990., 1561., 1665., 1528., 1807., 1402., 1045., 981., 1369.,
1417., 1317., 1256., 1574., 1652., 1479., 1427., 1658., 1738., 1579., 1460., 1863.,
1749., 1422., 1189., 972., 949., 1028., 660., 806., 1049., 974., 800.]
X = pd.date_range(start='2000-09-30', periods=len(y), freq='Q').astype(str)
endog = pd.DataFrame(X).assign(y=y).set_index(0)
exp_smooth = ExponentialSmoothing(endog=endog,
damped=True,
dates=None,
freq=None,
missing='none',
seasonal='add',
seasonal_periods=4,
trend='mul')
Convergence and RunTime warnings during fit
exp_smooth_fitted = exp_smooth.fit()
# working possibilities found
# exp_smooth_fitted = exp_smooth.fit(**{'smoothing_seasonal':0.1})
# exp_smooth_fitted = exp_smooth.fit(**{'use_boxcox':True})
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/base/tsa_model.py:163: ValueWarning: No frequency information was provided, so inferred frequency Q-DEC will be used.
% freq, ValueWarning)
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/holtwinters.py:725: ConvergenceWarning: Optimization failed to converge. Check mle_retvals.
ConvergenceWarning)
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/holtwinters.py:894: RuntimeWarning: invalid value encountered in power
(alphac * trended(lvls[i - 1], dampen(b[i - 1], phi)))
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/holtwinters.py:897: RuntimeWarning: invalid value encountered in power
(betac * dampen(b[i - 1], phi))
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/holtwinters.py:899: RuntimeWarning: invalid value encountered in power
(gamma * trended(lvls[i - 1], dampen(b[i - 1], phi))) + \
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/holtwinters.py:905: RuntimeWarning: invalid value encountered in power
b[:nobs] = dampen(b[:nobs], phi)
NaNs in forecast
exp_smooth_fitted.forecast(5)
2012-06-30 NaN
2012-09-30 NaN
2012-12-31 NaN
2013-03-31 NaN
2013-06-30 NaN
Freq: Q-DEC, dtype: float64
NaNs in MLE
exp_smooth_fitted.mle_retvals
fun: nan
hess_inv: <10x10 LbfgsInvHessProduct with dtype=float64>
jac: array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan])
message: b'ABNORMAL_TERMINATION_IN_LNSRCH'
nfev: 231
nit: 0
status: 2
success: False
x: array([2.63157895e-01, 1.05263158e-01, 0.00000000e+00, 1.65683333e+03,
9.65586667e-01, 5.26315789e-01, 6.85166667e+02, 5.69166667e+02,
8.30166667e+02, 1.42516667e+03])
Zero in gamma
exp_smooth_fitted.summary()
ExponentialSmoothing Model Results
================================================================================
Dep. Variable: endog No. Observations: 47
Model: ExponentialSmoothing SSE nan
Optimized: True AIC nan
Trend: Multiplicative BIC nan
Seasonal: Additive AICC nan
Seasonal Periods: 4 Date: Tue, 12 Nov 2019
Box-Cox: False Time: 08:32:59
Box-Cox Coeff.: None
=================================================================================
coeff code optimized
---------------------------------------------------------------------------------
smoothing_level 0.2631579 alpha True
smoothing_slope 0.1052632 beta True
smoothing_seasonal 0.000000 gamma True
initial_level 1656.8333 l.0 True
initial_slope 0.9817375 b.0 True
damping_slope 0.5263158 phi True
initial_seasons.0 685.16667 s.0 True
initial_seasons.1 569.16667 s.1 True
initial_seasons.2 830.16667 s.2 True
initial_seasons.3 1425.1667 s.3 True
---------------------------------------------------------------------------------
Note: As you can see, there are many issues on our GitHub tracker, so it is very possible that your issue has been posted before. Please check first before submitting so that we do not have to handle and close duplicates.
Note: Please be sure you are using the latest released version of statsmodels, or a recent build of master. If your problem has been fixed in an unreleased version, you might be able to use master until a new release occurs.
Note: If you are using a released version, have you verified that the bug exists in the master branch of this repository? It helps the limited resources if we know problems exist in the current master so that they do not need to check whether the code sample produces a bug in the next release.
If the issue has not been resolved, please file it in the issue tracker.
Expected Output
Values being in the result of forecast/predict method or exception raised in case model should return NaNs (ideally already in fit). Hint to the user how to fix it would also be nice.
Output of import statsmodels.api as sm; sm.show_versions()
INSTALLED VERSIONS
Python: 3.7.3.final.0 OS: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Sat Oct 12 00:02:19 PDT 2019; root:xnu-4903.278.12~1/RELEASE_X86_64 x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8
statsmodels
Installed: 0.11.0dev0+543.gc45cd757b (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels)
Required Dependencies
cython: 0.29.13 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/Cython) numpy: 1.16.4 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/numpy) scipy: 1.3.1 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/scipy) pandas: 0.25.2 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/pandas) dateutil: 2.8.0 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/dateutil) patsy: 0.5.1 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/patsy)
Optional Dependencies
matplotlib: 3.1.1 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/matplotlib) backend: module://ipykernel.pylab.backend_inline cvxopt: Not installed joblib: 0.13.2 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/joblib)
Developer Tools
IPython: 7.9.0 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/IPython) jinja2: 2.10.1 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/jinja2) sphinx: Not installed pygments: 2.4.2 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/pygments) pytest: 5.0.1 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages) virtualenv: Not installed
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 15 (11 by maintainers)
That is correct. Probably wont be until Summer. You can either build and install master or possiblly install a nightly from
https://anaconda.org/scipy-wheels-nightly/statsmodels
(these are a bit old and I’m not sure fixed).
Thanks very much for this report, and especially for all the detail you included.