statsmodels: BUG: ExponentialSmoothing returns NaNs from forecast method without any error

Describe the bug

ExponentialSmoothing is returning NaNs from the forecast method.

Expected output

Values being in the result of forecast/predict method or exception raised in case model should return NaNs (ideally already in fit). Hint to the user how to fix it would also be nice.

Code Sample, a copy-pastable example if possible

import pandas as pd
from statsmodels.tsa.holtwinters import ExponentialSmoothing

y = [2342., 2226., 2487., 3082., 2735., 2178., 1717., 2182., 2215., 1663., 1067., 1261.,
     1073.,  986.,  559.,  990., 1561., 1665., 1528., 1807., 1402., 1045.,  981., 1369.,
     1417., 1317., 1256., 1574., 1652., 1479., 1427., 1658., 1738., 1579., 1460., 1863.,
     1749., 1422., 1189.,  972.,  949., 1028.,  660.,  806., 1049.,  974.,  800.]

X = pd.date_range(start='2000-09-30', periods=len(y), freq='Q').astype(str)

endog = pd.DataFrame(X).assign(y=y).set_index(0)

exp_smooth = ExponentialSmoothing(endog=endog, 
                                  damped=True, 
                                  dates=None,
                                  freq=None,                                  
                                  missing='none', 
                                  seasonal='add', 
                                  seasonal_periods=4, 
                                  trend='mul')
Convergence and RunTime warnings during fit
exp_smooth_fitted = exp_smooth.fit()
# working possibilities found
# exp_smooth_fitted = exp_smooth.fit(**{'smoothing_seasonal':0.1})
# exp_smooth_fitted = exp_smooth.fit(**{'use_boxcox':True})
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/base/tsa_model.py:163: ValueWarning: No frequency information was provided, so inferred frequency Q-DEC will be used.
  % freq, ValueWarning)
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/holtwinters.py:725: ConvergenceWarning: Optimization failed to converge. Check mle_retvals.
  ConvergenceWarning)
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/holtwinters.py:894: RuntimeWarning: invalid value encountered in power
  (alphac * trended(lvls[i - 1], dampen(b[i - 1], phi)))
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/holtwinters.py:897: RuntimeWarning: invalid value encountered in power
  (betac * dampen(b[i - 1], phi))
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/holtwinters.py:899: RuntimeWarning: invalid value encountered in power
  (gamma * trended(lvls[i - 1], dampen(b[i - 1], phi))) + \
/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels/tsa/holtwinters.py:905: RuntimeWarning: invalid value encountered in power
  b[:nobs] = dampen(b[:nobs], phi)
NaNs in forecast
exp_smooth_fitted.forecast(5)
2012-06-30   NaN
2012-09-30   NaN
2012-12-31   NaN
2013-03-31   NaN
2013-06-30   NaN
Freq: Q-DEC, dtype: float64
NaNs in MLE
exp_smooth_fitted.mle_retvals
      fun: nan
 hess_inv: <10x10 LbfgsInvHessProduct with dtype=float64>
      jac: array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan])
  message: b'ABNORMAL_TERMINATION_IN_LNSRCH'
     nfev: 231
      nit: 0
   status: 2
  success: False
        x: array([2.63157895e-01, 1.05263158e-01, 0.00000000e+00, 1.65683333e+03,
       9.65586667e-01, 5.26315789e-01, 6.85166667e+02, 5.69166667e+02,
       8.30166667e+02, 1.42516667e+03])
Zero in gamma
exp_smooth_fitted.summary()
 ExponentialSmoothing Model Results                       
================================================================================
Dep. Variable:                    endog   No. Observations:                   47
Model:             ExponentialSmoothing   SSE                                nan
Optimized:                         True   AIC                                nan
Trend:                   Multiplicative   BIC                                nan
Seasonal:                      Additive   AICC                               nan
Seasonal Periods:                     4   Date:                 Tue, 12 Nov 2019
Box-Cox:                          False   Time:                         08:32:59
Box-Cox Coeff.:                    None                                         
=================================================================================
                          coeff                 code              optimized      
---------------------------------------------------------------------------------
smoothing_level               0.2631579                alpha                 True
smoothing_slope               0.1052632                 beta                 True
smoothing_seasonal             0.000000                gamma                 True
initial_level                 1656.8333                  l.0                 True
initial_slope                 0.9817375                  b.0                 True
damping_slope                 0.5263158                  phi                 True
initial_seasons.0             685.16667                  s.0                 True
initial_seasons.1             569.16667                  s.1                 True
initial_seasons.2             830.16667                  s.2                 True
initial_seasons.3             1425.1667                  s.3                 True
---------------------------------------------------------------------------------

Note: As you can see, there are many issues on our GitHub tracker, so it is very possible that your issue has been posted before. Please check first before submitting so that we do not have to handle and close duplicates.

Note: Please be sure you are using the latest released version of statsmodels, or a recent build of master. If your problem has been fixed in an unreleased version, you might be able to use master until a new release occurs.

Note: If you are using a released version, have you verified that the bug exists in the master branch of this repository? It helps the limited resources if we know problems exist in the current master so that they do not need to check whether the code sample produces a bug in the next release.

If the issue has not been resolved, please file it in the issue tracker.

Expected Output

Values being in the result of forecast/predict method or exception raised in case model should return NaNs (ideally already in fit). Hint to the user how to fix it would also be nice.

Output of import statsmodels.api as sm; sm.show_versions()

INSTALLED VERSIONS

Python: 3.7.3.final.0 OS: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Sat Oct 12 00:02:19 PDT 2019; root:xnu-4903.278.12~1/RELEASE_X86_64 x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8

statsmodels

Installed: 0.11.0dev0+543.gc45cd757b (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/statsmodels)

Required Dependencies

cython: 0.29.13 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/Cython) numpy: 1.16.4 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/numpy) scipy: 1.3.1 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/scipy) pandas: 0.25.2 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/pandas) dateutil: 2.8.0 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/dateutil) patsy: 0.5.1 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/patsy)

Optional Dependencies

matplotlib: 3.1.1 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/matplotlib) backend: module://ipykernel.pylab.backend_inline cvxopt: Not installed joblib: 0.13.2 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/joblib)

Developer Tools

IPython: 7.9.0 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/IPython) jinja2: 2.10.1 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/jinja2) sphinx: Not installed pygments: 2.4.2 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages/pygments) pytest: 5.0.1 (/Users/michalchromcak/miniconda3/envs/sias_backend/lib/python3.7/site-packages) virtualenv: Not installed

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 15 (11 by maintainers)

Most upvoted comments

That is correct. Probably wont be until Summer. You can either build and install master or possiblly install a nightly from

https://anaconda.org/scipy-wheels-nightly/statsmodels

(these are a bit old and I’m not sure fixed).

Thanks very much for this report, and especially for all the detail you included.