pytorch-forecasting: [BUG] `AttributeError: 'ExperimentWriter' object has no attribute 'add_figure'`

Stuck on my first few tutorials of PyTorch Forecasting because of a few bugs. I’ve sidestepped a KeyError: 'radam_buffer' error by introducing an optimizer into the model building from_dataset(). Would greatly appreciate some help!

Expected behavior

When calling fit on the model (using the exact code from the DeepAR tutorial), expected to fit the model and see a model summary according to the tutorial.

Actual behavior

However, error encountered after a single plot is produced and the following:

self.log_prediction(

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_forecasting/models/base_model.py", line 731, in log_prediction
    self.logger.experiment.add_figure(

AttributeError: 'ExperimentWriter' object has no attribute 'add_figure'

Code to reproduce the problem

net = DeepAR.from_dataset(
    training,
    learning_rate=0.1,
    optimizer='Adam',
    log_interval=10,
    log_val_interval=1,
    hidden_size=30,
    rnn_layers=2,
    loss=MultivariateNormalDistributionLoss(rank=30),
)

# =============================================================================
# fit the model
# =============================================================================
trainer.fit(
    net,
    train_dataloaders=train_dataloader,
    val_dataloaders=val_dataloader,
)

Paste the command(s) you ran and the output. Including a link to a colab notebook will speed up issue resolution. If there was a crash, please include the traceback here. The code used to initialize the TimeSeriesDataSet and model should be also included.

TRACEBACK:


  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/spyder_kernels/py3compat.py", line 356, in compat_exec
    exec(code, globals, locals)

  File "/Users/BenedictGrey/Documents/GitHub/algo_trading/models/python/pytorch_forecasting/DeepAR.py", line 160, in <module>
    trainer.fit(

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 608, in fit
    call._call_and_handle_interrupt(

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 38, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _fit_impl
    self._run(model, ckpt_path=self.ckpt_path)

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1103, in _run
    results = self._run_stage()

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1182, in _run_stage
    self._run_train()

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1195, in _run_train
    self._run_sanity_check()

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1267, in _run_sanity_check
    val_loop.run()

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
    self.advance(*args, **kwargs)

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 152, in advance
    dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
    self.advance(*args, **kwargs)

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 137, in advance
    output = self._evaluation_step(**kwargs)

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 234, in _evaluation_step
    output = self.trainer._call_strategy_hook(hook_name, *kwargs.values())

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1485, in _call_strategy_hook
    output = fn(*args, **kwargs)

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 390, in validation_step
    return self.model.validation_step(*args, **kwargs)

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_forecasting/models/base_model.py", line 420, in validation_step
    log.update(self.create_log(x, y, out, batch_idx))

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_forecasting/models/deepar/__init__.py", line 344, in create_log
    log = super().create_log(

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_forecasting/models/base_model.py", line 469, in create_log
    self.log_prediction(

  File "/Users/BenedictGrey/opt/miniconda3/envs/py310-env/lib/python3.10/site-packages/pytorch_forecasting/models/base_model.py", line 731, in log_prediction
    self.logger.experiment.add_figure(

AttributeError: 'ExperimentWriter' object has no attribute 'add_figure'

CODE TO REPRODUCE:

import warnings
warnings.filterwarnings("ignore")

import matplotlib.pyplot as plt
import pandas as pd
import pytorch_lightning as pl
from pytorch_lightning.callbacks import EarlyStopping
import torch

from pytorch_forecasting import Baseline, DeepAR, TimeSeriesDataSet
from pytorch_forecasting.data import NaNLabelEncoder
from pytorch_forecasting.data.examples import generate_ar_data
from pytorch_forecasting.metrics import SMAPE, MultivariateNormalDistributionLoss

# =============================================================================
# load data
# =============================================================================
data = generate_ar_data(seasonality=10.0, timesteps=400, n_series=1, seed=42)
data["static"] = 2
data["date"] = pd.Timestamp("2020-01-01") + pd.to_timedelta(data.time_idx, "D")
data.head()

data = data.astype(dict(series=str))


# create dataset and dataloaders
max_encoder_length = 60 # assuming this is AR term so lag?
max_prediction_length = 20 # n_steps for forecasting or prediction?

training_cutoff = data["time_idx"].max() - max_prediction_length # this seems to be finding latest date and then removing the max_pred length to find the size of the training set

context_length = max_encoder_length
prediction_length = max_prediction_length

training = TimeSeriesDataSet(
    data[lambda x: x.time_idx <= training_cutoff],
    time_idx="time_idx",
    target="value",
    categorical_encoders={"series": NaNLabelEncoder().fit(data.series)},
    group_ids=["series"],
    static_categoricals=[
        "series"
    ],  # as we plan to forecast correlations, it is important to use series characteristics (e.g. a series identifier)
    time_varying_unknown_reals=["value"],
    max_encoder_length=context_length,
    max_prediction_length=prediction_length,
)


validation = TimeSeriesDataSet.from_dataset(training, data, min_prediction_idx=training_cutoff + 1)
batch_size = 128

# synchronize samples in each batch over time - only necessary for DeepVAR, not for DeepAR
train_dataloader = training.to_dataloader(
    train=True, batch_size=batch_size, num_workers=0, batch_sampler="synchronized"
)
val_dataloader = validation.to_dataloader(
    train=False, batch_size=batch_size, num_workers=0, batch_sampler="synchronized"
)


# =============================================================================
# calculate baseline error
# =============================================================================
# calculate baseline absolute error
actuals = torch.cat([y[0] for x, y in iter(val_dataloader)])
baseline_predictions = Baseline().predict(val_dataloader)
SMAPE()(baseline_predictions, actuals)


# =============================================================================
# init network
# =============================================================================
pl.seed_everything(42)
import pytorch_forecasting as ptf

trainer = pl.Trainer(gpus=0, gradient_clip_val=1e-1)
net = DeepAR.from_dataset(
    training, learning_rate=3e-2, optimizer='Adam', hidden_size=30, rnn_layers=2, loss=MultivariateNormalDistributionLoss(rank=30)
)



# =============================================================================
# find optimal lr
# =============================================================================
res = trainer.tuner.lr_find(
    net,
    train_dataloaders=train_dataloader,
    val_dataloaders=val_dataloader,
    min_lr=1e-5,
    max_lr=1e0,
    early_stop_threshold=100,
)
print(f"suggested learning rate: {res.suggestion()}")
fig = res.plot(show=True, suggest=True)
fig.show()
net.hparams.learning_rate = res.suggestion()

# =============================================================================
# early stopping
# =============================================================================
early_stop_callback = EarlyStopping(monitor="val_loss", min_delta=1e-4, patience=10, verbose=False, mode="min")
trainer = pl.Trainer(
    max_epochs=30,
    gpus=0,
    enable_model_summary=True,
    gradient_clip_val=0.1,
    callbacks=[early_stop_callback],
    limit_train_batches=50,
    enable_checkpointing=True,
)

# =============================================================================
# new net with opt learning rate
# =============================================================================
net = DeepAR.from_dataset(
    training,
    learning_rate=0.1,
    optimizer='Adam',
    log_interval=10,
    log_val_interval=1,
    hidden_size=30,
    rnn_layers=2,
    loss=MultivariateNormalDistributionLoss(rank=30),
)

# =============================================================================
# fit the model
# =============================================================================
trainer.fit(
    net,
    train_dataloaders=train_dataloader,
    val_dataloaders=val_dataloader,
)

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 4
  • Comments: 17 (5 by maintainers)

Commits related to this issue

Most upvoted comments

I was able to fix the same error by installing tensorboard via pip. PL seems to check for the tensorboard installation.

@xhxlab This workaround worked for me. I pip installed tensorboard and created my own initialized tensorboardlogger and specified it in the trainer object.

from pytorch_lightning import loggers as pl_loggers
tensorboard = pl_loggers.TensorBoardLogger('./')

trainer = pl.Trainer(
    max_epochs=30,
    enable_model_summary=True,
    gradient_clip_val=0.1,
    callbacks=[early_stop_callback],
    limit_train_batches=50,
    enable_checkpointing=True,
    logger=tensorboard
)

@ewth, could you kindly open a pull request? Then I might also advise with the pre-commit, we have removed poetry from the dev tools.

It is indeed @fkiraly.

I went to create a bug fix PR but having some issues with poetry and pre-commit. Working to get my local env setup and will issue the PR.