pytorch-lightning: Inconsistent behaviour and "AttributeError: _old_init" when using Pytorch Lightning with the ogb library.
🐛 Bug
The simple example from the documentation page https://pytorch-lightning.readthedocs.io/en/latest/notebooks/lightning_examples/mnist-hello-world.html works fine. However, when I add the import instruction from ogb.nodeproppred import *, the code has inconsistent behaviour: sometimes it works and sometimes it throws an “AttributeError: _old_init” exception:
Traceback (most recent call last):
File "plexample.py", line 54, in <module>
trainer.fit(mnist_model, train_loader)
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 700, in fit
self._call_and_handle_interrupt(
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 654, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 741, in _fit_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1166, in _run
results = self._run_stage()
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1252, in _run_stage
return self._run_train()
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1282, in _run_train
self.fit_loop.run()
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/loops/loop.py", line 195, in run
self.on_run_start(*args, **kwargs)
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/loops/fit_loop.py", line 210, in on_run_start
self.trainer.reset_train_dataloader(self.trainer.lightning_module)
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1811, in reset_train_dataloader
self.train_dataloader = self._data_connector._request_dataloader(RunningStage.TRAINING)
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 430, in _request_dataloader
dataloader = source.dataloader()
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/contextlib.py", line 126, in __exit__
next(self.gen)
File "/home/schlyah/anaconda3/envs/pipinstallenv/lib/python3.9/site-packages/pytorch_lightning/utilities/data.py", line 527, in _replace_init_method
del cls._old_init
AttributeError: _old_init
To Reproduce
The documentation example code with the import instruction:
import os
import pandas as pd
import torch
from pytorch_lightning import LightningModule, Trainer
from pytorch_lightning.callbacks.progress import TQDMProgressBar
from pytorch_lightning.loggers import CSVLogger
from torch import nn
from torch.nn import functional as F
from torch.utils.data import DataLoader, random_split
from torchvision import transforms
from torchvision.datasets import MNIST
import argparse
from ogb.nodeproppred import *
PATH_DATASETS = os.environ.get("PATH_DATASETS", ".")
BATCH_SIZE = 256 if torch.cuda.is_available() else 64
class MNISTModel(LightningModule):
def __init__(self):
super().__init__()
self.l1 = torch.nn.Linear(28 * 28, 10)
def forward(self, x):
return torch.relu(self.l1(x.view(x.size(0), -1)))
def training_step(self, batch, batch_nb):
x, y = batch
loss = F.cross_entropy(self(x), y)
return loss
def configure_optimizers(self):
return torch.optim.Adam(self.parameters(), lr=0.02)
if __name__ == '__main__':
# Init our model
mnist_model = MNISTModel()
# Init DataLoader from MNIST Dataset
train_ds = MNIST(PATH_DATASETS, train=True, download=True, transform=transforms.ToTensor())
train_loader = DataLoader(train_ds, batch_size=BATCH_SIZE)
# Initialize a trainer
trainer = Trainer(
accelerator="auto",
devices=1 if torch.cuda.is_available() else None, # limiting got iPython runs
max_epochs=3,
callbacks=[TQDMProgressBar(refresh_rate=20)],
)
# Train the model ⚡
trainer.fit(mnist_model, train_loader)
Environment
- PyTorch Lightning Version: 1.7.0
- PyTorch Version: 1.12.0
- Python version: 3.9
- OS: Linux
- CUDA/cuDNN version: 10.2
- GPU models and configuration:
- How you installed PyTorch: pip
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 3
- Comments: 15 (9 by maintainers)
Now, the problem is fixed. Thank you
Hi, everyone. I used @PurpleSand123 example with one extra test script
When running this with PL
1.7.0, the output on my machine was:When running with the change in the linked PR, the output on my machine was:
Please, try your examples with the changes in the linked PR and check if it fixes it for you.
I am having the same problem when using torch_geometric.loader.DataLoader For debugging, I changed “_replace_init_method” code like this
It works well sometimes and get output as
However, sometimes, the error occurs and the output is
Hi @schlyah, I spent a while on this and am unable to reproduce. Would you mind sharing your full environment (i.e.
pip freeze) as well? Thanks a lot!