pytorch_geometric: RuntimeError: copy_if failed to synchronize: an illegal memory access was encountered
When I`am training, the model works fine. But when I start testing, this error occurs. Why?
epoch: 1 loss: 3.392380952835083 train_batch_acc: 0.0625
epoch: 1 loss: 2.941084146499634 train_batch_acc: 0.15625
epoch: 1 loss: 2.138481616973877 train_batch_acc: 0.34375
epoch: 1 loss: 1.3853774070739746 train_batch_acc: 0.59375
epoch: 1 loss: 1.3555893898010254 train_batch_acc: 0.65625
epoch: 1 loss: 0.9268301725387573 train_batch_acc: 0.75
epoch: 1 loss: 0.8250795006752014 train_batch_acc: 0.6875
train acc is 0.436500
epoch: 1 test_batch_acc: 0.03125
Traceback (most recent call last):
File "train_3d_graph_gait.py", line 174, in <module>
test_acc = test(test_loader)
File "train_3d_graph_gait.py", line 134, in test
end_point = model(data)
File "/home/anaconda/envs/graph-wyx/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__
result = self.forward(*input, **kwargs)
File "train_3d_graph_gait.py", line 82, in forward
x = max_pool_x(cluster, data.x, data.batch, size=1)
File "/home/anaconda/envs/graph-wyx/lib/python3.6/site-packages/torch_geometric/nn/pool/max_pool.py", line 33, in max_pool_x
return _max_pool_x(cluster, x, (batch.max().item() + 1) * size)
File "/home/anaconda/envs/graph-wyx/lib/python3.6/site-packages/torch_geometric/nn/pool/max_pool.py", line 9, in _max_pool_x
return scatter_('max', x, cluster, dim=0, dim_size=size)
File "/home/anaconda/envs/graph-wyx/lib/python3.6/site-packages/torch_geometric/utils/scatter.py", line 39, in scatter_
out[out == fill_value] = 0
RuntimeError: copy_if failed to synchronize: an illegal memory access was encountered
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 19 (5 by maintainers)
Seems like not all data is on the same device. Can you check that?
My fault. I forgot to move the model to CUDA.