mmdetection: CUDA out of memory
Dear All ,
I have a question reagrd the problem of CUDA out of the memory.
I try to train a CentriapetalNet on my custom dataset , which has the same format of COCO dataset. this is the config file : https://github.com/open-mmlab/mmdetection/tree/master/configs/centripetalnet I got such error massage on CUDA memory , I changed the following code from 63 to 164
data = dict(
samples_per_gpu=6,
workers_per_gpu=3,
I noticed that changing such batch size help in reduce the CUDA memory needs during the training. but it is still not working , I did not understand the theory behind . also I really confused what should I do to continues the training.
I read this document : https://mmdetection.readthedocs.io/en/latest/faq.html , the part of CUDA memory
and i did not know how to change those three items described there.
any help will be highly appreciated.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 30 (8 by maintainers)
scikit-image and pillow has resize functions, or if you have a small dataset, you can resize the images manually (by photo editor software) to make it work at first.
check this
https://mmdetection.readthedocs.io/en/latest/tutorials/data_pipeline.html