tensor_parallel: why raised cuda error?
i used tiiuae/falcon-40b
and want to doing full fine-tuning by lima instruction dataset
model = tp.tensor_parallel(model, sharded=True)
just use like this and i have 1) A100 80GB * 2 and another server 2) A100 80GB * 4
but when i running code on 1) device or 2) device
raised error like this
Model parameters were moved to incorrect devices, did call on model.cuda() or model.to(device)? If so, please avoid doing that
why?
About this issue
- Original URL
- State: closed
- Created a year ago
- Reactions: 1
- Comments: 18 (2 by maintainers)
upper transformers v4.30.0, i think this error is accured