text: sampler unable in BucketIterator

unable to use XLAs Distributed Data Sampler or any Multi-GPU training with BucketIterator because it doesnt have a sampler feature. train_iterator , valid_iterator = BucketIterator.splits((train_data, test_data), batch_size=batch_size, sort_within_batch=True, sort_key = lambda x: len(x.word_token), device=device)

so i am constraint to using only one GPU.

i used BucketIterator because it gives good batches with minimal padding, but the limiting scaling factor is a constraint.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 33 (14 by maintainers)

Most upvoted comments

@zhangguanheng66 I was thinking of contributing to torchtext , is it possible ? … I would love to contribute

For this kind of questions, you can use people on Pytorch NLP forum