bitsandbytes: non-existent path throwing Error invalid device ordinal

Hello, I’m trying to run the following HuggingFace notebook. The code runs fine on Colab, but when trying it (with no changes) on my system (Windows 11, WSL2, NVidia Quadro RTX 5000 16GB) I get the following error upon starting the training loop (trainer.train()):

You're using a GPTNeoXTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
/home/uccollab/support-text-gen/lib/python3.9/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py:230: UserWarning: where received a uint8 condition tensor. This behavior is deprecated and will be removed in a future version of PyTorch. Use a boolean condition instead. (Triggered internally at ../aten/src/ATen/native/TensorCompare.cpp:493.)
  attn_scores = torch.where(causal_mask, attn_scores, mask_value)
Error invalid device ordinal at line 359 in file /home/tim/git/bitsandbytes/csrc/pythonInterface.c

I’ve seen that this is potentially fixable by editing pythonInterface.c, according to this other thread

The problem is the specified path “/home/tim/git/bitsandbytes/csrc/pythonInterface.c”, there’s no such path on my WSL installation (also “tim” is not my username). I have no clue where bitesandbytes is getting that file from and a search doesn’t return anything as well. I also tried compiling from source but the issue persists.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 2
  • Comments: 27 (3 by maintainers)

Commits related to this issue

Most upvoted comments

@TimDettmers https://github.com/stoperro/bitsandbytes_windows/commit/e02f078000ed19fe57321c464dd16d60f18d6803

This is what is being used in my and acpopescu’s forks. Just skipping cudaMemPrefetchAsync when not supported.