bitsandbytes: non-existent path throwing Error invalid device ordinal
Hello, I’m trying to run the following HuggingFace notebook. The code runs fine on Colab, but when trying it (with no changes) on my system (Windows 11, WSL2, NVidia Quadro RTX 5000 16GB) I get the following error upon starting the training loop (trainer.train()):
You're using a GPTNeoXTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
/home/uccollab/support-text-gen/lib/python3.9/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py:230: UserWarning: where received a uint8 condition tensor. This behavior is deprecated and will be removed in a future version of PyTorch. Use a boolean condition instead. (Triggered internally at ../aten/src/ATen/native/TensorCompare.cpp:493.)
attn_scores = torch.where(causal_mask, attn_scores, mask_value)
Error invalid device ordinal at line 359 in file /home/tim/git/bitsandbytes/csrc/pythonInterface.c
I’ve seen that this is potentially fixable by editing pythonInterface.c, according to this other thread
The problem is the specified path “/home/tim/git/bitsandbytes/csrc/pythonInterface.c”, there’s no such path on my WSL installation (also “tim” is not my username). I have no clue where bitesandbytes is getting that file from and a search doesn’t return anything as well. I also tried compiling from source but the issue persists.
About this issue
- Original URL
- State: closed
- Created a year ago
- Reactions: 2
- Comments: 27 (3 by maintainers)
Commits related to this issue
- Guard for prefetchAsync GPU capability. #470 #451 #477 — committed to TimDettmers/bitsandbytes by TimDettmers a year ago
@TimDettmers https://github.com/stoperro/bitsandbytes_windows/commit/e02f078000ed19fe57321c464dd16d60f18d6803
This is what is being used in my and acpopescu’s forks. Just skipping
cudaMemPrefetchAsyncwhen not supported.