bitsandbytes: CUDA Setup failed despite GPU being available. Please run the following command to get more information:
System Info
Ubuntu 22.04.3, Python 3.10.12, i7 14700K, RTX 4090.
Reproduction
I’m struggling to run kohya ss, there are constant issues with bitsandbytes. No matter what I do:
- upgrade from pip install -U bitsandbytes (from either root, or venv)
- run ./setup.sh
- run ./gui.sh, and start training
- run python -m bitsandbytes from root
- run python -m bitsandbytes from venv I get the following output:
The following directories listed in your path were found to be non-existent: {PosixPath('/etc/xdg/xdg-ubuntu')}
The following directories listed in your path were found to be non-existent: {PosixPath('local/g-Z790-AERO-G'), PosixPath('@/tmp/.ICE-unix/2493,unix/g-Z790-AERO-G')}
The following directories listed in your path were found to be non-existent: {PosixPath('0'), PosixPath('1')}
The following directories listed in your path were found to be non-existent: {PosixPath('/etc/xdg/xdg-ubuntu')}
The following directories listed in your path were found to be non-existent: {PosixPath('unix'), PosixPath('path=/run/user/1000/bus,guid=d29f3ce95df599d870367a8065ca8048')}
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
DEBUG: Possible options found for libcudart.so: {PosixPath('/usr/local/cuda/lib64/libcudart.so')}
CUDA SETUP: PyTorch settings found: CUDA_VERSION=118, Highest Compute Capability: 8.9.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Loading binary /media/g/New Volume/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda118.so...
libcusparse.so.11: cannot open shared object file: No such file or directory
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone https://github.com/TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=118 make cuda11x
python setup.py install
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/lib/python3.10/runpy.py", line 146, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
__import__(pkg_name)
File "/media/g/New Volume/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/__init__.py", line 6, in <module>
from . import cuda_setup, utils, research
File "/media/g/New Volume/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/research/__init__.py", line 1, in <module>
from . import nn
File "/media/g/New Volume/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/research/nn/__init__.py", line 1, in <module>
from .modules import LinearFP8Mixed, LinearFP8Global
File "/media/g/New Volume/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/research/nn/modules.py", line 8, in <module>
from bitsandbytes.optim import GlobalOptimManager
File "/media/g/New Volume/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/optim/__init__.py", line 6, in <module>
from bitsandbytes.cextension import COMPILED_WITH_CUDA
File "/media/g/New Volume/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 20, in <module>
raise RuntimeError('''
RuntimeError:
CUDA Setup failed despite GPU being available. Please run the following command to get more information:
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
The same issue is on WIndows.
Expected behavior
It shouldn’t throw errors.
About this issue
- Original URL
- State: closed
- Created 5 months ago
- Comments: 19 (3 by maintainers)
I had the same issue but following steps helped me resolve the issue:
Hey all, sorry for the struggles you’ve been having! Often the issues are related to the underlying CUDA installation and not bitsandbytes itself. I agree that the error traces are quite confusing (they are actually trying to help fix the setup issues) and often look the same (but the details in the output are often different):
Currently, there’s very active work on improving those diagnostics to be better and clearer (just merged a big PR today). I plan significant improvements on that side for the next two releases.
BNB works for most setups, but we still have this long tail of issues where people are having a really difficult time and it’s not easy to diagnose. As we’re getting up to speed with maintenance (we took over not too long ago and there is a big backlog of technical debt), we’ll be sure to make your life easier.
For now, this is already the case for Windows users. Since release
0.43.0, we have official Windows support. You can simply install it withpip install bitsandbytes>=0.43.0Please, Windows users, install in this way and let us know that it worked without issues.
Thanks everyone for your patience and feedback 🤗
or you can download prebuilt binary wheels from workflows. for example: (login required) https://github.com/TimDettmers/bitsandbytes/actions/runs/7836890878
bdist_wheel_*zip. in this zip you can find*.whlfile.pip install bitsandbytes_*.whlI download this and fixed the error. https://pypi.org/project/bitsandbytes/#files
Hi
Thank you for efforts.
Unfortunetly, I tried all ways to install the library on Linux 20.4
Is there any way to do that?
doesn’t work on google collab:/. i get this: ERROR: bitsandbytes-0.41.1-py3-none-win_amd64.whl is not a supported wheel on this platform.
works on windows
https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl this wheel include almost all available libs of cuda versions. the size of this wheel is about ~146MB!
but we don’t have to compile/packaging all available cuda libs, torch for windows includes cuda libraries, for example, torch-2.1.x+cu121 includes
cudart64_14.dlland it can be installed on cuda 11.8 environment. so, bitsandbytes for cuda11.8 could be used under cuda 11.6 if cuda11.8 libs incuded in wheel. (or just copy cuda11.8 libs from somewhere likeconda-forgeand extract needed cuda libs intobitsandbytes/dir, and bitsandbytes for cuda11.8 could be used on cuda11.6 system)and another issue for linux (I guess this issue is related in this post), bitsandbytes have to compiled on ubuntu-20.04 to fix
GLIBC_2.xx.yysymbol issue.Thank you @Shivam-21-11. now it is working with cuda 11.5 in my system.
From what I encountered the error is caused due to wrong combination of version of cuda and bitsandbytes. I had to find bitsandbyte compatible with my cuda (11.6) and model (falco-7b from HuggingFace). Try to find the source package of bitsandbyte which is compatible with your cuda.