Try this: it’s worked for me in Window 10.

pip uninstall bitsandbytes
pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.0-py3-none-win_amd64.whl

Upgrading via pip install -U bitsandbytes should resolve this issue. I will close this issue. Please reopen if upgrading does not resolve the error.

exactly the same error for me - you may have to downgrade to 0.38.x (this will not have 4 bit training though)

Try this: it’s worked for me in Window 10.

pip uninstall bitsandbytes
pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.0-py3-none-win_amd64.whl

This solution works for me.

Still receiving this with the latest version. Using GCP Workbench Managed Notebook (with PyTorch kernel) and this is what I get:

===================================BUG REPORT===================================
================================================================================
The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib64'), PosixPath('/usr/local/lib/x86_64-linux-gnu'), PosixPath('/usr/local/cuda/lib'), PosixPath('/usr/local/nvidia/lib')}
The following directories listed in your path were found to be non-existent: {PosixPath('pytorch-gpu/1-12+cu113')}
The following directories listed in your path were found to be non-existent: {PosixPath('/proxy/%PORT%')}
The following directories listed in your path were found to be non-existent: {PosixPath('/proxy/%PORT%')}
The following directories listed in your path were found to be non-existent: {PosixPath('//matplotlib_inline.backend_inline'), PosixPath('module')}
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
DEBUG: Possible options found for libcudart.so: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}
CUDA SETUP: PyTorch settings found: CUDA_VERSION=113, Highest Compute Capability: 8.0.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Required library version not found: libbitsandbytes_cuda113.so. Maybe you need to compile it from source?
CUDA SETUP: Defaulting to libbitsandbytes_cpu.so...

================================================ERROR=====================================
CUDA SETUP: CUDA detection failed! Possible reasons:
1. You need to manually override the PyTorch CUDA version. Please see: "https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
2. CUDA driver not installed
3. CUDA not installed
4. You have multiple conflicting CUDA libraries
5. Required library not pre-compiled for this bitsandbytes release!
CUDA SETUP: If you compiled from source, try again with `make CUDA_VERSION=DETECTED_CUDA_VERSION` for example, `make CUDA_VERSION=113`.
CUDA SETUP: The CUDA version for the compile might depend on your conda install. Inspect CUDA version via `conda list | grep cuda`.
================================================================================

CUDA SETUP: Something unexpected happened. Please compile from source:
git clone https://github.com/TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=113 make cuda11x
python setup.py install
CUDA SETUP: Setup Failed!
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes


  warn(msg)
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: /opt/conda did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
  warn(msg)
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}.. We select the PyTorch default libcudart.so, which is {torch.version.cuda},but this might missmatch with the CUDA version that is needed for bitsandbytes.To override this behavior set the BNB_CUDA_VERSION=<version string, e.g. 122> environmental variableFor example, if you want to use the CUDA version 122BNB_CUDA_VERSION=122 python ...OR set the environmental variable in your .bashrc: export BNB_CUDA_VERSION=122In the case of a manual override, make sure you set the LD_LIBRARY_PATH, e.g.export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2
  warn(msg)
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: /usr/local/cuda/lib64:/usr/local/cuda/lib:/usr/local/lib/x86_64-linux-gnu:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
  warn(msg)

Building it manually doesn’t resolve the issue.

This is my LD_LIBRARY_PATH:

‘/usr/local/cuda/lib64:/usr/local/cuda/lib:/usr/local/lib/x86_64-linux-gnu:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64’

and libcudart.so actually exist in /usr/local/cuda/lib64

@ChensStudio thank you so much.

Adding the below lines at the end of .bahrc file saved me from getting the same error.

export CUDA_HOME=/usr/local/cuda-11.8
export LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64:/usr/lib/wsl/lib
export PATH=$CUDA_HOME/bin:$PATH

It was a classic scope problem again. Script can’t find the cuda location. I spent my hours to solve that. I really appreciate your help 🙏

Hi @matterphiz I am getting this same error when I am running the following code

from peft import LoraConfig, get_peft_model

Can you please provide a fix to this.

Thanks!

Downgraded to 0.38.1, exact same error.

Compiled from source code. same error here when run python -m bitsandbytes

using CUDA11.8 python 3.10

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

bin /usr/local/lib/python3.10/dist-packages/bitsandbytes-0.39.0-py3.10.egg/bitsandbytes/libbitsandbytes_cpu.so False CUDA exception! Error code: no CUDA-capable device is detected CUDA exception! Error code: initialization error CUDA SETUP: CUDA runtime path found: /usr/local/cuda-11.8/targets/x86_64-linux/lib/libcudart.so.11.0 /usr/local/lib/python3.10/dist-packages/bitsandbytes-0.39.0-py3.10.egg/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: No GPU detected! Check your CUDA paths. Proceeding to load CPU-only library… warn(msg) CUDA SETUP: Detected CUDA version 118 CUDA SETUP: Required library version not found: libbitsandbytes_cpu.so. Maybe you need to compile it from source? CUDA SETUP: Defaulting to libbitsandbytes_cpu.so…

================================================ERROR===================================== CUDA SETUP: CUDA detection failed! Possible reasons:

1. CUDA driver not installed
2. CUDA not installed
3. You have multiple conflicting CUDA libraries

4. Required library not pre-compiled for this bitsandbytes release! CUDA SETUP: If you compiled from source, try again with make CUDA_VERSION=DETECTED_CUDA_VERSION for example, make CUDA_VERSION=113. CUDA SETUP: The CUDA version for the compile might depend on your conda install. Inspect CUDA version via conda list | grep cuda.

CUDA SETUP: Something unexpected happened. Please compile from source: git clone git@github.com:TimDettmers/bitsandbytes.git cd bitsandbytes CUDA_VERSION=118 make cuda11x_nomatmul python setup.py install CUDA SETUP: Setup Failed! Traceback (most recent call last): File “/usr/lib/python3.10/runpy.py”, line 187, in _run_module_as_main mod_name, mod_spec, code = _get_module_details(mod_name, _Error) File “/usr/lib/python3.10/runpy.py”, line 146, in _get_module_details return _get_module_details(pkg_main_name, error) File “/usr/lib/python3.10/runpy.py”, line 110, in _get_module_details import(pkg_name) File “/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.39.0-py3.10.egg/bitsandbytes/init.py”, line 6, in from . import cuda_setup, utils, research File “/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.39.0-py3.10.egg/bitsandbytes/research/init.py”, line 1, in from . import nn File “/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.39.0-py3.10.egg/bitsandbytes/research/nn/init.py”, line 1, in from .modules import LinearFP8Mixed, LinearFP8Global File “/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.39.0-py3.10.egg/bitsandbytes/research/nn/modules.py”, line 8, in from bitsandbytes.optim import GlobalOptimManager File “/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.39.0-py3.10.egg/bitsandbytes/optim/init.py”, line 6, in from bitsandbytes.cextension import COMPILED_WITH_CUDA File “/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.39.0-py3.10.egg/bitsandbytes/cextension.py”, line 20, in raise RuntimeError(‘’’ RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

Got fixed. btw, I am using WSL2 in windows.

Have it fixed by downgrade to python 3.8 (may not be the root cause), and reinstall CUDA 11.8

And set below env export CUDA_HOME=/usr/local/cuda-11.8 export LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64:/usr/lib/wsl/lib export PATH=$CUDA_HOME/bin:$PATH