bitsandbytes: ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`

System Info

# FROM python:3.9

# WORKDIR /code
# COPY ./requirements.txt /code/requirements.txt
# RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt

# COPY . .
# # RUN pip install --no-cache-dir --upgrade -r /requirements.txt

# # uvicorn reward_modeling:app --host 0.0.0.0 --port 6006 --reload
# CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "6006"]
# # CMD ["./my-test-shell.sh"]

# Use the official Python 3.9 image
FROM python:3.10

# Set the working directory to /code
WORKDIR /code

# Copy the current directory contents into the container at /code
COPY ./requirements.txt /code/requirements.txt

RUN pip install -i https://pypi.org/simple/ bitsandbytes --upgrade
# Install requirements.txt 
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt


# Set up a new user named "user" with user ID 1000
RUN useradd -m -u 1000 user
# Switch to the "user" user
USER user
# Set home to the user's home directory
ENV HOME=/home/user \
	PATH=/home/user/.local/bin:$PATH

# Set the working directory to the user's home directory
WORKDIR $HOME/app

# Copy the current directory contents into the container at $HOME/app setting the owner to the user
COPY --chown=user . $HOME/app

# CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "7860"]
CMD ["uvicorn", "api-server-0226:app", "--host", "0.0.0.0", "--port", "7860"]
# uvicorn api-server-0226:app --host 0.0.0.0 --port 7860

Reproduction

ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes

a93dd0ab982d32397caef31a93a40b06

When I use auto train on Hugging Face and select the options as shown in the screenshot, I encounter an error. I noticed that most models trigger this error. Could it be related to the fact that I’m using a free [CPU?](log.txt)

Expected behavior

I hope the pros can help me figure out how to fix this error.

About this issue

Most upvoted comments

fwiw, downgrading to a lower version of transformers helped resolve the issue for me (4.38.2 to 4.31.0).

If you just installed the libraries such as pip install accelerate peft bitsandbytes transformers trl and running the Jupyter, you can try restart the kernel.

I had this on my local windows system and it was due to having the cpu version of pytorch, where a dependency of Accelerate requires the gpu version.

`import torch

print(torch.version)`

this should give something like 2.2.1+cu118 , if its says +cpu then you have the cpu version.