bitsandbytes: ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`
System Info
# FROM python:3.9
# WORKDIR /code
# COPY ./requirements.txt /code/requirements.txt
# RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
# COPY . .
# # RUN pip install --no-cache-dir --upgrade -r /requirements.txt
# # uvicorn reward_modeling:app --host 0.0.0.0 --port 6006 --reload
# CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "6006"]
# # CMD ["./my-test-shell.sh"]
# Use the official Python 3.9 image
FROM python:3.10
# Set the working directory to /code
WORKDIR /code
# Copy the current directory contents into the container at /code
COPY ./requirements.txt /code/requirements.txt
RUN pip install -i https://pypi.org/simple/ bitsandbytes --upgrade
# Install requirements.txt
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
# Set up a new user named "user" with user ID 1000
RUN useradd -m -u 1000 user
# Switch to the "user" user
USER user
# Set home to the user's home directory
ENV HOME=/home/user \
PATH=/home/user/.local/bin:$PATH
# Set the working directory to the user's home directory
WORKDIR $HOME/app
# Copy the current directory contents into the container at $HOME/app setting the owner to the user
COPY --chown=user . $HOME/app
# CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "7860"]
CMD ["uvicorn", "api-server-0226:app", "--host", "0.0.0.0", "--port", "7860"]
# uvicorn api-server-0226:app --host 0.0.0.0 --port 7860
Reproduction
ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes
When I use auto train on Hugging Face and select the options as shown in the screenshot, I encounter an error. I noticed that most models trigger this error. Could it be related to the fact that I’m using a free [CPU?](log.txt)
Expected behavior
I hope the pros can help me figure out how to fix this error.
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 18
fwiw, downgrading to a lower version of transformers helped resolve the issue for me (4.38.2 to 4.31.0).
If you just installed the libraries such as
pip install accelerate peft bitsandbytes transformers trland running the Jupyter, you can try restart the kernel.I had this on my local windows system and it was due to having the cpu version of pytorch, where a dependency of Accelerate requires the gpu version.
`import torch
print(torch.version)`
this should give something like 2.2.1+cu118 , if its says +cpu then you have the cpu version.