True. !pip install transformers==4.28.0 for previous version is easier solution. The newer version runs into dependency issues.

Hi @creek411 install version 4.28.0 of transformers by running this code !pip install transformers==4.28.0. Then restart and run all the code( if ur using colab).

Please also remember to restart the kernel ( Given you are using Colab/Jupyter ) ( I know it is silly but yes )

Bro, you don’t need to worry too much. Please downgrade the version. They are having stable version. Don’t stress too much. Previous version working as usual. We changed all our requirements today. Hectic process 😦

@RAravindDS Thanks for reporting. I suspect the issue is coming from the version of accelerate in your environment. Could you:

• Share the running environment info: copy-paste the output from running transformers-cli env in your terminal THEN
• Upgrade accelerate using pip install --upgrade accelerate
• Retry

I’m using Jupyter (as well as the VS Code notebooks extension, which is essentially the same) on Python 3.11 with no venv and the interpreter provided by asdf.

On re-test, accelerate 0.19 did work with transformers 4.29, as it turned out; I’m just not accustomed to notebooks and forgot that I needed to restart the kernel to freshen the dependencies. Classic n00b mistake.

I’m still a bit mystified as to why I had an older accelerate, as I had created my entire Python environment on the same day I commented. Possibly, it was a transitive dependency of something else I’d already installed.

@creek411 the solution would be to do pip install accelerate (and as now we have a release, it works OOTB with the normal pypi install), however the fact you have the error means you still probably are installing from dev and there’s some cache working in there. You can try pip uninstall transformers -y, run your code, make sure it fails because transformers isn’t installed, then install transformers again, either 4.28.0 or 4.29.0 and do pip install accelerate as well

For those having issues, can you tell me more about if you are working in Jupyter, Colab, or in regular Python? Again the solution hasn’t changed: in the correct environment you need to make sure that accelerate is installed and viewable. To test this in your environment you can try importing it import accelerate. If it fails, it’s not installed correctly.

This worked for me in colab, but it is important to restart the execution environment

!pip uninstall -y -r transformers accelerate !pip install transformers==4.29.0 !pip install git+https://github.com/huggingface/accelerate

Gracias amigo

Esto me funciono en colab, pero es importante reiniciar el entorno de ejecución

!pip uninstall -y -r transformers accelerate !pip install transformers==4.29.0 !pip install git+https://github.com/huggingface/accelerate

@creek411 the solution would be to do pip install accelerate (and as now we have a release, it works OOTB with the normal pypi install), however the fact you have the error means you still probably are installing from dev and there’s some cache working in there. You can try pip uninstall transformers -y, run your code, make sure it fails because transformers isn’t installed, then install transformers again, either 4.28.0 or 4.29.0 and do pip install accelerate as well

I get the same error with

Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (0.19.0)
Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (4.29.1)

on Colab

I had to install accelerate manually.

!pip install torch "argilla" datasets accelerate transformers setfit

I attempted to do your solution and could avoid the error. I appreciate for your advise.

Thank you for your reply. I tried to install 4.28.0 and run the code. However, this error recurred. In this code, I install and use transformers datasets. So should I install transformers datasets of previsous version?

I am using the following version of transformer, datasets and huggingface_hub.

I am running into the following error:

 NameError: name 'PartialState' is not defined.

How to resolve this issue to work with my versions of the transformer, datasets and huggingface_hub ?

@RAravindDS Yes, this is because the PartialState import was added as a dependency on the transformers development branch yesterday. PartialState was added in the 0.17.0 release in accelerate, and so for the development branch of transformers, accelerate >= 0.17.0 is required.

Downgrading the transformers version removes the code which is importing PartialState.