adapters: ImportError: cannot import name 'AutoModelWithHeads' from 'transformers'
Hi I am trying with this example colab: https://colab.research.google.com/github/Adapter-Hub/website/blob/master/app/static/notebooks/Adapter_Quickstart_Training.ipynb#scrollTo=Lbwb3NRf8mBF
getting this error:
Traceback (most recent call last):
File "test.py", line 11, in <module>
from transformers import AutoTokenizer, EvalPrediction, GlueDataset, GlueDataTrainingArguments, AutoModelWithHeads, AdapterType
ImportError: cannot import name 'AutoModelWithHeads' from 'transformers' (/idiap/user/rkarimi/libs/anaconda3/envs/adapter/lib/python3.7/site-packages/transformers/__init__.py)
versions
(adapter) rkarimi@italix17:/idiap/user/rkarimi/dev/internship/seq2seq/adapter-transformers$ conda list | grep transformers
adapter-transformers 1.0.1 <pip>
transformers 3.5.1 <pip>
(adapter) rkarimi@italix17:/idiap/user/rkarimi/dev/internship/seq2seq/adapter-transformers$ conda list | grep pytorch
pytorch-lightning 1.0.4 <pip>
adapter hub from github is installed
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 17 (7 by maintainers)
There’s no standard way to integrate adapters into custom models. If our implementations for bert-based models don’t work for you, please have a look at our implementation to get an idea how it might be adapted to your needs. Here are the important parts of the implementation:
modeling_bert.pyhas a corresponding mixin here. When supporting a custom model, you probably want to take this file and make changes to fit your model.modeling_bert.py), we only implement the mixins. No other changes are done here compared to Huggingface.Those are all the main classes where our changes are implemented.
Unfortunately, we can’t give detailed support for implementing custom models. If your model is part of the Huggingface repo, you can open a feature request so we can potentially support it officially. Otherwise, we can try to help out if you have any further specific questions to our implementation.
The pip
adapter-transformers1.0.1 is built ontransformers2.11.0 link However, the master branch ofadapter-transformersis already updated totransformers3.5.1 You can runpip install git+https://github.com/Adapter-Hub/adapter-transformers.gitto get that version.