unsloth: [FIXED] RuntimeError: Unsloth: Your repo has a LoRA adapter and a base model.
I’ve trained the unsloth/Llama-3.2-3B-Instruct-bnb-4bit model successfully, but when I try to use it with astLanguageModel.from_pretrained, I get this error:
Traceback (most recent call last):
File "/home/denis/Documents/ai/kubegen/server.py.gpu.unsloth.chat_template", line 17, in <module>
model, tokenizer = FastLanguageModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/denis/miniconda3/envs/unsloth_env/lib/python3.11/site-packages/unsloth/models/loader.py", line 196, in from_pretrained
raise RuntimeError(
RuntimeError: Unsloth: Your repo has a LoRA adapter and a base model.
You have 2 files `config.json` and `adapter_config.json`.
We must only allow one config file.
Please separate the LoRA and base models to 2 repos.
It’s weird because there’s no config.json file:
-rw-rw-r-- 1 denis denis 738 Sep 26 09:28 adapter_config.json
-rw-rw-r-- 1 denis denis 97307544 Sep 26 09:28 adapter_model.safetensors
-rw-rw-r-- 1 denis denis 5112 Sep 26 09:28 README.md
-rw-rw-r-- 1 denis denis 454 Sep 26 09:28 special_tokens_map.json
-rw-rw-r-- 1 denis denis 51752 Sep 26 09:28 tokenizer_config.json
-rw-rw-r-- 1 denis denis 17209920 Sep 26 09:28 tokenizer.json
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 2
- Comments: 34 (7 by maintainers)
Much much apologies just found the issue! It seems like transformers 4.45 is now using a weird way to detect if a LoRA was trained or not. I fixed it now! Please update Unsloth on local machines via:
No need to retrain LoRAs! Also do NOT downgrade transformers - use 4.45.1 or higher
For Colab and Kaggle, no need to update - just restart and delete runtime.
Sorry on the issue again! @razzaghi @djannot @ibadrehman90 @kholiavko-roman @BenBatsir @aksh-n @vietphuon @kiranpedvak
@vietphuon I also tried
vietphuon/Llama-3.2-1B-Instruct-bnb-4bit-240926-1- it should function now as well!Apologies everyone and thank you for your patience!
Ok will investigate asap - sorry for the issue!
Yes it seems like its a transformers issue - temporarily please use transformers<4.45 via
pip install --upgrade --no-cache-dir "transformers<4.45.0"- I’ll try seeing what the issue is - sorry guys!Still facing this with:
Tried downgrading transformers to 4.44.2, it gives me
No module named 'transformers.models.granite'@danielhanchen what’s the fix?I did do a
model.save_pretrained_merged(new_model_local, tokenizer, save_method = "merged_16bit",)after finetuningIt works like a charm !
Thanks a lot @danielhanchen for your reactivity
Apologies everyone!! Will get to the bottom of this today - thank you so much for your patience and sorry on the delay!!
The issue might have to do something with
transformersversion. The issue surfaces when I use transformers version 4.45.0 (which was released yesterday), but not when I use 4.44.2 (the default version colab currently uses, and the one before 4.45.0). maybe the new versions of unsloth lead us to install the new version of transformers, causing the issue?Thanks, it’s working now @danielhanchen
Thank you so much, @danielhanchen
Any update? Tried downgrading
transformersbut it didn’t do the trick. I wonder if I can simply comment out the RuntimeError on loader.py@vietphuon I didn’t have any issue with the 1B. It’s only with the 3B.
@danielhanchen should I train the 3B model again after downgrading to 4.44.2 ?
+1. I’m certain this issue was caused by something committed in the last 24 hours. I went back to the last commit on Sep 23 fb77505f8429566f5d21d6ea5318c342e8a67991, and there is no error.