unsloth: [FIXED] RuntimeError: Unsloth: Your repo has a LoRA adapter and a base model.

I’ve trained the unsloth/Llama-3.2-3B-Instruct-bnb-4bit model successfully, but when I try to use it with astLanguageModel.from_pretrained, I get this error:

Traceback (most recent call last):
  File "/home/denis/Documents/ai/kubegen/server.py.gpu.unsloth.chat_template", line 17, in <module>
    model, tokenizer = FastLanguageModel.from_pretrained(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/denis/miniconda3/envs/unsloth_env/lib/python3.11/site-packages/unsloth/models/loader.py", line 196, in from_pretrained
    raise RuntimeError(
RuntimeError: Unsloth: Your repo has a LoRA adapter and a base model.
You have 2 files `config.json` and `adapter_config.json`.
We must only allow one config file.
Please separate the LoRA and base models to 2 repos.

It’s weird because there’s no config.json file:

-rw-rw-r-- 1 denis denis      738 Sep 26 09:28 adapter_config.json
-rw-rw-r-- 1 denis denis 97307544 Sep 26 09:28 adapter_model.safetensors
-rw-rw-r-- 1 denis denis     5112 Sep 26 09:28 README.md
-rw-rw-r-- 1 denis denis      454 Sep 26 09:28 special_tokens_map.json
-rw-rw-r-- 1 denis denis    51752 Sep 26 09:28 tokenizer_config.json
-rw-rw-r-- 1 denis denis 17209920 Sep 26 09:28 tokenizer.json

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 34 (7 by maintainers)

Most upvoted comments

Much much apologies just found the issue! It seems like transformers 4.45 is now using a weird way to detect if a LoRA was trained or not. I fixed it now! Please update Unsloth on local machines via:

No need to retrain LoRAs! Also do NOT downgrade transformers - use 4.45.1 or higher

pip uninstall unsloth -y && pip install --upgrade --no-cache-dir "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"

For Colab and Kaggle, no need to update - just restart and delete runtime.

Sorry on the issue again! @razzaghi @djannot @ibadrehman90 @kholiavko-roman @BenBatsir @aksh-n @vietphuon @kiranpedvak

@vietphuon I also tried vietphuon/Llama-3.2-1B-Instruct-bnb-4bit-240926-1 - it should function now as well!

Apologies everyone and thank you for your patience!

Ok will investigate asap - sorry for the issue!

Yes it seems like its a transformers issue - temporarily please use transformers<4.45 via pip install --upgrade --no-cache-dir "transformers<4.45.0" - I’ll try seeing what the issue is - sorry guys!

Still facing this with:

tokenizers==0.21.0
transformers==4.48.2
unsloth @ git+https://github.com/unslothai/unsloth.git@038e6d4c8d40207a87297ab3aaf787c19b1006d1
unsloth_zoo==2025.1.5

Tried downgrading transformers to 4.44.2, it gives me No module named 'transformers.models.granite' @danielhanchen what’s the fix?

I did do a model.save_pretrained_merged(new_model_local, tokenizer, save_method = "merged_16bit",) after finetuning

It works like a charm !

Thanks a lot @danielhanchen for your reactivity

Apologies everyone!! Will get to the bottom of this today - thank you so much for your patience and sorry on the delay!!

The issue might have to do something with transformers version. The issue surfaces when I use transformers version 4.45.0 (which was released yesterday), but not when I use 4.44.2 (the default version colab currently uses, and the one before 4.45.0). maybe the new versions of unsloth lead us to install the new version of transformers, causing the issue?

Thanks, it’s working now @danielhanchen

Thank you so much, @danielhanchen

Any update? Tried downgrading transformers but it didn’t do the trick. I wonder if I can simply comment out the RuntimeError on loader.py

@vietphuon I didn’t have any issue with the 1B. It’s only with the 3B.

@danielhanchen should I train the 3B model again after downgrading to 4.44.2 ?

image So if I’m loading a LoRA adapter from the HF repo, this line should throw an exception and yield is_model = False right? I think next to investigate why this version of transformers’s AutoConfig ran.

+1. I’m certain this issue was caused by something committed in the last 24 hours. I went back to the last commit on Sep 23 fb77505f8429566f5d21d6ea5318c342e8a67991, and there is no error.