unsloth: [FIXED] `Qwen2VL` finetuning broken
Installing latest version of unsloth:
!pip uninstall unsloth -y && pip install --upgrade --no-cache-dir --no-deps git+https://github.com/unslothai/unsloth.git
breaks the Qwen2 7B Vision Colab it seems(?)
from unsloth import FastVisionModel # FastLanguageModel for LLMs
import torch
# 4bit pre quantized models we support for 4x faster downloading + no OOMs.
fourbit_models = [
"unsloth/Llama-3.2-11B-Vision-Instruct-bnb-4bit", # Llama 3.2 vision support
"unsloth/Llama-3.2-11B-Vision-bnb-4bit",
"unsloth/Llama-3.2-90B-Vision-Instruct-bnb-4bit", # Can fit in a 80GB card!
"unsloth/Llama-3.2-90B-Vision-bnb-4bit",
"unsloth/Pixtral-12B-2409-bnb-4bit", # Pixtral fits in 16GB!
"unsloth/Pixtral-12B-Base-2409-bnb-4bit", # Pixtral base model
"unsloth/Qwen2-VL-2B-Instruct-bnb-4bit", # Qwen2 VL support
"unsloth/Qwen2-VL-7B-Instruct-bnb-4bit",
"unsloth/Qwen2-VL-72B-Instruct-bnb-4bit",
"unsloth/llava-v1.6-mistral-7b-hf-bnb-4bit", # Any Llava variant works!
"unsloth/llava-1.5-7b-hf-bnb-4bit",
] # More models at https://huggingface.co/unsloth
model, tokenizer = FastVisionModel.from_pretrained(
"unsloth/Qwen2-VL-7B-Instruct",
load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.
use_gradient_checkpointing = "unsloth", # True or "unsloth" for long context
)
Leading to:
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
🦥 Unsloth Zoo will now patch everything to make training faster!
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
[<ipython-input-2-dd19b029c094>](https://localhost:8080/#) in <cell line: 22>()
20 ] # More models at https://huggingface.co/unsloth
21
---> 22 model, tokenizer = FastVisionModel.from_pretrained(
23 "unsloth/Qwen2-VL-7B-Instruct",
24 load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.
3 frames
[/usr/local/lib/python3.10/dist-packages/unsloth_zoo/compiler.py](https://localhost:8080/#) in patch_gradient_accumulation(modeling_file, module)
789 functions = dir(modeling_file)
790 module = eval(f"modeling_file.{module}")
--> 791 forward = module.forward
792 source = inspect.getsource(forward)
793 has_kwargs = tuple(inspect.signature(forward).parameters.values())[-1].kind == inspect._VAR_KEYWORD
AttributeError: type object 'Qwen2VLCausalLMOutputWithPast' has no attribute 'forward'
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 24 (5 by maintainers)
use
pip install "unsloth==2024.12.11"until the problem solvedHi I am seeing the same error in qwen2.5 vl…
My current hack: https://github.com/fzyzcjy/unsloth-zoo/commit/d2372ca5fc3bcb3ea41b6473c3b7d36de5265c55
Hope unsloth fix it later!
Hopefully I finally fixed it! Would appreciate it if anyone could check - thanks a lot!
For local machines, please update Unsloth via:
Colab / Kaggle should just restart
The latest version(=
2024.12.7) of ‘usloth-zoo’ has an issue.pip install "unsloth==2024.12.11"pip install "unsloth-zoo==2024.12.6"