unsloth: [FIXED] Error patching SFTTrainer
I want to download Llama 3.2 Vision Instruct Model with this code but I take an error `from unsloth import FastVisionModel # FastLanguageModel for LLMs import torch
4bit pre quantized models we support for 4x faster downloading + no OOMs.
fourbit_models = [ “unsloth/Llama-3.2-11B-Vision-Instruct-bnb-4bit”, # Llama 3.2 vision support “unsloth/Llama-3.2-11B-Vision-bnb-4bit”, “unsloth/Llama-3.2-90B-Vision-Instruct-bnb-4bit”, # Can fit in a 80GB card! “unsloth/Llama-3.2-90B-Vision-bnb-4bit”,
"unsloth/Pixtral-12B-2409-bnb-4bit", # Pixtral fits in 16GB!
"unsloth/Pixtral-12B-Base-2409-bnb-4bit", # Pixtral base model
"unsloth/Qwen2-VL-2B-Instruct-bnb-4bit", # Qwen2 VL support
"unsloth/Qwen2-VL-7B-Instruct-bnb-4bit",
"unsloth/Qwen2-VL-72B-Instruct-bnb-4bit",
"unsloth/llava-v1.6-mistral-7b-hf-bnb-4bit", # Any Llava variant works!
"unsloth/llava-1.5-7b-hf-bnb-4bit",
] # More models at https://huggingface.co/unsloth
model, tokenizer = FastVisionModel.from_pretrained( “unsloth/Llama-3.2-11B-Vision-Instruct”, load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA. use_gradient_checkpointing = “unsloth”, # True or “unsloth” for long context )`
Error
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning. 🦥 Unsloth Zoo will now patch everything to make training faster!
SyntaxError Traceback (most recent call last) /usr/local/lib/python3.11/dist-packages/unsloth/tokenizer_utils.py in <module> 1060 try: -> 1061 exec(trainer_text, globals()) 1062 except:
SyntaxError: invalid syntax (<string>, line 4)
During handling of the above exception, another exception occurred:
RuntimeError Traceback (most recent call last) 5 frames /usr/local/lib/python3.11/dist-packages/unsloth/tokenizer_utils.py in <module> 1061 exec(trainer_text, globals()) 1062 except: -> 1063 raise RuntimeError(f"Unsloth: Please file a bug report! Error patching {trainer_name}“) 1064 exec(f"trl.trainer.{trainer_name} = Unsloth{trainer_name}”, globals()) 1065 pass
RuntimeError: Unsloth: Please file a bug report! Error patching SFTTrainer
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (3 by maintainers)
Apologies on the issue - I’m working on a fix asap