unsloth: NOTHING IS WORKING AT ALL!!!!!!! HELP!!!! AttributeError: 'NoneType' object has no attribute 'startswith'
Hello! I referenced this: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen2_VL_(7B)-Vision.ipynb#scrollTo=2ejIt2xSNKKp
So, here’s the code:
Loading the model and shit:
!pip install --no-deps bitsandbytes accelerate xformers==0.0.29 peft trl triton
!pip install --no-deps cut_cross_entropy unsloth_zoo
!pip install sentencepiece protobuf datasets huggingface_hub hf_transfer
!pip install --no-deps unsloth
!python -m pip install git+https://github.com/huggingface/transformers
from unsloth import FastVisionModel # FastLanguageModel for LLMs
import torch
from PIL import Image
import os
model, tokenizer = FastVisionModel.from_pretrained(
"yukiarimo/yuna-ai-v4-miru-base",
load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.
use_gradient_checkpointing = "unsloth", # True or "unsloth" for long context
max_seq_length = 2048,
)
Fix shit so I can pass raw text:
chat_template = "{instruction}"
# Set the chat template for the tokenizer
tokenizer.chat_template = chat_template
lora:
model = FastVisionModel.get_peft_model(
model,
finetune_vision_layers = True, # False if not finetuning vision layers
finetune_language_layers = True, # False if not finetuning language layers
finetune_attention_modules = True, # False if not finetuning attention layers
finetune_mlp_modules = True, # False if not finetuning MLP layers
r = 16, # The larger, the higher the accuracy, but might overfit
lora_alpha = 16, # Recommended alpha == r at least
lora_dropout = 0.1,
bias = "lora_only", # "all" or "lora_only"
random_state = 42,
use_rslora = True, # We support rank stabilized LoRA
loftq_config = None, # And LoftQ
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj",
"embed_tokens", "lm_head"
],
)
Force image (dummy) into a dataset because idfc what it is, just wanna train on raw text:
from datasets import load_dataset
from PIL import Image
# Load the image from file
yuna_image = Image.open("/content/yuna.png")
def add_yuna_image(example):
# Create a nested structure with "messages" key
example["messages"] = [
{"role": "user", "content": [
{"type": "image", "image": "file://yuna.png"}, # Assuming image is already loaded
{"type": "text", "text": example["text"]} # Accessing text from example
]}
]
return example
dataset = load_dataset("json", data_files="/content/drive/MyDrive/pack-3-all.jsonl")
dataset = dataset.map(add_yuna_image)
(tried with just PIL)
trainer:
from unsloth import is_bf16_supported
from unsloth.trainer import UnslothVisionDataCollator
from trl import SFTTrainer, SFTConfig
FastVisionModel.for_training(model) # Enable for training!
trainer = SFTTrainer(
model = model,
tokenizer = tokenizer,
train_dataset=dataset["train"],
dataset_text_field="text",
packing=False,
args = SFTConfig(
per_device_train_batch_size = 1,
gradient_accumulation_steps = 4,
warmup_steps = 2,
max_steps = 100,
learning_rate = 1e-5,
fp16 = not is_bf16_supported(),
bf16 = is_bf16_supported(),
logging_steps = 1,
optim = "adamw_8bit",
weight_decay = 0.1,
lr_scheduler_type = "constant",
seed = 42,
output_dir = "outputs",
group_by_length=False,
report_to="tensorboard",
max_grad_norm=1.0,
gradient_checkpointing=True,
save_steps=10,
max_seq_length=2048,
dataset_text_field = "",
#dataset_num_proc = 4,
remove_unused_columns = False, # Ensure this is False
dataset_kwargs = {"skip_prepare_dataset": True}, # Add this
),
data_collator = UnslothVisionDataCollator(model, tokenizer), # Must use!
)
trainer_stats = trainer.train()
And???
Error:
[/usr/local/lib/python3.11/dist-packages/unsloth_zoo/vision_utils.py](https://duhsf5ybc2j-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab_20250210-060225_RC00_725142777#) in fetch_image(ele, size_factor)
123 if isinstance(image, Image.Image):
124 image_obj = image
--> 125 elif image.startswith("http://") or image.startswith("https://"):
126 image_obj = Image.open(requests.get(image, stream=True).raw)
127 elif image.startswith("file://"):
AttributeError: 'NoneType' object has no attribute 'startswith'
FOR NO REASON! I tried giving the wrong path and it says that cannot find, but if the path is current it cannot parse or wtf is that??? No text, no vision fine tuning. It just doesn’t work!
Please help!
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 21
Apologies did you update transformers and Unsloth-zoo to the latest version because Qwen 2.5 vl only works with that. Also we do know there’s currently an issue with vision models so we’ll update you if we fix it thank you!
Not yet. Waiting to be resolved