unsloth: NOTHING IS WORKING AT ALL!!!!!!! HELP!!!! AttributeError: 'NoneType' object has no attribute 'startswith'

Hello! I referenced this: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen2_VL_(7B)-Vision.ipynb#scrollTo=2ejIt2xSNKKp

So, here’s the code:

Loading the model and shit:

!pip install --no-deps bitsandbytes accelerate xformers==0.0.29 peft trl triton
!pip install --no-deps cut_cross_entropy unsloth_zoo
!pip install sentencepiece protobuf datasets huggingface_hub hf_transfer
!pip install --no-deps unsloth
!python -m pip install git+https://github.com/huggingface/transformers

from unsloth import FastVisionModel # FastLanguageModel for LLMs
import torch
from PIL import Image
import os

model, tokenizer = FastVisionModel.from_pretrained(
    "yukiarimo/yuna-ai-v4-miru-base",
    load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.
    use_gradient_checkpointing = "unsloth", # True or "unsloth" for long context
    max_seq_length = 2048,
)

Fix shit so I can pass raw text:

chat_template = "{instruction}"

# Set the chat template for the tokenizer
tokenizer.chat_template = chat_template

lora:

model = FastVisionModel.get_peft_model(
    model,
    finetune_vision_layers     = True, # False if not finetuning vision layers
    finetune_language_layers   = True, # False if not finetuning language layers
    finetune_attention_modules = True, # False if not finetuning attention layers
    finetune_mlp_modules       = True, # False if not finetuning MLP layers

    r = 16,           # The larger, the higher the accuracy, but might overfit
    lora_alpha = 16,  # Recommended alpha == r at least
    lora_dropout = 0.1,
    bias = "lora_only", # "all" or "lora_only"
    random_state = 42,
    use_rslora = True,  # We support rank stabilized LoRA
    loftq_config = None, # And LoftQ
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",
                      "embed_tokens", "lm_head"
                      ],
)

Force image (dummy) into a dataset because idfc what it is, just wanna train on raw text:

from datasets import load_dataset
from PIL import Image

# Load the image from file
yuna_image = Image.open("/content/yuna.png")

def add_yuna_image(example):
    # Create a nested structure with "messages" key
    example["messages"] = [
        {"role": "user", "content": [
            {"type": "image", "image": "file://yuna.png"}, # Assuming image is already loaded
            {"type": "text", "text": example["text"]} # Accessing text from example
        ]}
    ]
    return example

dataset = load_dataset("json", data_files="/content/drive/MyDrive/pack-3-all.jsonl")
dataset = dataset.map(add_yuna_image)

(tried with just PIL)

trainer:

from unsloth import is_bf16_supported
from unsloth.trainer import UnslothVisionDataCollator
from trl import SFTTrainer, SFTConfig

FastVisionModel.for_training(model) # Enable for training!

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset=dataset["train"],
    dataset_text_field="text",
    packing=False,
    args = SFTConfig(
        per_device_train_batch_size = 1,
        gradient_accumulation_steps = 4,
        warmup_steps = 2,
        max_steps = 100,
        learning_rate = 1e-5,
        fp16 = not is_bf16_supported(),
        bf16 = is_bf16_supported(),
        logging_steps = 1,
        optim = "adamw_8bit",
        weight_decay = 0.1,
        lr_scheduler_type = "constant",
        seed = 42,
        output_dir = "outputs",
        group_by_length=False,
        report_to="tensorboard",
        max_grad_norm=1.0,
        gradient_checkpointing=True,
        save_steps=10,
        max_seq_length=2048,
        dataset_text_field = "",
        #dataset_num_proc = 4,
        remove_unused_columns = False, # Ensure this is False
        dataset_kwargs = {"skip_prepare_dataset": True}, # Add this

    ),
    data_collator = UnslothVisionDataCollator(model, tokenizer), # Must use!
)

trainer_stats = trainer.train()

And???

Error:

[/usr/local/lib/python3.11/dist-packages/unsloth_zoo/vision_utils.py](https://duhsf5ybc2j-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab_20250210-060225_RC00_725142777#) in fetch_image(ele, size_factor)
    123     if isinstance(image, Image.Image):
    124         image_obj = image
--> 125     elif image.startswith("http://") or image.startswith("https://"):
    126         image_obj = Image.open(requests.get(image, stream=True).raw)
    127     elif image.startswith("file://"):

AttributeError: 'NoneType' object has no attribute 'startswith'

FOR NO REASON! I tried giving the wrong path and it says that cannot find, but if the path is current it cannot parse or wtf is that??? No text, no vision fine tuning. It just doesn’t work!

Please help!

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 21

Most upvoted comments

Apologies did you update transformers and Unsloth-zoo to the latest version because Qwen 2.5 vl only works with that. Also we do know there’s currently an issue with vision models so we’ll update you if we fix it thank you!

Not yet. Waiting to be resolved