unsloth: Qwen2.5-VL Fine-Tuning Error: Could not make a flat list of images

Hello. I’m trying to fine-tune Qwen2.5-VL and get the following error using this code:

%%capture
# Normally using pip install unsloth is enough

# Temporarily as of Jan 31st 2025, Colab has some issues with Pytorch
# Using pip install unsloth will take 3 minutes, whilst the below takes <1 minute:
!pip install --no-deps bitsandbytes accelerate xformers==0.0.29 peft trl triton
!pip install --no-deps cut_cross_entropy unsloth_zoo
!pip install sentencepiece protobuf datasets huggingface_hub hf_transfer
!pip install --no-deps unsloth
!python -m pip install git+https://github.com/huggingface/transformers
from unsloth import FastVisionModel # FastLanguageModel for LLMs
import torch

model, tokenizer = FastVisionModel.from_pretrained(
    "yukiarimo/yuna-ai-v4-miru-base", # private
    load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.
    use_gradient_checkpointing = "unsloth", # True or "unsloth" for long context
    max_seq_length = 2048,
)
model = FastVisionModel.get_peft_model(
    model,
    finetune_vision_layers     = False, # False if not finetuning vision layers
    finetune_language_layers   = True, # False if not finetuning language layers
    finetune_attention_modules = True, # False if not finetuning attention layers
    finetune_mlp_modules       = True, # False if not finetuning MLP layers

    r = 16,           # The larger, the higher the accuracy, but might overfit
    lora_alpha = 16,  # Recommended alpha == r at least
    lora_dropout = 0.1,
    bias = "lora_only", # "all" or "lora_only"
    random_state = 42,
    use_rslora = True,  # We support rank stabilized LoRA
    loftq_config = None, # And LoftQ
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",
                      "embed_tokens", "lm_head"
                      ],
)
from datasets import load_dataset

# Data prep
def formatting_prompts_func(examples):
    texts = examples["text"]
    return {"text": texts}

dataset = load_dataset("json", data_files="/content/drive/MyDrive/pack-3-all.jsonl") # raw text only

Trainer:

from unsloth import is_bf16_supported
from unsloth.trainer import UnslothVisionDataCollator
from trl import SFTTrainer, SFTConfig

FastVisionModel.for_training(model) # Enable for training!

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset=dataset["train"],
    dataset_text_field="text",
    packing=False,
    args = SFTConfig(
        per_device_train_batch_size = 1,
        gradient_accumulation_steps = 4,
        warmup_steps = 20,
        max_steps = 30,
        learning_rate = 1e-5,
        fp16 = not is_bf16_supported(),
        bf16 = is_bf16_supported(),
        logging_steps = 1,
        optim = "adamw_8bit",
        weight_decay = 0.1,
        lr_scheduler_type = "constant",
        seed = 42,
        output_dir = "outputs",
        group_by_length=True,
        report_to="tensorboard",
        max_grad_norm=1.0,
        gradient_checkpointing=True,
        save_steps=10,
        max_seq_length=2048
    ),
)

Error:

Map:   0%
 0/30 [00:00<?, ? examples/s]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-13-af15fe0115a5>](https://6ov7vzpx3ny-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab_20250206-060131_RC00_723872503#) in <cell line: 0>()
      5 FastVisionModel.for_training(model) # Enable for training!
      6 
----> 7 trainer = SFTTrainer(
      8     model = model,
      9     tokenizer = tokenizer,

13 frames
[/usr/local/lib/python3.11/dist-packages/transformers/image_utils.py](https://6ov7vzpx3ny-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab_20250206-060131_RC00_723872503#) in make_flat_list_of_images(images)
    264             return list(images)
    265 
--> 266     raise ValueError(f"Could not make a flat list of images from {images}")
    267 
    268
ValueError: Could not make a flat list of images from ['<|begin_of_text|>\n<dialog>\n<yuki>Hey, Yuna. You can see the following text [dataset text continues] ...

How to fix this? No images in the dataset!

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 26 (1 by maintainers)

Most upvoted comments

+1 same error here.

I successfully started the training! Thank you, @nickmitchko! You are the best (hopefully, everything will work after the burned GPU).

On LLaMa 3.1 the built in pad function of the tokenizer does that job fine without changing.

To conclude, the library should handle the no-image issue here to prevent the problems with these steps.