unsloth: Qwen2.5-VL Fine-Tuning Error: Could not make a flat list of images
Hello. I’m trying to fine-tune Qwen2.5-VL and get the following error using this code:
%%capture
# Normally using pip install unsloth is enough
# Temporarily as of Jan 31st 2025, Colab has some issues with Pytorch
# Using pip install unsloth will take 3 minutes, whilst the below takes <1 minute:
!pip install --no-deps bitsandbytes accelerate xformers==0.0.29 peft trl triton
!pip install --no-deps cut_cross_entropy unsloth_zoo
!pip install sentencepiece protobuf datasets huggingface_hub hf_transfer
!pip install --no-deps unsloth
!python -m pip install git+https://github.com/huggingface/transformers
from unsloth import FastVisionModel # FastLanguageModel for LLMs
import torch
model, tokenizer = FastVisionModel.from_pretrained(
"yukiarimo/yuna-ai-v4-miru-base", # private
load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.
use_gradient_checkpointing = "unsloth", # True or "unsloth" for long context
max_seq_length = 2048,
)
model = FastVisionModel.get_peft_model(
model,
finetune_vision_layers = False, # False if not finetuning vision layers
finetune_language_layers = True, # False if not finetuning language layers
finetune_attention_modules = True, # False if not finetuning attention layers
finetune_mlp_modules = True, # False if not finetuning MLP layers
r = 16, # The larger, the higher the accuracy, but might overfit
lora_alpha = 16, # Recommended alpha == r at least
lora_dropout = 0.1,
bias = "lora_only", # "all" or "lora_only"
random_state = 42,
use_rslora = True, # We support rank stabilized LoRA
loftq_config = None, # And LoftQ
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj",
"embed_tokens", "lm_head"
],
)
from datasets import load_dataset
# Data prep
def formatting_prompts_func(examples):
texts = examples["text"]
return {"text": texts}
dataset = load_dataset("json", data_files="/content/drive/MyDrive/pack-3-all.jsonl") # raw text only
Trainer:
from unsloth import is_bf16_supported
from unsloth.trainer import UnslothVisionDataCollator
from trl import SFTTrainer, SFTConfig
FastVisionModel.for_training(model) # Enable for training!
trainer = SFTTrainer(
model = model,
tokenizer = tokenizer,
train_dataset=dataset["train"],
dataset_text_field="text",
packing=False,
args = SFTConfig(
per_device_train_batch_size = 1,
gradient_accumulation_steps = 4,
warmup_steps = 20,
max_steps = 30,
learning_rate = 1e-5,
fp16 = not is_bf16_supported(),
bf16 = is_bf16_supported(),
logging_steps = 1,
optim = "adamw_8bit",
weight_decay = 0.1,
lr_scheduler_type = "constant",
seed = 42,
output_dir = "outputs",
group_by_length=True,
report_to="tensorboard",
max_grad_norm=1.0,
gradient_checkpointing=True,
save_steps=10,
max_seq_length=2048
),
)
Error:
Map: 0%
0/30 [00:00<?, ? examples/s]
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
[<ipython-input-13-af15fe0115a5>](https://6ov7vzpx3ny-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab_20250206-060131_RC00_723872503#) in <cell line: 0>()
5 FastVisionModel.for_training(model) # Enable for training!
6
----> 7 trainer = SFTTrainer(
8 model = model,
9 tokenizer = tokenizer,
13 frames
[/usr/local/lib/python3.11/dist-packages/transformers/image_utils.py](https://6ov7vzpx3ny-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab_20250206-060131_RC00_723872503#) in make_flat_list_of_images(images)
264 return list(images)
265
--> 266 raise ValueError(f"Could not make a flat list of images from {images}")
267
268
ValueError: Could not make a flat list of images from ['<|begin_of_text|>\n<dialog>\n<yuki>Hey, Yuna. You can see the following text [dataset text continues] ...
How to fix this? No images in the dataset!
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 26 (1 by maintainers)
+1 same error here.
I successfully started the training! Thank you, @nickmitchko! You are the best (hopefully, everything will work after the burned GPU).
On LLaMa 3.1 the built in
padfunction of the tokenizer does that job fine without changing.To conclude, the library should handle the no-image issue here to prevent the problems with these steps.