unsloth: TypeError: SFTTrainer.__init__() got an unexpected keyword argument 'dataset_text_field'
Hello there, when using the Google Colab. I reached this step:
from trl import SFTTrainer
from transformers import TrainingArguments, DataCollatorForSeq2Seq
from unsloth import is_bfloat16_supported
trainer = SFTTrainer(
model = model,
tokenizer = tokenizer,
train_dataset = dataset,
dataset_text_field = 'text',
max_seq_length = max_seq_length,
data_collator = DataCollatorForSeq2Seq(tokenizer = tokenizer),
dataset_num_proc = 2,
packing = False, # Can make training 5x faster for short sequences.
args = TrainingArguments(
per_device_train_batch_size = 2,
gradient_accumulation_steps = 4,
warmup_steps = 5,
# num_train_epochs = 1, # Set this for 1 full training run.
max_steps = 60,
learning_rate = 2e-4,
fp16 = not is_bfloat16_supported(),
bf16 = is_bfloat16_supported(),
logging_steps = 1,
optim = "adamw_8bit",
weight_decay = 0.01,
lr_scheduler_type = "linear",
seed = 3407,
output_dir = "outputs",
report_to = "none", # Use this for WandB etc
),
)
However, I get the following error:
TypeError: SFTTrainer.__init__() got an unexpected keyword argument 'dataset_text_field' . Is there any method of fixing this issue?
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 5
- Comments: 15 (1 by maintainers)
Hello, @officialsahyaboutorabi Fix issue by means of importing SFTConfig from trl, replacing TrainingArguments with it and moving all necessary parameters (mentioned there https://github.com/huggingface/trl/blob/main/trl/trainer/sft_config.py) out of SFTTrainer to SFTConfig.
So, now my cell with SFTTrainer looks like:
Use this version:
pip install trl==0.12.2
You must pass
dataset_text_fieldinSFTConfig, not inSFTTrainerBack to old versions is not a solution. Find changes.Also Read this in 0.12.0 release note:
Default dataset_text_field to “text”
Since many users use “text” as the column name for textual data in datasets, we’ve made it the default (previously a required argument) in SFTConfig. Now, specifying dataset_text_field=“text” is no longer necessary.
so, remove it if
dataset_text_field="text", otherwise move toSFTConfigThis is closed, but I will put what I’ve found, because I don’t understand why it works. For some reason, the order of the imports is very important. The following imports did not work and gave the exception mentioned by OP:
If I change the order to this:
No errors.
I’ll update Unsloth to add a
dataset_text_fieldoption allow for backwards compatibility - sorry on the issue everyone!@stepetal This fixed all of my issues, thank you so much!