mlx-examples: "missing 5 required positional arguments" when I try to train using lora on llama2
When I run :
python lora.py \
--model ./mlx_model \
--train \
--iters 600 \
--batch-size 1 \
--lora-layers 4
I get the following error:
Loading pretrained model
Traceback (most recent call last):
File "/Users/macstudiosct/projects/finetune_mlx/mlx-examples/lora/lora.py", line 297, in <module>
model, tokenizer, _ = models.load(args.model)
File "/Users/macstudiosct/projects/finetune_mlx/mlx-examples/lora/models.py", line 319, in load
model_args = ModelArgs.from_dict(config)
File "/Users/macstudiosct/projects/finetune_mlx/mlx-examples/lora/models.py", line 46, in from_dict
return cls(
TypeError: ModelArgs.__init__() missing 5 required positional arguments: 'hidden_size', 'num_hidden_layers', 'intermediate_size', 'num_attention_heads', and 'rms_norm_eps'
I have also tried the arguments in the order mentioned in python lora.py -h, no dice
System:
M2 ultra with 128gb memory; 76 cores GPU.
Edit:
My mlx version is 0.07
About this issue
- Original URL
- State: closed
- Created 6 months ago
- Comments: 19 (11 by maintainers)
Most of what you guys spoke about went right over my head 😃 But well, it just shows how far I have to go.
I will apply the temporary fix though and will research a bit to understand the long one.
Got it, thanks for the explanation! @Satyam7166-tech the fix suggested by @mzbac would get you going quickly (add
strict=Falsewhere we load the weights.For an actual fix I think it would be preferrable to pop those weights out of the model on the fly (for llama style models). I really like the strict weight loading as it saves a lot of time debugging 😃
Also fine-tuning a 70B is impressive! I hope that runs fast enough for you 🤞