mlx-examples: "missing 5 required positional arguments" when I try to train using lora on llama2

When I run :

python lora.py \
   --model ./mlx_model \
   --train \
   --iters 600 \
   --batch-size 1 \
   --lora-layers 4

I get the following error:

Loading pretrained model
Traceback (most recent call last):
  File "/Users/macstudiosct/projects/finetune_mlx/mlx-examples/lora/lora.py", line 297, in <module>
    model, tokenizer, _ = models.load(args.model)
  File "/Users/macstudiosct/projects/finetune_mlx/mlx-examples/lora/models.py", line 319, in load
    model_args = ModelArgs.from_dict(config)
  File "/Users/macstudiosct/projects/finetune_mlx/mlx-examples/lora/models.py", line 46, in from_dict
    return cls(
TypeError: ModelArgs.__init__() missing 5 required positional arguments: 'hidden_size', 'num_hidden_layers', 'intermediate_size', 'num_attention_heads', and 'rms_norm_eps'

I have also tried the arguments in the order mentioned in python lora.py -h, no dice

System:

M2 ultra with 128gb memory; 76 cores GPU.

Edit:

My mlx version is 0.07

About this issue

  • Original URL
  • State: closed
  • Created 6 months ago
  • Comments: 19 (11 by maintainers)

Most upvoted comments

Most of what you guys spoke about went right over my head 😃 But well, it just shows how far I have to go.

I will apply the temporary fix though and will research a bit to understand the long one.

A quick workaround would be disable the restrict load weight.

Got it, thanks for the explanation! @Satyam7166-tech the fix suggested by @mzbac would get you going quickly (add strict=False where we load the weights.

For an actual fix I think it would be preferrable to pop those weights out of the model on the fly (for llama style models). I really like the strict weight loading as it saves a lot of time debugging 😃

Also fine-tuning a 70B is impressive! I hope that runs fast enough for you 🤞