unsloth: OSError: does not appear to have a file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt or flax_model.msgpack.

Hello, I m following the code in “Llama 3.2 Vision Finetuning Unsloth - Radiography.ipynb” for finetuning on my dataset. but after pushing the merged model to hugging face I got the error when trying to load it:

/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py:3491: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
  warnings.warn(
Unused kwargs: ['quant_method']. These kwargs are not used in <class 'transformers.utils.quantization_config.BitsAndBytesConfig'>.
`low_cpu_mem_usage` was None, now default to True since model is quantized.
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
[<ipython-input-2-01d76ccf350d>](https://localhost:8080/#) in <cell line: 4>()
      2 from transformers import MllamaForConditionalGeneration
      3 
----> 4 model, tokenizer = MllamaForConditionalGeneration.from_pretrained("Hessa/llama3.2_10k", use_auth_token="hf_***", torch_dtype=torch.float16) #torch_dtype=torch.bfloat16

[/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py](https://localhost:8080/#) in from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, weights_only, *model_args, **kwargs)
   3927                                 )
   3928                             else:
-> 3929                                 raise EnvironmentError(
   3930                                     f"{pretrained_model_name_or_path} does not appear to have a file named"
   3931                                     f" {_add_variant(WEIGHTS_NAME, variant)}, {_add_variant(SAFE_WEIGHTS_NAME, variant)},"

OSError: Hessa/llama3.2_10k does not appear to have a file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt or flax_model.msgpack.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (4 by maintainers)

Most upvoted comments

@hessaAlawwad @nazarenodefrancesc @pranav-upd @TheWelcomer @adamtcdev @void-mckenzie @FloLey @mosama1994 @garg-aayush

we merged a PR yesterday. This should be solved now. You might need to re-merge and repush your models to HF though using

push_to_hub_merged()

or add the model.safetensors.index.json manually to your current repo if you don’t want to re-merge and push. however the behavior of both

push_to_hub_merged()

and

save_pretrained_merged()

has now been resolved. Make sure to update your unsloth installation using:

pip install --force-reinstall git+https://github.com/unslothai/unsloth-zoo.git

and

pip install --force-reinstall git+https://github.com/unslothai/unsloth.git

Will now close this as complete. Feel free to reopen if you encounter problems.

Same issue. I finetuned a VLM and pushed it to hub, can’t load it back now. Major Major bug 😕

I’ve also had this bug. For me, it might have something to do with the model.safetensors.index.json and other components not being generated before/when I pushed to the hub? I believe the file is necessary when you have multiple .safetensors shards. I am looking into using the save_pretrained_merged as opposed to save_pretrained, which I was told might not save the base model and LoRA layers together.

@garg-aayush @mosama1994 @hessaAlawwad Much apologies on the delay - I’m currently working on refactoring code to make Unsloth more resilient on saving vision models to 16bit - I’m currently tracking the issue here: #1559

@danielhanchen, how much time is left until the bug fix is released? I have to work on a project with my AI at the end of this month.

I’ve also had this bug. For me, it might have something to do with the model.safetensors.index.json and other components not being generated before/when I pushed to the hub? I believe the file is necessary when you have multiple .safetensors shards. I am looking into using the save_pretrained_merged as opposed to save_pretrained, which I was told might not save the base model and LoRA layers together.

I got it working after adding the model.safetensors.index.json file from the base model.

I’m working on a fix - I’ll try adding it in the next few days - apologies on the delay!