unsloth: [TEMP FIX] Ollama / llama.cpp: cannot find tokenizer merges in model file
Thank you for developing this useful resource. The Ollama notebook reports
{"error":"llama runner process has terminated: error loading modelvocabulary: cannot find tokenizer merges in model file"}
This is the notebook with the error. It is a copy of the original notebook.
This seems similar to the issue reported in #1062.
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 6
- Comments: 37 (7 by maintainers)
I just communicated with the Hugging Face team - they will upstream updates to
llama.cpplater in the week. It seems liketokenizers>=0.20.0is the culprit.I re-uploaded all
Llama-3.2models and as a temporary fix, Unsloth will usetransformers==4.44.2.Please try again and see if it works! This unfortunately means you need to re-finetune the model if you did not save the 16bit merged HF weights or the LoRA weights - extreme apologiesnif you saved them, simply update Unsloth then reload them and convert to GGUF.
Update Unsloth via:
I will update everyone once the Hugging Face team resolves the issue! Sorry again!
Pinging everyone (and apologies for the issues and inconvenience again!!) @xmaayy @avvRobertoAlma @thackmann @kingabzpro @williamzebrowskI @FotieMConstant @laoc81 @gianmarcoalessio @ThaisBarrosAlvim @Franky-W @Saber120 @adampetr @David33706 @Mukunda-Gogoi
Apologies guys - was out for a few days and its been hectic, so sorry on the delay!! Will get to the bottom of fix and hopefully can fix it today! Sorry and thank you all for your patience!
It is working on Kaggle now. Thank you.
@kingabzpro I just updated pypi so
pip install unslothshould have the temporary fixes - you might have to restart KaggleThank you @danielhanchen for the quick fix. The original notebook is now working.
Thank you for the update! I followed the steps you provided, and I’m happy to report that it worked perfectly on my end. I updated Unsloth, reloaded the saved weights, and successfully converted them to GGUF. Everything is running smoothly now with the transformers==4.44.2 fix.
I appreciate the quick re-upload and the detailed instructions. I’ll keep an eye out for the official update from Hugging Face, but for now, everything seems to be working great.
Thanks again for your efforts!
Best regards,
Thanks @danielhanchen, and sorry for the disturbances; to give the context as to what is happening here, we updated the format of merges serialization in
tokenizersto be much more flexible (this was done in this commit):The change was done to be backwards-compatible :
tokenizersand all libraries that depend on it will keep the ability to load merge files which were serialized in the old way.However, it could not be forwards-compatible: if a file is serialized with the new format, older versions of
tokenizerswill not be able to load it.This is why we’re seeing this issue: new files are serialized using the new version, and these files are not loadable in llama.cpp, yet. We’re updating all other codepaths (namely llama.cpp) to adapt to the new version. Once that is shipped, all your trained checkpoints will be directly loadable as usual. We’re working with llama.cpp to ship this as fast as possible.
Thank you!
Issue tracker in llama.cpp: https://github.com/ggerganov/llama.cpp/issues/9692