llama.cpp: Convert consolidated pth files to gguf / missing script?
I’m trying to run llama2 models on a mac M1 pro max
- I’ve downloaded the llama2 models with the link that was provided by email
- I’ve downloaded llama.cpp & compiled with make
- If I understand correctly llama.cpp needs the models to be converted to gguf (ggml?) format
- I see in the models only consolidated.00.pth files
- I’m looking for the script
convert-pth-to-gguf.py
but it’s not there
what am I missing?
I guess after this little step I can run
make -j && ./main -m models/llama-13b-v2/ggml-model-q4_0.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e
Version:
Python 3.11.6
GNU Make 3.81
Apple clang version 15.0.0 (clang-1500.0.40.1)
When i try to run:
python3 -m venv .env
source .env/bin/activate
pip install -r requirements.txt
python convert.py ../llama/llama-2-7b-chat
I get:
Loading model file ../llama/llama-2-7b-chat/consolidated.00.pth
Traceback (most recent call last):
File "/Users/alex/dev/llama.cpp/convert.py", line 1207, in <module>
main()
File "/Users/alex/dev/llama.cpp/convert.py", line 1140, in main
model_plus = load_some_model(args.model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/alex/dev/llama.cpp/convert.py", line 1055, in load_some_model
models_plus.append(lazy_load_file(path))
^^^^^^^^^^^^^^^^^^^^
File "/Users/alex/dev/llama.cpp/convert.py", line 740, in lazy_load_file
raise ValueError(f"unknown format: {path}")
ValueError: unknown format: ../llama/llama-2-7b-chat/consolidated.00.pth
About this issue
- Original URL
- State: closed
- Created 8 months ago
- Comments: 15 (6 by maintainers)
Just encountered this. The root cause in my case was putting in the wrong value for the first step in LLaMa’s download script. You’re supposed to put in the link from the email but I accidentally put in my actual email address.
params.json
was full of HTML because Meta was essentially denying my download request. As others have said, just delete the failed download directory and start over.