llama.cpp: illegal instrution

system:android 13 python:3.11 model:vicuna-7b-v1.3.ggmlv3.q4_1.bin

`~/…/models/7B $ ln -s ~/storage/downloads/python/vicuna-7b-v1.3.ggmlv3.q4_1.bin ggml-model.bin

~/…/models/7B $ ls ggml-model.bin ggml-model.bin.old

~/…/models/7B $ cd …/…

~/llama.cpp $ ./main
main: build = 776 (55dbb91)
main: seed = 1688404061 llama.cpp: loading model from models/7B/ggml-model.bin
llama_model_load_internal: format = ggjt v3 (latest) llama_model_load_internal: n_vocab = 32000 llama_model_load_internal: n_ctx = 512 llama_model_load_internal: n_embd = 4096 llama_model_load_internal: n_mult = 256 llama_model_load_internal: n_head = 32 llama_model_load_internal: n_layer = 32 llama_model_load_internal: n_rot = 128 llama_model_load_internal: ftype = 3 (mostly Q4_1) llama_model_load_internal: n_ff = 11008 llama_model_load_internal: model size = 7B llama_model_load_internal: ggml ctx size = 0.08 MB
Illegal instruction`

what’ the next?

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 27 (1 by maintainers)

Most upvoted comments

sed -i ‘s/add_compile_options(-mcpu=native)//g’ CMakeLists.txt

strange that:

  1. LLAMA_NATIVE is OFF by default, add_compile_options(-march=native) should not be executed.
  2. @adaaaaaa 's case: the main built with cmake works. make CFLAGS contains -mcpu=native but no -mfpu, that means $(UNAME_M) matches aarch64, but does not match armvX.

Related issue (previous fix) https://github.com/ggerganov/llama.cpp/issues/1210