llama-cpp-python: (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')) on M2
Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [X ] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [X ] I carefully followed the README.md.
- [ X] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- [ X] I reviewed the Discussions, and have a new bug or useful enhancement to share.
Expected Behavior
I’m trying to launch the Llama API following the guide in https://github.com/abetlen/llama-cpp-python#:~:text=docs/install/macos.md, on my Macbook M2.
Current Behavior
Unfortunately, after running the last command, which is
python3 -m llama_cpp.server --model $MODEL --n_gpu_layers 1
I get
RuntimeError: Failed to load shared library '/Users/alessandropaticchio/miniforge3/envs/llama/lib/python3.9/site-packages/llama_cpp/libllama.dylib': dlopen(/Users/alessandropaticchio/miniforge3/envs/llama/lib/python3.9/site-packages/llama_cpp/libllama.dylib, 0x0006): tried: '/Users/alessandropaticchio/miniforge3/envs/llama/lib/python3.9/site-packages/llama_cpp/libllama.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/alessandropaticchio/miniforge3/envs/llama/lib/python3.9/site-packages/llama_cpp/libllama.dylib' (no such file), '/Users/alessandropaticchio/miniforge3/envs/llama/lib/python3.9/site-packages/llama_cpp/libllama.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))
Any help would be much appreciated!
About this issue
- Original URL
- State: open
- Created 8 months ago
- Comments: 18 (2 by maintainers)
Commits related to this issue
- Add new binaries to flake.nix (#847) — committed to antoine-lizee/llama-cpp-python by longregen a year ago
@YAY-3M-TA3 comment + using a previous version (0.2.11) worked for me.
Here is what worked for me after all (with latest version):
CMAKE_ARGS="-DCMAKE_OSX_ARCHITECTURES=arm64 -DCMAKE_APPLE_SILICON_PROCESSOR=arm64 -DLLAMA_METAL=on" pip install --upgrade --verbose --force-reinstall --no-cache-dir llama-cpp-python
I started digging from the logline:
and stumbled upon this thread https://discourse.cmake.org/t/macos-cmake-detect-system-processor-incorrectly-on-apple-silicon/5129/5
@abetlen probably makes sense to add this in the troubleshooting / README?
Probably has something to do with llava1.5 multi-modal support being added to 0.2.14 (https://github.com/abetlen/llama-cpp-python/commit/aab74f0b2bd25cc2a9baeb743a2057dd5cada6e4) - since a second lib has to be built (libllava.dylib) - maybe this is why @steveoOn 's fix no longer works…?