llama-cpp-python: Can't install llama-cpp-python -libpython3.11.a file not found during building wheel (pyproject.toml)

Prerequisites

Please answer the following questions for yourself before submitting an issue.

  • I am running the latest code. Development is very rapid so there are no tagged versions as of now.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

Trying to install llama-cpp-python as stated by this document: https://github.com/KillianLucas/open-interpreter/blob/main/docs/MACOS.md.

Current Behavior

Getting the following error while running CMAKE_ARGS=“-DLLAMA_METAL=on” FORCE_CMAKE=1 pip install -U llama-cpp-python --no-cache-dir:

 Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [44 lines of output]
      *** scikit-build-core 0.5.0 using CMake 3.27.4 (wheel)
      *** Configuring CMake...
      2023-09-14 11:43:30,426 - scikit_build_core - WARNING - libdir/ldlibrary: /Users/arjen/miniforge3/envs/oi/lib/libpython3.11.a is not a real file!
      2023-09-14 11:43:30,426 - scikit_build_core - WARNING - Can't find a Python library, got libdir=/Users/arjen/miniforge3/envs/oi/lib, ldlibrary=libpython3.11.a, multiarch=darwin, masd=None

Environment and Context

python3.11.4 in a conda environment from miniforge3 for arm64 support, as stated in the readme file:

"Note: If you are using Apple Silicon (M1) Mac, make sure you have installed a version of Python that supports arm64 architecture. For example:

wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
bash Miniforge3-MacOSX-arm64.sh"

Tried to find online for solutions, installed packages like gcc, xcode c++, etc. but nothing resolved the issue. The error states it requires libpython3.11.a in that location, but such a file does not exist there. However, the libpython3.11 does. If I search online for that file, I can only find .deb packages for linux containing that file. But I can’t unpack such packages, even with the ar command I can’t get that file to release from the package.

  • Physical (or virtual) hardware you are using: Macbook m1 max 10core cpu - 24core gpu - 32gb ram - 2tb disk

  • Operating System: MacOS Ventura 13.0:

Darwin Laptop-van-Arjen.local 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:15:09 PDT 2022; root:xnu-8792.41.9~2/RELEASE_ARM64_T6000 arm64

$ python3 --version = 3.11.4
$ make --version = GNU Make 3.81
This program built for i386-apple-darwin11.3.0
$ g++ --version = Apple clang version 14.0.0 (clang-1400.0.29.202)
Target: arm64-apple-darwin22.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Error output during install:

Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [44 lines of output]
      *** scikit-build-core 0.5.0 using CMake 3.27.4 (wheel)
      *** Configuring CMake...
      2023-09-14 11:43:30,426 - scikit_build_core - WARNING - libdir/ldlibrary: /Users/arjen/miniforge3/envs/oi/lib/libpython3.11.a is not a real file!
      2023-09-14 11:43:30,426 - scikit_build_core - WARNING - Can't find a Python library, got libdir=/Users/arjen/miniforge3/envs/oi/lib, ldlibrary=libpython3.11.a, multiarch=darwin, masd=None
      loading initial cache file /var/folders/r4/9mrbh04j1_gc4h5z0m3f52d80000gn/T/tmpnhw2qb6p/build/CMakeInit.txt
      -- The C compiler identification is AppleClang 14.0.0.14000029
      -- The CXX compiler identification is AppleClang 14.0.0.14000029
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found Git: /usr/bin/git (found version "2.37.1 (Apple Git-137.1)")
      fatal: not a git repository (or any of the parent directories): .git
      fatal: not a git repository (or any of the parent directories): .git
      CMake Warning at vendor/llama.cpp/CMakeLists.txt:125 (message):
        Git repository not found; to enable automatic generation of build info,
        make sure Git is installed and the project is a Git repository.
      
      
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
      -- Check if compiler accepts -pthread
      -- Check if compiler accepts -pthread - no
      -- Looking for pthread_create in pthreads
      -- Looking for pthread_create in pthreads - not found
      -- Looking for pthread_create in pthread
      -- Looking for pthread_create in pthread - not found
      CMake Error at /opt/homebrew/Cellar/cmake/3.27.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find Threads (missing: Threads_FOUND)
      Call Stack (most recent call first):
        /opt/homebrew/Cellar/cmake/3.27.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
        /opt/homebrew/Cellar/cmake/3.27.4/share/cmake/Modules/FindThreads.cmake:226 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
        vendor/llama.cpp/CMakeLists.txt:137 (find_package)
      
      
      -- Configuring incomplete, errors occurred!
      
      *** CMake configuration failed
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

About this issue

  • Original URL
  • State: closed
  • Created 10 months ago
  • Reactions: 2
  • Comments: 45 (15 by maintainers)

Most upvoted comments

@jzavala-gonzalez wooooh haha okay now we’re getting somewhere. Thank you, for now I’ll guard that block so it’s not set for arm64 apple computers.

Just rebuilt it 6 times and it’s the last two lines (individually) that crash the build. The ones that set the C and/or CXX flags

I think this might be related! Following dev instructions still gives same error, but doing so after removing these lines appears to work successfully.

If you’ve cloned the llama-cpp-python repo, can you try removing these lines from the root CMakeLists.txt file and rebuilding

    if (APPLE)
        # Need to disable these llama.cpp flags on Apple
        # otherwise users may encounter invalid instruction errors
        set(LLAMA_AVX "Off" CACHE BOOL "llama: enable AVX" FORCE)
        set(LLAMA_AVX2 "Off" CACHE BOOL "llama: enable AVX2" FORCE)
        set(LLAMA_FMA "Off" CACHE BOOL "llama: enable FMA" FORCE)
        set(LLAMA_F16C "Off" CACHE BOOL "llama: enable F16C" FORCE)
        set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -march=native -mtune=native")
        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -march=native -mtune=native")
    endif()

Shouldn’t be related as it just disables some hw accelerations that were failing in ci but you never know.

Here is the log from after commenting out that whole block:

❯ pip install -e .
Obtaining file:///Users/gabriel/Documents/GitHub/Repos/abetlen/llama-cpp-python
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=4.5.0 in /Users/gabriel/anaconda3/envs/cabildo-extract/lib/python3.11/site-packages (from llama_cpp_python==0.2.4) (4.7.1)
Requirement already satisfied: numpy>=1.20.0 in /Users/gabriel/anaconda3/envs/cabildo-extract/lib/python3.11/site-packages (from llama_cpp_python==0.2.4) (1.25.2)
Collecting diskcache>=5.6.1 (from llama_cpp_python==0.2.4)
  Obtaining dependency information for diskcache>=5.6.1 from https://files.pythonhosted.org/packages/3f/27/4570e78fc0bf5ea0ca45eb1de3818a23787af9b390c0b0a0033a1b8236f9/diskcache-5.6.3-py3-none-any.whl.metadata
  Using cached diskcache-5.6.3-py3-none-any.whl.metadata (20 kB)
Using cached diskcache-5.6.3-py3-none-any.whl (45 kB)
Building wheels for collected packages: llama_cpp_python
  Building editable for llama_cpp_python (pyproject.toml) ... done
  Created wheel for llama_cpp_python: filename=llama_cpp_python-0.2.4-cp311-cp311-macosx_12_0_arm64.whl size=783349 sha256=959bdf597bb6c1927ebf098febc9d3c393069e701c5b9a5bacc998add251dd60
  Stored in directory: /private/var/folders/ts/_0b3z_qn7sd72nfffjkydm8h0000gn/T/pip-ephem-wheel-cache-_5uc4lxg/wheels/c3/7e/26/56f285bcabdedcf4b9a95a571bbfd43a52a306c01ca92fee82
Successfully built llama_cpp_python
Installing collected packages: diskcache, llama_cpp_python
Successfully installed diskcache-5.6.3 llama_cpp_python-0.2.4

@icecoldt369 okay looks like your issue is actually a llama.cpp build error, do you mind opening an issue there, if it gets resolved I can merge in the fix and llama-cpp-python should work for you.

Hey @primemp looking at tthe logs the issue related to the libpython.11.a file thing is just a warning, the actual error that is causing the build to fail is from cmake here:

      -- Looking for pthread_create in pthread - not found
      CMake Error at /opt/homebrew/Cellar/cmake/3.27.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find Threads (missing: Threads_FOUND)
      Call Stack (most recent call first):
        /opt/homebrew/Cellar/cmake/3.27.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
        /opt/homebrew/Cellar/cmake/3.27.4/share/cmake/Modules/FindThreads.cmake:226 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
        vendor/llama.cpp/CMakeLists.txt:137 (find_package)

Can you try installing llama.cpp with cmake and posting that log? Should help me narrow down the issue.

@icecoldt369 @remixer-dec are you getting the same Could not find Threads error?