nmslib: Installation errors apple m1
Hello!
I tried to install via
pip install nmslib
There are following logs and errors:
Collecting nmslib
Using cached nmslib-2.1.1.tar.gz (188 kB)
Requirement already satisfied: pybind11<2.6.2 in /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages (from nmslib) (2.6.1)
Requirement already satisfied: psutil in /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages (from nmslib) (5.8.0)
Requirement already satisfied: numpy>=1.10.0 in /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages (from nmslib) (1.19.5)
Building wheels for collected packages: nmslib
Building wheel for nmslib (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"'; __file__='"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-wheel-imwlifeo
cwd: /private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/
Complete output (23 lines):
Dependence list: ['pybind11<2.6.2', 'psutil', "numpy>=1.10.0,<1.17 ; python_version=='2.7'", "numpy>=1.10.0 ; python_version>='3.5'"]
running bdist_wheel
running build
running build_ext
creating var
creating var/folders
creating var/folders/lk
creating var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn
creating var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c /var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpqhrb07ue.cpp -o var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpqhrb07ue.o -std=c++14
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c /var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmp8tdl7mo4.cpp -o var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmp8tdl7mo4.o -fvisibility=hidden
Extra compilation arguments: ['-O3', '-march=native', '-stdlib=libc++', '-mmacosx-version-min=10.7', '-DVERSION_INFO="2.1.1"', '-std=c++14', '-fvisibility=hidden']
building 'nmslib' extension
creating build
creating build/temp.macosx-11.0-arm64-3.8
creating build/temp.macosx-11.0-arm64-3.8/tensorflow
creating build/temp.macosx-11.0-arm64-3.8/similarity_search
creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src
creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src/space
creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src/method
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I./similarity_search/include -Itensorflow -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/numpy/core/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c nmslib.cc -o build/temp.macosx-11.0-arm64-3.8/nmslib.o -O3 -march=native -stdlib=libc++ -mmacosx-version-min=10.7 -DVERSION_INFO="2.1.1" -std=c++14 -fvisibility=hidden
clang: error: the clang compiler does not support '-march=native'
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for nmslib
Running setup.py clean for nmslib
Failed to build nmslib
Installing collected packages: nmslib
Running setup.py install for nmslib ... error
ERROR: Command errored out with exit status 1:
command: /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"'; __file__='"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-record-ccpn80un/install-record.txt --single-version-externally-managed --compile --install-headers /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8/nmslib
cwd: /private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/
Complete output (18 lines):
Dependence list: ['pybind11<2.6.2', 'psutil', "numpy>=1.10.0,<1.17 ; python_version=='2.7'", "numpy>=1.10.0 ; python_version>='3.5'"]
running install
running build
running build_ext
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c /var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpci3coski.cpp -o var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpci3coski.o -std=c++14
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c /var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpfun71y94.cpp -o var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpfun71y94.o -fvisibility=hidden
Extra compilation arguments: ['-O3', '-march=native', '-stdlib=libc++', '-mmacosx-version-min=10.7', '-DVERSION_INFO="2.1.1"', '-std=c++14', '-fvisibility=hidden']
building 'nmslib' extension
creating build
creating build/temp.macosx-11.0-arm64-3.8
creating build/temp.macosx-11.0-arm64-3.8/tensorflow
creating build/temp.macosx-11.0-arm64-3.8/similarity_search
creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src
creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src/space
creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src/method
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I./similarity_search/include -Itensorflow -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/numpy/core/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c nmslib.cc -o build/temp.macosx-11.0-arm64-3.8/nmslib.o -O3 -march=native -stdlib=libc++ -mmacosx-version-min=10.7 -DVERSION_INFO="2.1.1" -std=c++14 -fvisibility=hidden
clang: error: the clang compiler does not support '-march=native'
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"'; __file__='"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-record-ccpn80un/install-record.txt --single-version-externally-managed --compile --install-headers /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8/nmslib Check the logs for full command output.
Also I tried to install via
pip install --no-binary :all: nmslib
but the error still the same.
Versions: Python 3.8.8 pip 21.0.1
Could you please help me to figure out what are the ways to install nmslib on Apple Silicon M1? Or maybe I’m missing something.
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 35 (12 by maintainers)
I can get this to install using the following command:
CFLAGS="-mavx -DWARN(a)=(a)" pip install nmslibNote that this seems to disable a lot of acceleration, so I wouldn’t recommend it for production.I don’t know if this will work for you but I had tried
CFLAGS="-mavx -DWARN(a)=(a)" pip install nmsliband got back this error:So I ran
CFLAGS="-mavx -DWARN(a)=(a)" pip install --use-pep517 nmslibwith the--use-pep517flag and that worked!I got here after trying to install
nmslibon a MacBookPro (Big Sur, M1 chip).What finally worked for me was a variation on a few of these recommendations:
conda(miniforge) to manage my environmentpip install nmslibnorpip install nmslib --no-binary :all:workedclangsetting from-march=nativeto-mcpu=apple-a14. I did this by addingset(SIMD_FLAGS " -mcpu=apple-a14")intoCMakeLists.txtbut I’m sure there’s an environment variable way to do this…clangwas still complaining about a nested macro in a#pragma messageinsimilarity_search/src/distcomp_scalar.cc. Commenting out those two warnings (innocent) permitted everything to build (cmake .; make)bunit(n.b.bunitlooks for data in../sample_databy default but that’s not the correct relative path from its default build location … by soft-linking it into../and then running there it worked)python_bindings/setup.pychangec_opts['unix'].append('-march=native')toc_opts['unix'].append('-mcpu=apple-a14')python setup.py installworked perfectly fine and installednmslib==2.1.1into mycondaenvironment. Verified everything by running the tutorial example in the docs.there’s a PR, I will merge it soon.
The only solution that worked for me was the one pointed by @8W9aG . If you are installing with Poetry also, simply write
CFLAGS="-mavx -DWARN(a)=(a)" poetry add .....Also ran into this issue on the M1 chip. Installing on a conda environment with
pip install --no-binary :all: nmslibresolved itI could install it following @drkeoni’s instructions. I hope they will be taken into account for a future release out-of-the-box compatible with M1.
@azhx I don’t maintain a conda installation, but apparently it doesn’t distinguish between Intel and m1 MAC. So, then it simply crashes. I have heard M1 can run Intel code, but, apparently, this does not work ideally. They may also remove that flag from their installation code. That is one reason why you can install from sources. When you install from sources, clearly, there is no issue with running Intel binary on M1, b/c you compile into native N1 code.