velox: setup-macos.sh fails on Apple M1

It is recommended to use INSTALL_PREFIX say export INSTALL_PREFIX=/Users/velox/velox_dependency_install when installing the dependencies to avoid polluting the default install location. This should also help with the permissions issue.

Thanks @jayzhan211. Running the shared export command before running CPU_TARGET="arm64" ./scripts/setup-macos.sh worked 👍

For future reference to anyone who stumbles on the same issue, here is the list of steps I followed for successfully setting up Velox on M1 Macs.

$ export INSTALL_PREFIX=/Users/{username}/velox/velox_dependency_install
$ CPU_TARGET="arm64" ./scripts/setup-macos.sh
$ export OPENSSL_ROOT_DIR=/opt/homebrew/opt/openssl@1.1
$ CPU_TARGET="arm64" make

NOTE: The command for exporting OPENSSL_ROOT_DIR was to solve a setup error wherein the build script could not find OpenSSL because it required this environment variable to be defined.

@majetideepak I think adding these commands to the project README.md would be worthwhile, possibly under a Troubleshooting heading. This would be a helpful exercise because I found multiple similar issues/threads on the same topic while trying to debug this issue. The number of such problems can be reduced by modifying the project README.md.

If it makes sense, I would love to create a PR.

Thanks.

@dulvinw sure! We have to make this change to the other OS setup scripts as well. We should modify the README and suggest users invoke the setup scripts from the Velox root directory. We need a BUILD_PREFIX for builds similar to INSTALL_PREFIX. We can then create a dependencies_build directory for the builds and a dependencies_install directory for the installs in the Velox root directory. And add these directories to .gitignore Then change setups scripts to add the above defaults for BUILD_PREFIX, INSTALL_PREFIX, and also the root CMakeLists.txt to include CMAKE_PREFIX_PATH CC: @kgpai

i used same way to deal with protobuf, and finally suceessed!

@majetideepak @rishitc thanks you so much!

Quick question: I’m getting a set of failed test cases when running CPU_TARGET="arm64" make unittest on my M1 Mac. I’m sharing the list below.

This is expected. The unit tests are not tested on MacOS and some of them fail as you reported.

my system already had version 23.x installed (this came from some other packages I had installed via brew), and Velox required version 21.x

23.x protobuf issue is very recent. We could throw an error, but it is cumbersome to keep track of and maintain such incompatible versions on different OSes. The ideal solution is to bundle Protobuf and other dependencies which I believe is a work in progress. Installing to the system path via brew or other means is a recipe for such conflicts. Users can alternatively use a container or VM to avoid such build conflicts.

Hi @majetideepak, Yes, I’d like to work on it 👍

On MacOS, you have to set -DCMAKE_CXX_FLAGS=" -isystem /Users/velox/velox_dependency_install"
-DCMAKE_PREFIX_PATH=/Users/velox/velox_dependency_install for the build to pick up the dependencies.

It seems there is still build error for m1

-- Performing Test RANGES_HAS_MTUNE_NATIVE - Success
-- Performing Test RANGE_V3_ALIGNED_NEW_PROBE
-- Performing Test RANGE_V3_ALIGNED_NEW_PROBE - Success
-- Performing Test RANGES_HAS_FCOROUTINES_TS
-- Performing Test RANGES_HAS_FCOROUTINES_TS - Failed
-- Performing Test RANGE_V3_HAS_FCONCEPTS
-- Performing Test RANGE_V3_HAS_FCONCEPTS - Failed
-- Doxygen not found; the 'range-v3-doc' and 'range-v3-gh-pages.{clean,copy,update}' targets will be unavailable.
-- Configuring done (2.6s)
-- Generating done (0.0s)
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_TESTING


-- Build files have been written to: /Users/jayzhan/velox/range-v3/_build
+ ninja -C _build install
ninja: Entering directory `_build'
[0/1] Install the project...
-- Install configuration: "Release"
CMake Error at cmake_install.cmake:57 (file):
  file cannot create directory: /usr/local/lib/cmake/range-v3.  Maybe need
  administrative privileges.


FAILED: CMakeFiles/install.util 
cd /Users/jayzhan/velox/range-v3/_build && /opt/homebrew/Cellar/cmake/3.26.3/bin/cmake -P cmake_install.cmake
ninja: build stopped: subcommand failed.

real    0m15.734s
user    0m3.557s
sys     0m1.973s
+ echo 'Failed to run install_ranges_v3 .'
Failed to run install_ranges_v3 .
+ exit 1

Any workaround or PR to fix?

@kgpai This is a permission issue on directory /usr/local. We got it on Intel processor too (MacOS).

@yingsu00 sure, will open an PR soon.