ray: Failed to install ray above 0.6.4 from source in Arm64 architecture

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): ubuntu 18.04 (Arm64 architecture)
Ray installed from (source or binary): source
Ray version: 0.6.4
Python version: 3.6.7
Exact command to reproduce: pip3 install -e . --verbose

Hi!

I’m trying to install ray 0.6.4 on the NVIDIA Xavier board which has Arm64 CPU.

However, install command fails with following error

Could not find a version that satisfies the requirement pyarrow==0.12.0.RAY (from versions: 0.9.0, 0.10.0, 0.11.0, 0.11.1, 0.12.0, 0.12.1, 0.13.0)
    No matching distribution found for pyarrow==0.12.0.RAY

I found out that build.sh is changed since 0.6.4 so it uses snapshot of pyarrow compiled in X86.

Therefore manually downloading and install whl didn’t work.

Is there a workaround for this problem?

If pyarrow==0.12.0.RAY is same as pyarrow 0.12, can I install it and skip this part in build.sh?

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 17 (2 by maintainers)

Most upvoted comments

Well, that was quite a rabbit hole I had to follow, but I finally got it built on my power 9 system.

Some notes on the process:

I followed the instructions given by @pcmoritz above:

git clone https://github.com/apache/arrow/
cd arrow/cpp
mkdir build
cd build
cmake -DARROW_PLASMA=on -DARROW_PYTHON=on -DARROW_DEPENDENCY_SOURCE=BUNDLED ..
make -j8
sudo make install

Then (note the additional ARROW_HOME variable I needed to set, though there’s probably a more elegant way of setting that with cmake):

cd arrow/python
PYARROW_WITH_PLASMA=1  ARROW_HOME=/usr/local python setup.py bdist_wheel

I had an older version of bazel, but that was no good, so I followed the instructions here (https://docs.bazel.build/versions/master/install-compile-source.html) and bootstrapped from a zip distribution of version 0.21.0.

Then finally, I had an outdated version of git, 1.8, (since I’m running on centos 7.4) which didn’t have the command line flags that the ray build was expecting. So, I built a bleeding edge version (2.9.5).

I was then able to do

pip install -e . --verbose

The only other gotcha I encountered was needing to put the arrow .so files into my LD_LIBRARY_PATH, though I suspect that I didn’t quite build arrow right. There’s probably some rpath flag or another that might help that issue.

From there I was able to do a minimal “ray.put” and “ray.get” test and it seemed to work.

sammuli on Apr 3, 2019