ray: Cannot be installed directly from repository

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
  • Ray installed from (source or binary): source (at 1a39fee9c6fa0f81e1184182b47ef106269613fe)
  • Python version: 3.6.7
  • Exact command to reproduce:
$ pip3 install -e 'git+https://github.com/ray-project/ray.git#subdirectory=python&egg=ray'
Obtaining ray from git+https://github.com/ray-project/ray.git#subdirectory=python&egg=ray
  Cloning https://github.com/ray-project/ray.git to /src/ray
Requirement already satisfied: numpy>=1.14 in /usr/local/lib/python3.6/dist-packages (from ray) (1.16.3)
Requirement already satisfied: filelock in /usr/local/lib/python3.6/dist-packages (from ray) (3.0.12)
Requirement already satisfied: funcsigs in /usr/local/lib/python3.6/dist-packages (from ray) (1.0.2)
Requirement already satisfied: click in /usr/local/lib/python3.6/dist-packages (from ray) (7.0)
Requirement already satisfied: colorama in /usr/local/lib/python3.6/dist-packages (from ray) (0.4.1)
Requirement already satisfied: pytest in /usr/local/lib/python3.6/dist-packages (from ray) (4.5.0)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from ray) (5.1)
Requirement already satisfied: redis in /usr/local/lib/python3.6/dist-packages (from ray) (3.2.1)
Requirement already satisfied: six>=1.0.0 in /usr/lib/python3/dist-packages (from ray) (1.11.0)
Requirement already satisfied: typing in /usr/local/lib/python3.6/dist-packages (from ray) (3.6.6)
Requirement already satisfied: flatbuffers in /usr/local/lib/python3.6/dist-packages (from ray) (1.11)
Requirement already satisfied: atomicwrites>=1.0 in /usr/local/lib/python3.6/dist-packages (from pytest->ray) (1.3.0)
Requirement already satisfied: more-itertools>=4.0.0; python_version > "2.7" in /usr/local/lib/python3.6/dist-packages (from pytest->ray) (7.0.0)
Requirement already satisfied: wcwidth in /usr/local/lib/python3.6/dist-packages (from pytest->ray) (0.1.7)
Requirement already satisfied: py>=1.5.0 in /usr/local/lib/python3.6/dist-packages (from pytest->ray) (1.8.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.6/dist-packages (from pytest->ray) (19.1.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from pytest->ray) (41.0.1)
Requirement already satisfied: pluggy!=0.10,<1.0,>=0.9 in /usr/local/lib/python3.6/dist-packages (from pytest->ray) (0.11.0)
Installing collected packages: ray
  Running setup.py develop for ray
    Complete output from command /usr/bin/python3 -c "import setuptools, tokenize;__file__='/src/ray/python/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps:
    running develop
    running egg_info
    writing ray.egg-info/PKG-INFO
    writing dependency_links to ray.egg-info/dependency_links.txt
    writing entry points to ray.egg-info/entry_points.txt
    writing requirements to ray.egg-info/requires.txt
    writing top-level names to ray.egg-info/top_level.txt
    reading manifest file 'ray.egg-info/SOURCES.txt'
    writing manifest file 'ray.egg-info/SOURCES.txt'
    running build_ext
    + set -e
    +++ dirname ../build.sh
    ++ cd ..
    ++ pwd
    + ROOT_DIR=/src/ray
    ++ uname
    + unamestr=Linux
    + [[ Linux == \L\i\n\u\x ]]
    + PARALLEL=1
    + RAY_BUILD_PYTHON=YES
    + RAY_BUILD_JAVA=NO
    + PYTHON_EXECUTABLE=
    + BUILD_DIR=
    + [[ 2 -gt 0 ]]
    + key=-p
    + case $key in
    + PYTHON_EXECUTABLE=/usr/bin/python3
    + shift
    + shift
    + [[ 0 -gt 0 ]]
    + [[ -z /usr/bin/python3 ]]
    + echo 'Using Python executable /usr/bin/python3.'
    Using Python executable /usr/bin/python3.
    ++ PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.bazel/bin
    ++ which bazel
    + BAZEL_EXECUTABLE=
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/src/ray/python/setup.py", line 183, in <module>
        license="Apache 2.0")
      File "/usr/local/lib/python3.6/dist-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.6/dist-packages/setuptools/command/develop.py", line 38, in run
        self.install_for_development()
      File "/usr/local/lib/python3.6/dist-packages/setuptools/command/develop.py", line 140, in install_for_development
        self.run_command('build_ext')
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/src/ray/python/setup.py", line 74, in run
        subprocess.check_call(command)
      File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['../build.sh', '-p', '/usr/bin/python3']' returned non-zero exit status 1.
    
    ----------------------------------------
Command "/usr/bin/python3 -c "import setuptools, tokenize;__file__='/src/ray/python/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps" failed with error code 1 in /src/ray/python
You are using pip version 19.0.3, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Describe the problem

In documentation it is written that I can install Ray directly from the repository. This has also worked well before migration to Bazel. But it seems it is not working anymore.

This is important for us so that we can list all our dependencies in requirements.txt and that if we make a fork that we can continue to use Ray without having to publish it to PyPi. So installing directly from the repository using pip is important to support.

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Comments: 15 (11 by maintainers)

Most upvoted comments

This line on the installation instructions:

pip install git+https://github.com/ray-project/ray.git#subdirectory=python

Is quite misleading. At least, it should be noted that the user must git clone and run the bazel installation.

@jovany-wang I don’t think the main README is the right place for it, but it should be in the installation instructions.

Users should just use pip install.

Yes, pip install from the branch in the GitHub repo. 😃

So imagine that user reports an issue, you make MR, you ask them to test it, so they want to test that branch. They will do pip install git+https://github.com/ray-project/ray.git@branch#subdirectory=python and things will not work.