ray: [Bug] Dashboard doesn't work in ray 1.7.0 for local cluster

Search before asking

• I searched the issues and found no similar issues.

Ray Component

Dashboard

What happened + What you expected to happen

Running ray dashboard cluster.yaml and opening http://localhost:8265/ shows an empty page. Upon reloading something is drawn but vanishes after a split second. Dev console reports:

GEThttp://localhost:8265/#/
[HTTP/1.1 304 Not Modified 2ms]

GEThttp://localhost:8265/static/css/2.43a1c8b7.chunk.css
[HTTP/1.1 304 Not Modified 1ms]

GEThttp://localhost:8265/static/css/main.d41b9198.chunk.css
[HTTP/1.1 304 Not Modified 3ms]

GEThttp://localhost:8265/static/js/2.a5682c7a.chunk.js
[HTTP/1.1 304 Not Modified 2ms]

GEThttp://localhost:8265/static/js/main.20b4b076.chunk.js
[HTTP/1.1 304 Not Modified 2ms]

Unknown property ‘-moz-osx-font-smoothing’.  Declaration dropped. localhost:8265:5:27
Error in parsing value for ‘animation’.  Declaration dropped. localhost:8265
Error in parsing value for ‘animation’.  Declaration dropped. localhost:8265
Error in parsing value for ‘animation’.  Declaration dropped. localhost:8265
Error in parsing value for ‘animation’.  Declaration dropped. localhost:8265
Error in parsing value for ‘animation’.  Declaration dropped. localhost:8265
Error in parsing value for ‘animation’.  Declaration dropped. localhost:8265
XHRGEThttp://localhost:8265/nodes?view=details
[HTTP/1.1 200 OK 1ms]

XHRGEThttp://localhost:8265/tune/availability
[HTTP/1.1 200 OK 3ms]

XHRGEThttp://localhost:8265/nodes?view=summary
[HTTP/1.1 200 OK 3ms]

XHRGEThttp://localhost:8265/logical/actor_groups
[HTTP/1.1 200 OK 3ms]

GEThttp://localhost:8265/favicon.ico
[HTTP/1.1 200 OK 0ms]

TypeError: e is undefined
    M NodeInfo.tsx:211
    _n NodeInfo.tsx:211
    React 7
    unstable_runWithPriority scheduler.production.min.js:19
    React 4
    Redux 5
    t Dashboard.tsx:70
    c runtime.js:63
    _invoke runtime.js:293
    b runtime.js:118
    Babel 6
    ja Dashboard.tsx:83
    React 2
    unstable_runWithPriority scheduler.production.min.js:19
    React 19
    162 index.tsx:6
    u (index):1
    161 main.20b4b076.chunk.js:1
    u (index):1
    t (index):1
    r (index):1
    <anonymous> main.20b4b076.chunk.js:1
react-dom.production.min.js:209:194
    React 5
    unstable_runWithPriority scheduler.production.min.js:19
    React 4
    unstable_runWithPriority scheduler.production.min.js:19
    React 4
    Redux 5
    t Dashboard.tsx:70
    c runtime.js:63
    _invoke runtime.js:293
    b runtime.js:118
    Babel 6
    ja Dashboard.tsx:83
    React 2
    unstable_runWithPriority scheduler.production.min.js:19
    React 19
    162 index.tsx:6
    u (index):1
    161 main.20b4b076.chunk.js:1
    u (index):1
    t (index):1
    r (index):1
    <anonymous> main.20b4b076.chunk.js:1

Reproduction script

Spin up your local cluster (ray up --no-config-cache cluster.yaml), wait till ray monitor cluster.yaml reports all nodes are available, then ray dashboard cluster.yaml, open http://localhost:8265/

Anything else

ray==1.7.0 on all nodes

Head node’s dashboard.log:

$ tail /tmp/ray/session_latest/logs/dashboard.log                                                                                                                                                                                                                                                              
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi", line 147, in grpc._cython.cygrpc.PollerCompletionQueue._handle_events                                                                                                                                                                          
BlockingIOError: [Errno 11] Resource temporarily unavailable                                                                                                                                                                                                                                                                 
2021-10-08 01:20:56,774 ERROR base_events.py:1619 -- Exception in callback PollerCompletionQueue._handle_events(<_UnixSelecto...e debug=False>)()                                                                                                                                                                            
handle: <Handle PollerCompletionQueue._handle_events(<_UnixSelecto...e debug=False>)()>                                                                                                                                                                                                                                      
Traceback (most recent call last):                                                                                                                                                                                                                                                                                           
  File "/home/test/PROGS/miniconda3/envs/puma-lab/lib/python3.7/asyncio/events.py", line 88, in _run                                                                                                                                                                                                                      
    self._context.run(self._callback, *self._args)                                                                                                                                                                                                                                                                           
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi", line 147, in grpc._cython.cygrpc.PollerCompletionQueue._handle_events                                                                                                                                                                          
BlockingIOError: [Errno 11] Resource temporarily unavailable                                                                                                                                                                                                                                                                 
2021-10-08 01:20:57,324 INFO node_head.py:257 -- Received a log for 192.168.0.101 and autoscaler 

Head node:

$ pip install 'ray[default]'
Requirement already satisfied: ray[default] in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (1.7.0)
Requirement already satisfied: attrs in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (21.2.0)
Requirement already satisfied: grpcio>=1.28.1 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (1.40.0)
Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (1.0.2)
Requirement already satisfied: pyyaml in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (5.4.1)
Requirement already satisfied: click>=7.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (8.0.1)
Requirement already satisfied: numpy>=1.16 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (1.20.3)
Requirement already satisfied: redis>=3.5.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (3.5.3)
Requirement already satisfied: protobuf>=3.15.3 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (3.17.3)
Requirement already satisfied: filelock in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (3.0.12)
Requirement already satisfied: py-spy>=0.2.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (0.3.9)
Requirement already satisfied: jsonschema in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (3.2.0)
Requirement already satisfied: aioredis<2 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (1.3.1)
Requirement already satisfied: aiohttp in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (3.7.4.post0)
Requirement already satisfied: requests in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (2.26.0)
Requirement already satisfied: aiohttp-cors in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (0.7.0)
Requirement already satisfied: gpustat in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (0.6.0)
Requirement already satisfied: opencensus in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (0.7.13)
Requirement already satisfied: prometheus-client>=0.7.1 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (0.11.0)
Requirement already satisfied: colorful in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from ray[default]) (0.5.4)
Requirement already satisfied: hiredis in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from aioredis<2->ray[default]) (2.0.0)
Requirement already satisfied: async-timeout in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from aioredis<2->ray[default]) (3.0.1)
Requirement already satisfied: importlib-metadata in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from click>=7.0->ray[default]) (4.8.1)
Requirement already satisfied: six>=1.5.2 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from grpcio>=1.28.1->ray[default]) (1.16.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from aiohttp->ray[default]) (5.1.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from aiohttp->ray[default]) (1.6.3)
Requirement already satisfied: typing-extensions>=3.6.5 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from aiohttp->ray[default]) (3.10.0.0)
Requirement already satisfied: chardet<5.0,>=2.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from aiohttp->ray[default]) (4.0.0)
Requirement already satisfied: idna>=2.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->ray[default]) (3.1)
Requirement already satisfied: psutil in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from gpustat->ray[default]) (5.8.0)
Requirement already satisfied: nvidia-ml-py3>=7.352.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from gpustat->ray[default]) (7.352.0)
Requirement already satisfied: blessings>=1.6 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from gpustat->ray[default]) (1.7)
Requirement already satisfied: zipp>=0.5 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from importlib-metadata->click>=7.0->ray[default]) (3.5.0)
Requirement already satisfied: pyrsistent>=0.14.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from jsonschema->ray[default]) (0.17.3)
Requirement already satisfied: setuptools in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from jsonschema->ray[default]) (58.0.4)
Requirement already satisfied: opencensus-context==0.1.2 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from opencensus->ray[default]) (0.1.2)
Requirement already satisfied: google-api-core<2.0.0,>=1.0.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from opencensus->ray[default]) (1.31.2)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from google-api-core<2.0.0,>=1.0.0->opencensus->ray[default]) (1.53.0)
Requirement already satisfied: google-auth<2.0dev,>=1.25.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from google-api-core<2.0.0,>=1.0.0->opencensus->ray[default]) (1.35.0)
Requirement already satisfied: packaging>=14.3 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from google-api-core<2.0.0,>=1.0.0->opencensus->ray[default]) (21.0)
Requirement already satisfied: pytz in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from google-api-core<2.0.0,>=1.0.0->opencensus->ray[default]) (2021.1)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from google-auth<2.0dev,>=1.25.0->google-api-core<2.0.0,>=1.0.0->opencensus->ray[default]) (4.2.2)
Requirement already satisfied: rsa<5,>=3.1.4 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from google-auth<2.0dev,>=1.25.0->google-api-core<2.0.0,>=1.0.0->opencensus->ray[default]) (4.7.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from google-auth<2.0dev,>=1.25.0->google-api-core<2.0.0,>=1.0.0->opencensus->ray[default]) (0.2.8)
Requirement already satisfied: pyparsing>=2.0.2 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from packaging>=14.3->google-api-core<2.0.0,>=1.0.0->opencensus->ray[default]) (2.4.7)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2.0dev,>=1.25.0->google-api-core<2.0.0,>=1.0.0->opencensus->ray[default]) (0.4.8)
Requirement already satisfied: certifi>=2017.4.17 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from requests->ray[default]) (2021.5.30)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from requests->ray[default]) (2.0.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/toaster/PROGS/miniconda3/envs/puma-lab/lib/python3.7/site-packages (from requests->ray[default]) (1.26.6)

Are you willing to submit a PR?

• Yes I am willing to submit a PR!