ray: [dashboard] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/ray/session_2022-07-06_16-46-30_058837_12938/logs/dashboard.log'
What happened + What you expected to happen
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/ray/session_2022-07-06_16-46-30_058837_12938/logs/dashboard.log'
2022-07-06 16:46:31,076 DEBUG gcs_utils.py:224 -- internal_kv_put b'CLUSTER_METADATA' b'{"ray_version": "1.13.0", "python_version": "3.9.13"}' True b'cluster'
2022-07-06 16:46:31,096 DEBUG gcs_utils.py:207 -- internal_kv_get b'dashboard' b'dashboard'
2022-07-06 16:46:31,202 DEBUG gcs_utils.py:207 -- internal_kv_get b'dashboard' b'dashboard'
2022-07-06 16:46:31,307 DEBUG gcs_utils.py:207 -- internal_kv_get b'dashboard' b'dashboard'
2022-07-06 16:46:31,413 DEBUG gcs_utils.py:207 -- internal_kv_get b'dashboard' b'dashboard'
2022-07-06 16:46:31,519 DEBUG gcs_utils.py:207 -- internal_kv_get b'dashboard' b'dashboard'
2022-07-06 16:46:31,520 ERROR services.py:1488 -- Failed to start the dashboard: Failed to start the dashboard, return code 1
Failed to read dashboard log: [Errno 2] No such file or directory: '/tmp/ray/session_2022-07-06_16-46-30_058837_12938/logs/dashboard.log'
2022-07-06 16:46:31,520 ERROR services.py:1489 -- Failed to start the dashboard, return code 1
Failed to read dashboard log: [Errno 2] No such file or directory: '/tmp/ray/session_2022-07-06_16-46-30_058837_12938/logs/dashboard.log'
Versions / Dependencies
1.13
Reproduction script
import ray
ray.init()
@ray.remote
def f(x):
return x * x
futures = [f.remote(i) for i in range(4)]
print(ray.get(futers)) # [0, 1, 4, 9]
macOS, full trace back:
2022-07-06 17:00:23,648 ERROR services.py:1488 -- Failed to start the dashboard: Failed to start the dashboard, return code 1
Failed to read dashboard log: [Errno 2] No such file or directory: '/tmp/ray/session_2022-07-06_17-00-22_187900_14307/logs/dashboard.log'
2022-07-06 17:00:23,648 ERROR services.py:1489 -- Failed to start the dashboard, return code 1
Failed to read dashboard log: [Errno 2] No such file or directory: '/tmp/ray/session_2022-07-06_17-00-22_187900_14307/logs/dashboard.log'
Traceback (most recent call last):
with open(dashboard_log, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/ray/session_2022-07-06_17-00-22_187900_14307/logs/dashboard.log'
Issue Severity
High: It blocks me from completing my task.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 25 (11 by maintainers)
Ah, got it.
pip uninstall grpcioand thenconda install grpcioworked for me.pip install grpcio == 1.49.1 pip uninstall -y ray pip install -U “ray[default]”
Appears so far to have solved all of the problems I was observing. However, the errors have a very flaky nature. I’ll reply to this thread if I’ll observe the same problems with additional use. Thanks for the help!
@deepwonwoo I think your issue is a separate issue from the dashboard (it is a log monitor). Do you mind creating a separate issue for that?