ray: [Bug] ray monitor reports negative usage
Search before asking
- I searched the issues and found no similar issues.
Ray Component
Ray Core, Monitoring & Debugging
What happened + What you expected to happen
Started a script, left it run for a couple of hours, interrupted with ctrl+c, scripts exits, ray monitor shows negative usage:
Usage:
-2.0/52.0 CPU (0.0 used of 2.0 reserved in placement groups)
0.0/2.0 GPU
0.0/1.0 accelerator_type:G
0.0/1.0 accelerator_type:GT
0.00/103.633 GiB memory
0.00/48.406 GiB object_store_memory
Demands:
(no resource demands)
Reproduction script
search_optuna = optuna.OptunaSearch()
tune.run(
tune.with_parameters(trainable, data=[df.index.values, df.bid.values, df.ask.values, df.decimals_price[0]]),
name="DEMA_single_OM_process",
search_alg=search_optuna,
time_budget_s=3600*24*30,
# resume=True,
num_samples=-1,
verbose=3,
config=conf,
metric="score",
mode="max",
fail_fast=True,
reuse_actors=True
)
Anything else
ray 1.7.0
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 16 (16 by maintainers)
Btw, we will have a release for 1.8 very soon. We can revisit this if it still happens on 1.8