syne-tune: Promotion Logic Bug
There seems to be a problem with the Hyperband promotion logic.
How to reproduce:
Add type="promotion" to https://github.com/awslabs/syne-tune/blob/main/benchmarking/nursery/benchmark_automl/baselines.py#L69
Run python benchmarking/nursery/benchmark_automl/benchmark_main.py --num_seeds 1 --method ASHA --benchmark lcbench-airlines
File "/syne-tune/benchmarking/nursery/benchmark_automl/benchmark_main.py", line 209, in <module>
tuner.run()
File "/syne-tune/syne_tune/tuner.py", line 240, in run
raise e
File "/syne-tune/syne_tune/tuner.py", line 175, in run
new_done_trial_statuses, new_results = self._process_new_results(
File "/syne-tune/syne_tune/tuner.py", line 345, in _process_new_results
done_trials_statuses = self._update_running_trials(
File "/syne-tune/syne_tune/tuner.py", line 465, in _update_running_trials
decision = self.scheduler.on_trial_result(trial=trial, result=result)
File "/syne-tune/syne_tune/optimizer/schedulers/hyperband.py", line 779, in on_trial_result
task_info = self.terminator.on_task_report(trial_id, result)
File "/syne-tune/syne_tune/optimizer/schedulers/hyperband.py", line 1124, in on_task_report
rung_sys.on_task_report(trial_id, result, skip_rungs=skip_rungs)
File "/syne-tune/syne_tune/optimizer/schedulers/hyperband_promotion.py", line 221, in on_task_report
assert resource == milestone, (
AssertionError: trial_id 1: resource = 4 > 3 milestone. Make sure to report time attributes covering all milestones```
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 16
Commits related to this issue
- Fix bug in _BlackboxSimulatorBackend for pause/resume scheduling (issue #304) — committed to awslabs/syne-tune by mseeger 2 years ago
- Fix bug in _BlackboxSimulatorBackend for pause/resume scheduling (iss… (#317) * Fix bug in _BlackboxSimulatorBackend for pause/resume scheduling (issue #304) * Small fix — committed to awslabs/syne-tune by mseeger 2 years ago
- Fix bug in _BlackboxSimulatorBackend for pause/resume scheduling (iss… (#317) * Fix bug in _BlackboxSimulatorBackend for pause/resume scheduling (issue #304) * Small fix — committed to awslabs/syne-tune by mseeger 2 years ago
I can give this a try