apscheduler: Repeating job with very short interval triggers exception on shutdown: 'RuntimeError: cannot schedule new futures after shutdown'
This code hits the problem pretty reliably (~75% of the time on python2, only ~60% of the time on python3 for some reason):
from apscheduler.schedulers.background import BackgroundScheduler
import time
import logging
logging.basicConfig()
def dummy_job():
print("test")
scheduler = BackgroundScheduler()
executors = {
'default': {
'type': 'threadpool',
'max_workers': 20
}
}
scheduler.configure(executors=executors)
scheduler.start()
scheduler.add_job(func=dummy_job, trigger='interval', seconds=0.05)
time.sleep(0.5)
scheduler.shutdown()
This code gives the following output when run:
test
test
test
test
test
test
test
test
test
ERROR:apscheduler.scheduler:Error submitting job "dummy_job (trigger: interval[0:00:00.050000], next run at: 2018-02-03 12:40:57 GMT)" to executor "default"
Traceback (most recent call last):
File "/data/build/ext/apscheduler/apscheduler/schedulers/base.py", line 960, in _process_jobs
executor.submit_job(job, run_times)
File "/data/build/ext/apscheduler/apscheduler/executors/base.py", line 71, in submit_job
self._do_submit_job(job, run_times)
File "/data/build/ext/apscheduler/apscheduler/executors/pool.py", line 22, in _do_submit_job
f = self._pool.submit(run_job, job, job._jobstore_alias, run_times, self._logger.name)
File "/usr/lib64/python3.3/concurrent/futures/thread.py", line 97, in submit
raise RuntimeError('cannot schedule new futures after shutdown')
RuntimeError: cannot schedule new futures after shutdown
This is a regression introduced by #268 - reverting that fix causes it to cease hitting this problem at all.
The reason we are hitting this is that we have a repeating job running with a 100ms interval whenever the scheduler is running. Since upgrading to v3.5.1 we hit this error every time the scheduler is stopped.
I’m not sure what the right fix is - obviously you can’t just revert #268. Maybe BaseScheduler._process_jobs
could check whether the scheduler is running before it does anything else?
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 9
- Comments: 22 (8 by maintainers)
Commits related to this issue
- Fixed RuntimeError when shutting down the scheduler from a scheduled job Fixes #285. — committed to agronholm/apscheduler by agronholm 2 years ago
- Fixed RuntimeError when shutting down the scheduler from a scheduled job Fixes #285. — committed to RobusGauli/apscheduler by agronholm 2 years ago
I’ve fixed this in the 3.x branch now (at least I can’t repro anymore). Release is imminent, but feel free to test already.
Is there any update on this issue? I’ve recently encountered this on python 3.9
I am in the process of doing a major refactoring on the codebase and I will fix this one way or the other in v4.0.
@filwillian I have no idea what you’re suggesting here.
Thank you for fixing it. Do you know when you are planning on releasing?