apscheduler: Repeating job with very short interval triggers exception on shutdown: 'RuntimeError: cannot schedule new futures after shutdown'

This code hits the problem pretty reliably (~75% of the time on python2, only ~60% of the time on python3 for some reason):

from apscheduler.schedulers.background import BackgroundScheduler
import time
import logging

logging.basicConfig()

def dummy_job():
    print("test")

scheduler = BackgroundScheduler()
executors = {
    'default': {
        'type': 'threadpool',
        'max_workers': 20
    }
}
scheduler.configure(executors=executors)
scheduler.start()

scheduler.add_job(func=dummy_job, trigger='interval', seconds=0.05)
time.sleep(0.5)
scheduler.shutdown()

This code gives the following output when run:

test
test
test
test
test
test
test
test
test
ERROR:apscheduler.scheduler:Error submitting job "dummy_job (trigger: interval[0:00:00.050000], next run at: 2018-02-03 12:40:57 GMT)" to executor "default"
Traceback (most recent call last):
  File "/data/build/ext/apscheduler/apscheduler/schedulers/base.py", line 960, in _process_jobs
    executor.submit_job(job, run_times)
  File "/data/build/ext/apscheduler/apscheduler/executors/base.py", line 71, in submit_job
    self._do_submit_job(job, run_times)
  File "/data/build/ext/apscheduler/apscheduler/executors/pool.py", line 22, in _do_submit_job
    f = self._pool.submit(run_job, job, job._jobstore_alias, run_times, self._logger.name)
  File "/usr/lib64/python3.3/concurrent/futures/thread.py", line 97, in submit
    raise RuntimeError('cannot schedule new futures after shutdown')
RuntimeError: cannot schedule new futures after shutdown

This is a regression introduced by #268 - reverting that fix causes it to cease hitting this problem at all.

The reason we are hitting this is that we have a repeating job running with a 100ms interval whenever the scheduler is running. Since upgrading to v3.5.1 we hit this error every time the scheduler is stopped.

I’m not sure what the right fix is - obviously you can’t just revert #268. Maybe BaseScheduler._process_jobs could check whether the scheduler is running before it does anything else?