bullmq: timeout still working?

@adamreisnz I think I can pull out something soon that is good enough.

I’m doing something like this:

new Worker('paint', async () => {
  await Promise.race([
    timeout(5000),
    doSomeAsyncTask()
  ])
})

async function timeout(ms: number) {
  return new Promise((resolve, reject) => {
    setTimeout(() => reject(), ms)
  })
}

This only implements a timeout inside a single run of the worker task. What I expected timeout to do and would need, is for the message to be removed from the queue, if it hasn’t been (successfully) processed, including wait & retries, for a certain amount of time. What you are describing sounds more like the stalled job feature to me. “When a worker is not able to notify the queue that it is still working on a given job, that job is moved back to the waiting list.” Assuming retries are not exhausted, that is exactly what your timeout reject would do, at least from a use case perspective, right?

So what is the plan with this? Why does this settings exists if it is not expected to work now? It took me hours to find out that this is just a dummy setting.

I thinktimeout prop should be either completely removed to reduce confusion or implemented in a way @acro5piano suggested (with a potential side-effect warning).

I’m doing something like this:

new Worker('paint', async () => {
  await Promise.race([
    timeout(5000),
    doSomeAsyncTask()
  ])
})

async function timeout(ms: number) {
  return new Promise((resolve, reject) => {
    setTimeout(() => reject(), ms)
  })
}

I’ve scanned through all of the comments and can’t find a clear answer about whether BullMQ timeouts are working or not, is there a conclusion we can draw?

@autarchprinceps Yeah, as other people mentioned, that implementation just tells BullMQ “the job failed” and the job is still running even if it was failing. However it should be better than nothing reported.

I strongly agree with @motiejunas as the current type definition of the option is confusing. It actually doesn’t work but it has the option.

@motiejunas yes, we should take care of this ASAP. I think we will remove this setting entirely but maybe add another one that works in a different way, not as a timeout that kills the current job that is being processed but that fails it directly if the job is too old when it arrives at the worker. For a clean mechanism to actually timeout jobs I would refer to the Pro version, where we support observables and can therefore cleanly cancel a running job.

@acro5piano it is important to point out that even though the job will “fail” if it times out, the “doSomeAsyncTask” will continue to work with potential side-effects as a result… if this is acceptable or not depends on what that function is doing.

@acro5piano that’s an interesting solution. Note that if using Bluebird promises, you can also simply use Promise.timeout() as the second array item for Promise.race()

mark