bullmq: [Bug]: Workers not working after redis reconnecting, even if ioredis is able to reconnect.

Version

v7.8.1

Platform

NodeJS

What happened?

When the workers are started, Redis is running. If Redis goes down, the workers (using ioredis) lose their connection. When Redis comes back online, ioredis successfully reconnects.

However, the workers do not resume processing jobs afterward.

I suspect that the IoredisListener may not handle events other than ioredis:close, but I am unsure of the exact cause. Unfortunately, I do not have any stack traces to provide further insight.

How to reproduce.

Steps to reproduce

  1. Start Redis
  2. Start Workers (the workers are working at this point)
  3. Stop Redis
  4. Wait 10s
  5. Start Redis
  6. The workers are not picking up jobs after reconnecting and worker count has become 0.
 docker run -d --name redis-small \
  --memory=200m -p 6379:6379 \
  redis:latest redis-server \
--maxmemory 200mb \
--maxmemory-policy noeviction \
--enable-debug-command yes
});
import {Queue, Worker} from 'bullmq';
import {Redis} from 'ioredis'
const queue = new Queue('queue-1');
queue.on('error', ()=>{})
const worker = new Worker('queue-1', async (job) => {
    console.log(job.data);
    return job.data;
}, {connection: new Redis({
    maxRetriesPerRequest: null
})})
worker.on("ready", () => {  console.log("🟒 Worker is ready and connected to Redis");})
worker.on("active", (job) => {  console.log(`πŸš€ Job ${job.id} is now active and being processed`);})
worker.on("completed", (job) => {  console.log(`βœ… Job ${job.id} completed successfully`);})
worker.on("failed", (job, error) => {  console.log(`❌ Job ${job.id} failed`, error.stack);})
worker.on("progress", (job, progress) => {  console.log(`⏳ Job ${job.id} progress: ${progress}%`);})
worker.on("stalled", (job) => {  console.log(`⚠️ Job ${job} is stalled`);})
worker.on("ioredis:close", () => {  console.log(`❌ Redis is not reachable`);})
worker.on("error", (error) => {  console.log(`❌ Worker Error ${error}`);})
worker.on("closing", () => {  console.log(`πŸ”΄ Worker is closing`);})
worker.on("closed", () => {  console.log(`πŸ”΄ Worker is closed`);})
worker.on("drained", () => {  console.log(`πŸ«— queue is drained`);})
worker.on("paused", () => {  console.log(`⏸ Worker is paused`);})

async function f() {
    console.log(await queue.getWorkersCount())
    // console.log(await queue.getWorkers());
}

setInterval(f, 5000);

Relevant log output

No response

Code of Conduct

  • I agree to follow this project’s Code of Conduct

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 17 (7 by maintainers)

Most upvoted comments