bullmq: Bug? - Error in processor permanently breaks entire queue
Consider the following setup:
// Queuer
import { Queue } from 'bullmq'
const queue = new Queue('mailer'),
mails = [
{
from: "manast@taskforce.sh",
subject: "This is a simple test",
text: "An email sent using BullMQ",
to: 'user@localhost',
attachments: [{ filename: 'test.txt', content: Buffer.from('hello world!', 'utf-8') }]
},
{
from: "manast@taskforce.sh",
subject: "This is a simple test",
text: "An email sent using BullMQ",
to: 'user@localhost',
attachments: []
},
{
from: "manast@taskforce.sh",
subject: "This is a simple test",
text: "An email sent using BullMQ",
to: 'user@localhost',
attachments: []
},
{
from: "manast@taskforce.sh",
subject: "This is a simple test",
text: "An email sent using BullMQ",
to: 'user@localhost',
attachments: []
}
]
;(async () => {
for (let mail of mails) {
await queue.add('mails', mail, {
attempts: 0,
backoff: {
type: 'exponential',
delay: 1000
}
})
}
})
// Logger
import { QueueEvents } from 'bullmq'
const queueEvents = new QueueEvents('mailer')
queueEvents.on('failed', async ({ jobId, failedReason }) => {
console.log(`Job ${jobId} permanantly failed, because: ${failedReason}`)
})
// Processor
import nodemailer from 'nodemailer'
import { Job } from 'bullmq'
const transporter = nodemailer.createTransport({ sendmail: true, newline: 'unix' })
export default async (job: Job) => {
const response = await transporter.sendMail(job.data)
console.log(job.id, response.messageId)
}
When running this, the queue breaks when the first erroneous mail is processed. The โloggerโ will console.log something like Failed job 9880, because: The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. Received an instance of Object, which is correct and expected, but all subsequent jobs will fail, too. Even though they are correct and Nodemailer doesnโt throw an error here.
Am I missing something?
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 18 (6 by maintainers)
Great, thanks for following up and making BullMQ even more robust!
๐ This issue has been resolved in version 1.64.1 ๐
The release is available on:
Your semantic-release bot ๐ฆ๐
I found that the callstack for the error looks like this:
So this adds to the assumption that it is a bug in node, since the error is produced in the core and probably breaks V8. Now the question is if using a sandboxed processor in BullMQ could somehow detect this and spawn a new child process, I am spending some time to investigate if this is possible or not.