bullmq: [Bug]: Sandboxed processors, require() of ES Module not supported
Version
v3.7.0
Platform
NodeJS
What happened?
I’m trying to use Sandboxed processors, I made separate file processor.js:
import { Job } from "bullmq";
export default async function (job) {
await job.log("Start processing job");
console.log("Doing something useful...", job.id, job.data);
}
but always receive an error in bullmq:
Error [ERR_REQUIRE_ESM]: require() of ES Module /root/registration/utils/processor.js from /root/registration/node_modules/bullmq/dist/cjs/classes/child-processor.js not supported.
Instead change the require of processor.js in /root/registration/node_modules/bullmq/dist/cjs/classes/child-processor.js to a dynamic import() which is available in all CommonJS modules.
at ChildProcessor.init (/root/registration/node_modules/bullmq/dist/cjs/classes/child-processor.js:25:25)
at process.<anonymous> (/root/registration/node_modules/bullmq/dist/cjs/classes/master.js:12:38)
at process.emit (node:events:513:28)
I’m not sure, but I think problem here: https://github.com/taskforcesh/bullmq/blob/master/src/classes/child-processor.ts#:~:text=processor%20%3D%20require(processorFile)%3B
How to reproduce.
No response
Relevant log output
No response
Code of Conduct
- I agree to follow this project’s Code of Conduct
About this issue
- Original URL
- State: closed
- Created a year ago
- Reactions: 11
- Comments: 31 (12 by maintainers)
Can you try removing
"type": "module"from the package.json?Also can reproduce. Attempting to use Sandboxed Jobs fails due to incompatibility with pure ESM modules. I think that Bull MQ should ideally be compatible with pure ESM modules (i.e.
"type": "module"inpackage.json).same problem here… my whole project it’s using esm and therefore I can’t use sandboxed workers this way
In my second day of converting my microservice to use bullmq, I hit this, and tried to find the reason for a couple of hours, then I find this issue post 😦
My whole node/express microservices stack is typescript / esbuild / esm modules.
I tried to compile Sandboxed processors separately with esbuild as cjs, and I could! And of course, I hit another error. All my libraries also should also be exposing commonjs versions now 😦
And this:
Very sad…
@manast I still didn’t solve it. In my case I need to use type script, because another library that I use requires “type”: “module” in package.json. On the one hand it requires “type”: “module”, on the other hand (bullMQ) it requires to use “require” and remove “type”: “module”
Any updates on this? I cannot use the workaround of removing
"type": "module"from package.json, other thirdparty libraries that I use requires it. 😦This should do it: https://github.com/taskforcesh/bullmq/pull/2318
This can work on simple stuff, but when you have complex structures where you depend on other modules it fails. I have my tasks defined in separate modules, which use custom or other libraries, so it is no use.
I’m also having the same issue. It’s quite inconvenient because I’m also using other libraries that only use esm…
https://github.com/taskforcesh/bullmq/pull/2315
In theory this should work both in commonjs as in ESM:
So I think it is safe to do this change.
Closing as it seems to be resolved now.
Working for me now. Nice job!
Updated my example to use 4.15.1 and getting the same thing - https://github.com/dholst/bullmq-esm-processor
I don’t envy package publishers - https://blog.isquaredsoftware.com/2023/08/esm-modernization-lessons/
Hmm, it could be the import at the top that breaks it:
Can you try to use require instead of import?