nx: @nrwl/node:node executor does not support importing ESM libs

Same here. Right now my whole “Nx experience” feels like product owners went all crazy with plugins, “build anything” capabilities, and left behind the simplicity and truly greatness of nx: its tools for managing development directories.

I’ve got a ground-breaking idea, instead of NX constantly bring out more and more new feature’s, maybe start triaging issues that are older than 12 months and FIX THEM FIRST. You can add all the new hotness you want, but what bloody good is it when folks get legitimately blocked on issues like this? ill show myself out…

Thanks @beeman for your insight - it does help to get this perspective on things and is much appreciated!

A few thoughts:

I think this might be a bit of a too-simple view of things

Yes, my view of things might be simple and simply off-base even?!? Hence my line of questioning 😃

To me, this Nx error:

Error [ERR_REQUIRE_ESM]: require() of ES Module

suggests that Nx code (@nx/js/src/executors/node/node-with-require-overrides.js) is erroneously trying to require an ESM module (it even tells you that!) instead doing the correct thing and importing it.

This is nothing to do with Nest.js per se, or really any specific framework or library. It’s to do with the general case of libraries that export ESM-style. Specifically that Nx code isn’t correctly importing libraries that export ESM-style. That seems to be an Nx bug!

it depends on what you want

My wants are quite simple. I want: (1) An Nx setup where a import statements in my source code work for third party libraries that use CommonJS exports and for those that use ESM exports. (2) I don’t want unexpected surprises like this error (it was an unexpected and unwanted surprise!) I don’t even care about my transpiled output format, as long as it runs correctly and I can get source-map debugging going with it. (3) And I want to minimise the amount of time required to get to such an Nx setup - for me and for developers everywhere! 😃

Not a lot that Nx can do here

Here’s where I’d have a slightly different take on it. A healthy number of developers have contributed text or emoji-love to comments in this thread, suggesting there is at least a reasonable interest in this topic. The Nx developers (hi!) at the very least could: (a) clarify their position on this issue (e.g. Is it an Nx bug? Is it a configuration problem? Is it due to a bug/shortcoming in an Nx dependency like esbuild? etc.), and (b) also give developers a steer on a solution.

I feel like I’m floundering a bit here, and I’m wondering if others are as well?

I just had to update one of my dependencies, which is now ESM-only. I have yet to find a way to make it work with nx, and I’m wondering if there is even a viable workaround for this?

We are running into this same issue with nanoid 4.0 and other ESM only libraries. This is a major blocker for us!

It got fixed by including “find-cache-dir”: “^3.3.2” package into the project.

Yeah, had the same thing today. More and more libraries will switch to ESM only and Nx will break more and more and ugly workarounds like this have to be created… Where can I find the roadmap regarding Nx and ESM? Our company has to invest more and more into keeping the stuff running.

My current solution for an esm module built from esbuild is to update node-with-require-overrides.js and use patch-package https://github.com/nrwl/nx/pull/10414#issuecomment-1369314024

// node-with-require-overrides.js line 19

// require(fileToRun);
import(fileToRun).catch((error) => {
  console.error('Error importing main module:', error);
});

I encountered this same problem when running my project that includes the trading-signal npm library (version 4.0.0):

/home/mark/myrepo/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js:18
        return originalLoader.apply(this, arguments);
                              ^
Error [ERR_REQUIRE_ESM]: require() of ES Module /home/mark/myrepo/node_modules/trading-signals/dist/index.js from /home/mark/myrepo/dist/apps/myapp/main.js not supported.
Instead change the require of index.js in /home/mark/myrepo/dist/apps/myapp/main.js to a dynamic import() which is available in all CommonJS modules.

For reference my npx nx report is:

mark@mymachine:~/myrepo$ npx nx report
<...snip...>

   Node   : 20.5.1
   OS     : linux-x64
   npm    : 10.4.0
   
   nx                 : 17.3.1
   @nx/js             : 17.3.1
   @nx/jest           : 17.3.1
   @nx/linter         : 17.3.1
   @nx/eslint         : 17.3.1
   @nx/workspace      : 17.3.1
   @nx/devkit         : 17.3.1
   @nx/esbuild        : 17.3.1
   @nx/eslint-plugin  : 17.3.1
   @nx/node           : 17.3.1
   @nrwl/tao          : 17.3.1
   @nx/webpack        : 17.3.1
   typescript         : 5.3.3

The trading-signals library switched to ESM imports in version 4.0.0.

My searching found this thread and I attempted these solutions from it:

1. https://github.com/nrwl/nx/issues/11335#issuecomment-1518373862 from @danielmahon

The structure of my file node_modules/@nx/js/src/executors/node/node-with-require-overrides.js seemed different and so I couldn’t make the same code substitution in this file, so this solution didn’t work for my case.

1. https://github.com/nrwl/nx/issues/11335#issuecomment-1652139636 from @christopher-caldwell

I studied the repo for what might have made the working branch work, but couldn’t reproduce the success! @christopher-caldwell - were you able to pinpoint the actual fix in your working branch?

1. https://github.com/nrwl/nx/issues/11335#issuecomment-1754091042 from @beeman

I tried to switch executor from webpack to esbuild, and “format” from “cjs” to “esm”. I encountered a condition where local imports only worked with a “.js” file extension, and could not work out how to fix this, so I couldn’t get this to work. (@danielmahon also mentioned this solution here: https://github.com/nrwl/nx/issues/11335#issuecomment-1756172475)

1. https://github.com/nrwl/nx/issues/11335#issuecomment-1764875067 from @dmastag

This looked NestJS-specific, and I was unsure if this translated to a more generalised solution (e.g. I don’t use NestJS)

1. Rolling back to a previous version of the trading-signals npm (3.7.0) before its ESM refactor.

This produced a working solution, albeit a non-ideal one. Any solution that sacrifices potential module updates or fixes - particularly security fixes - is not ideal by any stretch!

So the time spent trying different solutions to this problem has probably been around 4 hours. I’m not sure if this would be a typical developer experience or not in this case?!?

So this raised for me some genuine questions about what seems to me to be a serious shortcoming in the Nx tooling such that ESM modules are not usable (this may be too strong a statement, but it fits my current experience - would be over the moon to be proven wrong though if I’m missing an obvious solution!). And this has been a documented and unfixed problem for around 18 months now. Although there is a healthy interest in this thread, there doesn’t seem to be an urgency to fix.

(a) Are there not many ESM libraries out there, so that this is not hitting many developers yet and therefore not that important yet? (despite a general consensus that ESM-style imports are preferred over CommonJS-style going forward) (b) Are developers hitting these problems and rolling back their third-party library versions to pre-ESM versions? (like I did! That seems bad though!) © Are developers finding other workarounds, or getting some of the above solutions (or other solutions) to this working?

And none of this is a slight on Nx developers - I see 722 open issues as I type this, so hands will be very full out there, prioritisation will obviously be a necessity, and further I’m unfortunately in no position to understand let alone develop a potential PR for this! Just trying to get a bit of mental clarity on the state of play here!

Sorry for the long write-up - congratulations if you made it to the end! 😃

@sancelot I made a reproduction repo showing the issue (for me at least) https://github.com/christopher-caldwell/esm-require

I’ve landed on this method of importing esm libs:

export const dynamicImport = new Function(
  'specifier',
  'return import(specifier)'
) as <T = never>(specifier: string) => Promise<T>;

usage:

await dynamicImport<typeof import('thumbhash')>('thumbhash');

would be cool to not have to do so, but it works for now

it’s also how nx does it internally: https://github.com/nrwl/nx/blob/7a7cbeca44e44ff62ae3ab120bccb0850e09884c/packages/js/src/executors/node/node-with-require-overrides.ts#L5

Here’s another repro with the steps taken from the offical docs for nx (without any changes + installing chalk): https://github.com/nicu-chiciuc/repro-nx-chalk

At this point, I’m thinking about switching from Nx to a different stack, since I’ve been stuck on these issues for several weeks.

Tested @christopher-caldwell against the new 16.0.0-beta.1 and still fails with the same error:

[ watch ] build succeeded, watching for changes...
Debugger listening on ws://localhost:9229/e5b6f3bd-c0db-4a6f-b2b6-766685931683
For help, see: https://nodejs.org/en/docs/inspector

/home/caio/github/christopher-caldwell/esm-require/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js:16
        return originalLoader.apply(this, arguments);
                              ^
Error [ERR_REQUIRE_ESM]: require() of ES Module /home/caio/github/christopher-caldwell/esm-require/dist/packages/api/main.js from /home/caio/github/christopher-caldwell/esm-require/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js not supported.
Instead change the require of main.js in /home/caio/github/christopher-caldwell/esm-require/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js to a dynamic import() which is available in all CommonJS modules.
    at Module._load (/home/caio/github/christopher-caldwell/esm-require/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js:16:31)
    at Object.<anonymous> (/home/caio/github/christopher-caldwell/esm-require/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js:19:1)

Do we know if this will be fixed any time soon? Any timeline?

any updates ? Hitting the same issue

@Mydayyy this one has me beat 😮‍💨

I tried a bunch of different things to try and get the output to support dynamic imports, but nothing worked

• tsconfig.app.json: set module to “esnext”
• webpack.config.js: set output.environment.dynamicImport to true

Then updated the source code to this:

const { fileTypeFromFile } = await import(/* webpackIgnore: true */ 'file-type');

And yet even with these changes (and a few others I tried) something in the nx/nest/webpack/ts stack is stubbornly transpiling the dynamic import.

Is there any way you can take the nest starter template and replace webpack with esbuild, or does nest require the use of the webpack bundler?

Same issue with the ESM-only library “file-type”. Bummed to see this is an issue for several years now.

As I write this comment the issue still persists with NX 15.7.2 and TypeScript 4.9.5.

Same for both @keycloak/keycloak-admin-client@21.0.0 and nanoi@4.0.1.

Also out of curiosity, I tried the new serve executor @nrwl/js:node / build executor @nrwl/esbuild:esbuild and the result is the same.

I’m not sure @alejandrombc why you cannot reproduce the issue because there is nothing really custom setup. You get the error straight away with all default settings.

If I’m getting more feedback of not replicable I will consider spending some time to create a repo. Although I doubt how useful will that be since everything is default NX settings.

Same here with nanoid version 4.0.1 😕, any news?

@alejandrombc nobody can solve your problem, if you don’t provide sample code repository AND environment . As I wrote it works.

That depends on your project setup. something is missing or badly setted up …

Hello! Any update on this? Still can’t find a way to make this work 😓