pkg: ESM Modules not supported

What version of pkg are you using?

5.11.1

What version of Node.js are you using?

16.19.1

What operating system are you using?

macOS

What CPU architecture are you using?

x86_64

What Node versions, OSs and CPU architectures are you building for?

node16-macos-x64

Describe the Bug

This issue is mentioned in the original package, but the package is archived, then I found a solution in issue’s comment: https://github.com/vercel/pkg/issues/1936

"pkg": {
   "scripts": [
     "node_modules/axios/dist/node/*",
   	[...]
   ]

But it still does not work

Expected Behavior

Compile app without axios (lastest version) error

To Reproduce

A Project that include the lastest version of axios (e.g: v1.6.4)

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 1
  • Comments: 41 (18 by maintainers)

Most upvoted comments

Let’s use this issue for all the others. Seems that starting from nodejs 18.19.0 and for all nodejs 20+ versions some modules that support both esm and cjs are not bundled correctly. This is IMO due to resolve package that is not able to correctly parse package.json exports fields (see issue).

Let’s use this issue as a tracker for such issues instead of having multiple open ones.

The error will look like:

Error [ERR_REQUIRE_ESM]: require() of ES Module C:\snapshot\my-yao-pkg\node_modules\@apollo\server\dist\cjs\index.js from C:\snapshot\my-yao-pkg\main.cjs not supported.
index.js is treated as an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which declares all .js files in that package scope as ES modules.
Instead either rename index.js to end in .cjs, change the requiring code to use dynamic import() which is available in all CommonJS modules, or change "type": "module" to "type": "commonjs" in C:\snapshot\my-yao-pkg\node_modules\@apollo\server\package.json to treat all .js files as CommonJS (using .mjs for all ES modules instead).

    at Module.require (pkg/prelude/bootstrap.js:1851:31)
    at Object.<anonymous> (C:\snapshot\my-yao-pkg\main.cjs)
    at Module._compile (pkg/prelude/bootstrap.js:1930:22) {
  code: 'ERR_REQUIRE_ESM'
}

Alternative solutions

Below some solutions I tried and other possible working ones

[recommended] Use node --experimental-require-module option

In most cases adding --options experimental-require-module to pkg command line will solve the issue. This will not work if the imported module uses top level awaits

Using esbuild

Actually a possible solution is to use esbuild to pre-compile the bundle in cjs and then use pkg on the output bundle file. I have a working example of this in this repo: https://github.com/zwave-js/zwave-js-ui/blob/master/esbuild.js

Note that this solution doesn’t only solve this issue but also produces very small binaries. That example also handles .node native addons, them require a little monkey patch on the file that is described in the link above. Feel free to add some more context to this.

Using bpkg

Other possible solution is to use bpkg tool instead of esbuild. I still never tried it so if someone does please give me some feedbacks. It should handle node native addons out of the box better then esbuild as it encodes them in base64 into the bundle.

Use Nodejs Single executable

Docs: https://nodejs.org/api/single-executable-applications.html.

I belive a lot in this feature and I think it will be the future of packaging nodejs applications. Anyway it’s still experimental, if you would like to give it a try please add your feedback here.

@SagePacheco long story, when I started this fork I know it would have been a pain but didn’t thought like this (reason why most of people maintaining this project have give up at a certain point)

I spent weeks on this PR in order to fix compatibility with node 20.14 and ended up being able to make everything working except for windows that actually builds but fails on runtime (check conversation if you are curious).

After many thoughts on this I decided to just merge the PR so users are able to use latest nodejs versions with linux and mac (hope someone will help to fix windows one once issues start) then I discovered that GH removed macos-11 from runners, so I bumped macos runners to use macos-13 and guess what? now mac build are not working and the sad thing is that they just hang without any output.

I’m a bit stuck now as the free time I have to invest in this is very limited and the tests here require long waiting time. I hope someone could help with this

Someone wants to give this a try? https://github.com/yao-pkg/pkg/pull/110 🚀

I have a CommonJS application that depends on axios and @inquirer/prompts and I have been able to package it like this:

esbuild index.js --bundle --platform=node --target=node20 --outfile=my-app.js
pkg my-app.js --targets node20-win-x64 -o my-app.exe

pkg 5.13.0 is ou now with nodejs 20.17.0 and 18.20.4 support 🎉

Just a quick update here. I finally managed to make all build working and new release is coming with latest nodejs 18 and 20.17.0 support 🎉

Just need to wait for this to finish now

Please consider sending some support here: https://github.com/sponsors/robertsLando 🙏🏼 ❤️

v20.17.0 is now released and has --experimental-require-module.

Let’s use this issue for all the others. Seems that starting from nodejs 18.19.0 and for all nodejs 20+ versions some modules that support both esm and cjs are not bundled correctly. This is IMO due to resolve package that is not able to correctly parse package.json exports fields (see issue).

Let’s use this issue as a tracker for such issues instead of having multiple open ones.

The error will look like:

Error [ERR_REQUIRE_ESM]: require() of ES Module C:\snapshot\my-yao-pkg\node_modules\@apollo\server\dist\cjs\index.js from C:\snapshot\my-yao-pkg\main.cjs not supported.
index.js is treated as an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which declares all .js files in that package scope as ES modules.
Instead either rename index.js to end in .cjs, change the requiring code to use dynamic import() which is available in all CommonJS modules, or change "type": "module" to "type": "commonjs" in C:\snapshot\my-yao-pkg\node_modules\@apollo\server\package.json to treat all .js files as CommonJS (using .mjs for all ES modules instead).

    at Module.require (pkg/prelude/bootstrap.js:1851:31)
    at Object.<anonymous> (C:\snapshot\my-yao-pkg\main.cjs)
    at Module._compile (pkg/prelude/bootstrap.js:1930:22) {
  code: 'ERR_REQUIRE_ESM'
}

Alternative solutions

Below some solutions I tried and other possible working ones

Using esbuild

Actually a possible solution is to use esbuild to pre-compile the bundle in cjs and then use pkg on the output bundle file. I have a working example of this in this repo: https://github.com/zwave-js/zwave-js-ui/blob/master/esbuild.js

Note that this solution doesn’t only solve this issue but also produces very small binaries. That example also handles .node native addons, them require a little monkey patch on the file that is described in the link above. Feel free to add some more context to this.

Using bpkg

Other possible solution is to use bpkg tool instead of esbuild. I still never tried it so if someone does please give me some feedbacks. It should handle node native addons out of the box better then esbuild as it encodes them in base64 into the bundle.

Use Nodejs Single executable

Docs: https://nodejs.org/api/single-executable-applications.html.

I belive a lot in this feature and I think it will be the future of packaging nodejs applications. Anyway it’s still experimental, if you would like to give it a try please add your feedback here.

While still not an ideal solution, I had similar issues upgrading from Node 18 to Node 20 and was able to get basic support for a pkg built executable with native node modules, embedded C++ binaries, and other JS dist assets bundled and functioning with minimal effort using https://github.com/yao-pkg/pkg/issues/10#issuecomment-1938585889

I still agree that for existing known mitigations, refactoring to use ESBuild may prove to be a better long term tactic.

@laurencefass it could with some limitations like top level await is not supported

@jameskhedley You have to use esbuild to bundle your application before packaging

Seems the majority of problems solved with experimental-require-module

Here is a working docker file which I use for my project:

FROM node:20.17.0-alpine as build

# prepare
RUN mkdir -p app
WORKDIR /app

# install dependencies
RUN npm i -g @yao-pkg/pkg@5.15.0
COPY . .
RUN npm i
# build binary
RUN pkg -C GZip --options experimental-require-module --targets node20-alpine -o app-alpine .

FROM node:20.17.0-alpine as release

# for alpine
RUN apk --no-cache add ca-certificates && apk --update --no-cache add curl tzdata bash

WORKDIR /app
COPY --from=build /app/app-alpine ./

# run
# CMD ["bash", "-c", "./app-alpine"]
ENTRYPOINT ["/app/app-alpine"]

@SagePacheco thanks for looking at that, was on my TODOs list but no idea when and if I would ever found the time to look at it. ATM the best way (it’s what I do on my projects) is to pre-compile everything with esbuild and I have to say that’s very powerful as you also reduce binary size a lot

Hey there, following the post of @CMCDragonkai I also tried to move our CLI to use ESM with esbuild (on node18).

I could manage to get the full flow working, including with native librairies.

The build script is pretty similar to the one in Polykey: https://github.com/Dashlane/dashlane-cli/blob/master/scripts/build.js I had to do a few changes to make it work well.

I could use got and chai with ESM without problems.

One caveat I encountered is that esbuild is not transpiling the dynamic imports await import into require when bundling to CJS so you have to force the usage of require() in your code if you need this. In my case I have platform specific dependencies to manage so it’s important.

If you have time to take a look, I’m really looking for feedback 😃