next.js: [ERR_REQUIRE_ESM]: require() of ES Module `shiki` when using `14.2.x`

Link to the code that reproduces this issue

https://github.com/lobehub/lobe-chat/tree/arvinxx-patch-1

To Reproduce

Clone Repo

git clone https://github.com/lobehub/lobe-chat

Checkout branch

git checkout arvinxx-patch-1

install deps

pnpm install / bun install

run build

npm run build

Current vs. Expected behavior

with next@14.1.4, this can be build success.

but with next@14.2.1, the build is failed. the console error:

Error [ERR_REQUIRE_ESM]: require() of ES Module /Users/arvinxx/CodeProjects/LobeHub/lobe-chat/node_modules/.pnpm/shiki@1.2.1/node_modules/shiki/dist/index.mjs not supported.
Instead change the require of /Users/arvinxx/CodeProjects/LobeHub/lobe-chat/node_modules/.pnpm/shiki@1.2.1/node_modules/shiki/dist/index.mjs to a dynamic import() which is available in all CommonJS modules.
    at mod.require (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/node_modules/.pnpm/next@14.2.1_@babel+core@7.23.6_react-dom@18.2.0_react@18.2.0/node_modules/next/dist/server/require-hook.js:65:28)
    at 13076 (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/app/chat/(mobile)/mobile/page.js:1:835)
    at t (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/webpack-runtime.js:1:143)
    at 14730 (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/chunks/3251.js:11:494067)
    at t (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/webpack-runtime.js:1:143)
    at 73251 (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/chunks/3251.js:11:473537)
    at t (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/webpack-runtime.js:1:143)
    at 83337 (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/chunks/2596.js:6:28070)
    at t (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/webpack-runtime.js:1:143)
    at 793 (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/chunks/4905.js:1:36842)
    at t (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/webpack-runtime.js:1:143)
    at 78518 (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/chunks/1680.js:79:4281)
    at t (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/webpack-runtime.js:1:143)
    at 40921 (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/chunks/1680.js:215:508)
    at t (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/webpack-runtime.js:1:143)
    at 70080 (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/app/chat/(mobile)/mobile/page.js:6:250)
    at Object.t [as require] (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/.next/server/webpack-runtime.js:1:143)
    at require (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/node_modules/.pnpm/next@14.2.1_@babel+core@7.23.6_react-dom@18.2.0_react@18.2.0/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:16:18270)
    at I (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/node_modules/.pnpm/next@14.2.1_@babel+core@7.23.6_react-dom@18.2.0_react@18.2.0/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:94362)
    at /Users/arvinxx/CodeProjects/LobeHub/lobe-chat/node_modules/.pnpm/next@14.2.1_@babel+core@7.23.6_react-dom@18.2.0_react@18.2.0/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:96668
    at F._fromJSON (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/node_modules/.pnpm/next@14.2.1_@babel+core@7.23.6_react-dom@18.2.0_react@18.2.0/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:97106)
    at JSON.parse (<anonymous>)
    at O (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/node_modules/.pnpm/next@14.2.1_@babel+core@7.23.6_react-dom@18.2.0_react@18.2.0/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:94083)
    at t (/Users/arvinxx/CodeProjects/LobeHub/lobe-chat/node_modules/.pnpm/next@14.2.1_@babel+core@7.23.6_react-dom@18.2.0_react@18.2.0/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:100582)
 ✓ Generating static pages (17/17)

> Export encountered errors on following paths:
        /chat/(mobile)/mobile/page: /chat/mobile
        /settings/agent/page: /settings/agent
 ELIFECYCLE  Command failed with exit code 1.

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000
  Available memory (MB): 65536
  Available CPU cores: 10
Binaries:
  Node: 20.9.0
  npm: 9.2.0
  Yarn: 1.22.19
  pnpm: 8.15.3
Relevant Packages:
  next: 14.2.1 // Latest available version is detected (14.2.1).
  eslint-config-next: N/A
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.4.5
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

App Router, Module resolution (CJS / ESM, module resolving)

Which stage(s) are affected? (Select all that apply)

next build (local)

Additional context

I have check the Next.js 14.2.1-canary.3. this issue is still remain

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 3
  • Comments: 17 (6 by maintainers)

Commits related to this issue

Most upvoted comments

Let me introduce a more easy way to fix it:

// next.config.mjs


const nextConfig = {
  // ... other config

  experimental: {
    optimizePackageImports: [
       // ... other packages
+      'shiki',
    ],

  // ...
}

// ...

Adding “shiki” to experimental.optimizePackageImports didn’t work for me. It just gave a different error message: TypeError: Cannot read properties of undefined (reading 'transformers') on codeToHtml() calls.

Adding “shiki” to transpilePackages did fix the issue for me:

next.config.js:

module.exports = {
  transpilePackages: ['components', 'shared', 'shiki'],
};

Amazing, thanks for confirming. I’ll close this as resolved!

Hi @huozhi, good to let me know this good news. I’ve made a new branch: upgrade-to-v14_2_8 (no pkg optimization, no transpile pkg)

Work as intended, cool 🎉

2024-09-05_no_esm_pkgs_errs copy

Currently this is the only solution, you can only import in async to ignore errors.

I’m looking at this as well.

CleanShot 2024-04-18 at 10  24 19@2x

Edit:

Top level async/await will get a warning. So we can use use hook.

Use it in async component or dynamic import this component.



const codeHighlighterPromise = (async () => {
  if (isServerSide) return
  const [{ getHighlighterCore }, getWasm, { codeHighlighter }] =
    await Promise.all([
      import('shiki/core'),
      import('shiki/wasm').then((m) => m.default),
      import('./core'),
    ])

  const core = await getHighlighterCore({
    themes: [
      import('shiki/themes/github-light.mjs'),
      import('shiki/themes/github-dark.mjs'),
    ],
    langs: [],
    loadWasm: getWasm,
  })

  return {
    codeHighlighter: core,
    fn: (o: { lang: string; attrs: string; code: string }) => {
      return codeHighlighter(core, o)
    },
  }
})()
export const ShikiHighLighter: FC<Props> = (props) => {
  const { lang: language, content: value, attrs } = props
  const codeHighlighter = use(codeHighlighterPromise)
// ...
}

We had mad some module resolution improvements for ESM package on 14.2.8, the original repro branch is gone. Would someone else help check if it’s working well now without the above workaround (no pkg optimization, no transpile pkg)? Thanks 🙏 If it’s not working please lmk with your reproduction, I’d love to help take a look

Hi @arvinxx, thanks for sharing, really a good solution, nice! 👍

I add this solution to a new branch in the repo which I made for reproducing this issue.

new branch: with-optimize-package-imports

Good, after I got the solution from @Innei provided --> quick link

I tried this solution in the repo, which I made for reproducing this issue before. I made a new branch, and tried this solution, and it works. (with dynamic import)

The new branch in the repo for reproducing this issue: with-dynamic-import

Key diff code in src/app/page.tsx:

import { EditorProvider } from "@/components/editor/editorContext";
- import EditorResult from "@/components/editor/editorResult";
import EditorTextArea from "@/components/editor/editorTextArea";
+ import dynamic from "next/dynamic";

export default function Home() {
+  const EditorResult = dynamic(
+    () => import("@/components/editor/editorResult"),
+    { ssr: false }
+  );
  return (
    <main className="flex min-h-screen flex-col items-center justify-between p-24">
      <EditorProvider>
        <div className="lg:max-w-[1024px] mx-auto flex flex-col gap-5 mt-10 p-2">
          <h1>Editor</h1>
          <EditorTextArea className="text-black" />
          <article className="prose dark:text-white dark:prose-headings:text-white">
            <EditorResult />
          </article>
        </div>
      </EditorProvider>
    </main>
  );
}

Thanks @Innei pointed out the solution

https://github.com/dev-pi2pie/next-v14_2-with-rehype-pretty-code-and-shiki-transformer

I’ve made a repo which reproducing the similar issue which happened on the build stage. And this repo is a simple editor, which input something like markdown syntax to the text area and get the result back.

And the issue happened in the code:

src/lib/mdsource.ts

I think the issue is from the code:

import {
  transformerNotationDiff,
  transformerNotationWordHighlight,
  transformerNotationHighlight,
} from "@shikijs/transformers";

If I do not use the transformers in the rehype-pretty-code, and the build will be success.

However, good to see @arvinxx mentioned the issue I encountered too.