visx: ERR_REQUIRE_ESM in Next.js app

I’ll take a shot at fixing this over the next week or so

also looking for a solution for this. solved it temporarily by importing the component that uses @visx/xychart like so:

import dynamic from 'next/dynamic';
const MyLineChart = dynamic(() => import('./MyLineChart'), { ssr: false });

Sorry for the delay here. Trying to make the right decision and we are leaning toward creating transpiled versions of the esm-only d3-* dependencies that we have (similar to victory chart’s vendor packages) so that we can continue to ship csj + esm versions of all visx packages.

We also put it on vercel’s radar https://twitter.com/timneutkens/status/1653849232908886019

No concrete timeline but hoping to do this as soon as we can. If someone from the community is super motivated to try it we’re happy to discuss and provide PR review.

Hey, as promised I had a look into this.

As @ljharb pointed out ESM adoption is quite the problem. The reason it doesn’t just work in Next.js is that the package seems to have been published at a time before Node.js had built-in ESM. Because of that it’s using main and module to differentiate CommonJS vs ESM. However, Node.js ESM is much stricter than what we had with bundlers.

E.g. node -e "import('@visx/scale')" and node -e "require('@visx/scale')" doesn’t run currently.

Which is why this problem happens, the package is not compatible with Node.js’s stricter ESM and requires bundling in order to work.

My recommendation would be to leverage conditional exports: https://nodejs.org/api/packages.html#conditional-exports. That way Next.js would correctly handle the package using import() instead of require().

Still this means the package has to change to be compatible with the rules like explicit extensions and such.

–

(not recommended)

I managed to get it working adding the follow:

Add this next.config.js:

// next.config.js
/** @type {import('next').NextConfig} */
const nextConfig = {
  transpilePackages: [
    "@visx/mock-data",
    "@visx/group",
    "@visx/shape",
    "@visx/scale",
    "d3",
    "d3-interpolate",
  ],
};

module.exports = nextConfig;

Change node_modules/@visx/scale/package.json to have "type": "module".

Here’s a sandbox with the changes applied:

• CodeSandbox
• CodeSandbox URL

Thanks for the great thoughts, Jordan 🙏 I agree going ESM-only would be the least ideal. Here are my thoughts on the other options

1. Since d3-* are so fundamental to @visx/* I don’t think this is feasible unfortunately
2. d3 made the ESM-only decision quite a while ago and there have since been several security fixes I don’t think they’ll backwards-patch (the entire reason we upgraded to these ESM-only packages and released a major version bump was to fix a security vulnerability). One possible option we could consider here is using victory-vendor/d3-* packages. They faced a similar dllemma and decided to transpile & release their own CJS version of d3-* packages. This could be viable, my only concern is possible TS clashes and this wouldn’t work for any future lib we depend on that went ESM-only
3. I think this solution would be the simplest solution if it works, so is worth trying / would be the first thing I would try from here. I would note that we use next.js for our demo site, so we could test the feasibility ourselves with this.
4. again, hoping that we can avoid this.

I am very limited on bandwidth the next couple of weeks, so if anyone would like to explore #3 I’d be happy to review/collaborate on a branch.

alright I think this is fixed folks. try 3.2.0 and let me know if you have issues, see https://github.com/airbnb/visx/pull/1716 where we introduced @visx/vendor for more details.

3.2.0 works without special transpiling logic in our next@11 demo app, the package contents look good, and the sandbox works. I didn’t do another major version bump since csj was effectively broken in 3.0.x and 3.1.x anyway.

for anyone stumbling into this issue, here is a step-by-step of what worked for me on next13 using the pages router:

1. install visx:

npm install @visx/visx

1. add the following to next.config.js:

const nextConfig = {
  reactStrictMode: true,
  transpilePackages: ['d3-scale', '@visx/scale']
}

module.exports = nextConfig

1. install patch-package. you can use this to make changes to dependencies and have those automatically applied after installation:

npm install patch-package

1. add a postinstall script to your package.json. the package manager will run this after every install and apply the changes you’ll make in the following steps.

  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint",
    "postinstall": "patch-package" 
  },

1. add the following to the package.json of all packages under node_modules/@visx:

"exports": {
    ".": {
        "import":"./esm/index.js", 
        "require": "./lib/index.js"
    }
  }

1. for any @visx/ submodule that gives you the ERR_REQUIRE_ESM error, replace all instances of require("d3 with import("d3 (notice the space in the beginning of these strings) in the submodule’s lib folder. I tried doing this everywhere within the @visx/ folder but that broke, so I decided to just do it selectively, everywhere I run into an actual issue For example:

• I was getting this error from @visx/scale
• using VSCode’s Find in Files functionality (Cmd + Shift + F), I replaced all requires in the lib/ subfolder with imports

1. run patch-package for every single visx submodule. this will save the changes you made to a patches/ folder in your repository. you need to set the exclude flag to some dummy value, otherwise patch-package ignores the changes made in package.json:

npx patch-package --exclude 'nothingatall' @visx/annotation
npx patch-package --exclude 'nothingatall' @visx/axis
npx patch-package --exclude 'nothingatall' @visx/bounds
npx patch-package --exclude 'nothingatall' @visx/brush
npx patch-package --exclude 'nothingatall' @visx/clip-path
npx patch-package --exclude 'nothingatall' @visx/curve
npx patch-package --exclude 'nothingatall' @visx/drag
npx patch-package --exclude 'nothingatall' @visx/event
npx patch-package --exclude 'nothingatall' @visx/geo
npx patch-package --exclude 'nothingatall' @visx/glyph
npx patch-package --exclude 'nothingatall' @visx/gradient
npx patch-package --exclude 'nothingatall' @visx/grid
npx patch-package --exclude 'nothingatall' @visx/group
npx patch-package --exclude 'nothingatall' @visx/heatmap
npx patch-package --exclude 'nothingatall' @visx/hierarchy
npx patch-package --exclude 'nothingatall' @visx/legend
npx patch-package --exclude 'nothingatall' @visx/marker
npx patch-package --exclude 'nothingatall' @visx/mock-data
npx patch-package --exclude 'nothingatall' @visx/network
npx patch-package --exclude 'nothingatall' @visx/pattern
npx patch-package --exclude 'nothingatall' @visx/point
npx patch-package --exclude 'nothingatall' @visx/react-spring
npx patch-package --exclude 'nothingatall' @visx/responsive
npx patch-package --exclude 'nothingatall' @visx/scale
npx patch-package --exclude 'nothingatall' @visx/shape
npx patch-package --exclude 'nothingatall' @visx/text
npx patch-package --exclude 'nothingatall' @visx/threshold
npx patch-package --exclude 'nothingatall' @visx/tooltip
npx patch-package --exclude 'nothingatall' @visx/visx
npx patch-package --exclude 'nothingatall' @visx/voronoi
npx patch-package --exclude 'nothingatall' @visx/wordcloud
npx patch-package --exclude 'nothingatall' @visx/xychart

1. commit the changes to the repo

Summary of the steps:

• change next.config.js to transpile relevant packages
• change @visx/* submodules package.json by adding exports
• wherever needed replace require() with import()

I’m sure parts of this could be done smarter, but this worked for me so I thought I’d share

@williaster yes - esm-only dependencies are user-hostile, because they ensure that everything using them synchronously must also be native ESM, and thus you can’t ship CJS.

Your choices (in order of my recommendation) are:

1. avoid using any packages that (or packages by maintainers who) go ESM-only, and find better alternatives
2. permanently stay on pre-ESM versions of those packages, and/or fork the ESM-only versions
3. try to use them asynchronously, via import() - this still creates potential problems for consumers who use bundlers, but hopefully minor ones
4. go ESM-only yourself

There’s simply no value in going ESM-only - native ESM in node still lacks most of the tooling and ecosystem support that CJS has - and I strongly suggest authoring in ESM but transpiling to CJS for the foreseeable future.

To help anyone that stumbles upon this with Remix, I configured my remix.config.js in the following way to have it work.

  serverDependenciesToBundle: [
    'd3',
    /^d3-*/,
    /^@visx\/*/,
    'delaunator',
    'internmap',
  ],

Hopefully, it helps those users and gleans something for Next users.

I found good blog post regarding this Publish ESM and CJS in a single package

@williaster vitest should support esm modules natively, so is no need for transpiling.

I think that the issue is with invalid package.json. For package providing both commonjs and esm -modules there should be exports -field.

if I add following to @visx/scale -package file. Import from esm is working correctly:

  "exports": {
    ".": {
        "import":"./esm/index.js", 
        "require": "./lib/index.js"
    }
  },

Hey @kyhorne there are two issues I see with your package.json

1. you’re not using the latest @visx/*, can you try ^3.2.0 (some packages you use may only have 3.0.0 versions, but those with 3.2.0 versions published should be used)?
2. you have versions of d3-scale and d3-array in your dependencies which are ESM-only. next@13 might be able to handle that okay, but if you aren’t importing from them directly in your project, you might consider removing them.

^considering another validation in the wild, I’m going to go ahead and close this. but post any issues you have/we can re-open if necessary.

Worked for me with next@13

{
  "name": "visx-vendor-test",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start"
  },
  "dependencies": {
    "@visx/axis": "3.2.0",
    "@visx/grid": "3.2.0",
    "@visx/group": "3.0.0",
    "@visx/mock-data": "3.0.0",
    "@visx/scale": "3.2.0",
    "@visx/shape": "3.2.0",
    "next": "13.4.7",
    "react": "18.2.0",
    "react-dom": "18.2.0"
  }
}

also thank you very much @timneutkens for digging into this and giving some clarity on our esm/cjs module structure, will incorporate this as part of the fix. really appreciate your time! ❤️

@Jules-7 I think the problem comes from @visx/scale package not from d3-scale. Try to add @visx/scale to your transpilePackages.

The problem is that ~/app/node_modules/@visx/scale/lib/scales/band.js is from cjs and it’s trying to use d3 ES Module. There is ESM version of @visx/scale available in ~/node_modules/@visx/scale/esm/scales/band.js. When I made that exports change, vitest was correctly able to use ESM version.

I think that client side code (e.g. React) is using ESM version of @visx/scale and therefore there is no error. Problem with next.js could be that when next.js is doing server rendering it’s using node which is using CJS version. This is of course all my speculation so take it with grain of salt 🧂.

Awesome work @williaster, thank you! 🚀

@williaster for back compat, yes, you’d keep a top-level “types” key alongside the “exports” “types” key.

The best strategy imo is for ESM files to always be .mjs, for CJS files to always be .js, and to not set the “type” field at all.

I don’t know for sure how TS works with d.mjs vs d.cjs but it kind of seems like types would always stay as .d.ts, since nothing uses them at runtime?

Hi guys! I’ve stumbled upon the same issue in my NextJS v13.4.5 project. I first tried the - not recommended - solution proposed by timneutkens, but somehow it didn’t work, so I decided to test the one suggested by [@korompaiistvan] (#1637 (comment)) (which I already used in another NextJS project some time ago and back then it worked), but sadly this time after the fix I get the following error:

- error node_modules/@visx/tooltip/lib/Portal.js (9:94) @ prototype
- error Error [TypeError]: Cannot read properties of undefined (reading 'prototype')
    at _inheritsLoose (webpack-internal:///(sc_server)/./node_modules/@visx/tooltip/lib/Portal.js:13:51)
    at eval (webpack-internal:///(sc_server)/./node_modules/@visx/tooltip/lib/Portal.js:25:5)
    at eval (webpack-internal:///(sc_server)/./node_modules/@visx/tooltip/lib/Portal.js:49:2)
    at Object.(sc_server)/./node_modules/@visx/tooltip/lib/Portal.js (/Users/Fabio/Desktop/test/project/.next/server/app/[locale]/dashboard/page.js:5038:1)
    at __webpack_require__ (/Users/Fabio/Desktop/test/project/.next/server/webpack-runtime.js:33:43)
    at eval (webpack-internal:///(sc_server)/./node_modules/@visx/tooltip/lib/hooks/useTooltipInPortal.js:6:38)
    at Object.(sc_server)/./node_modules/@visx/tooltip/lib/hooks/useTooltipInPortal.js (/Users/Fabio/Desktop/test/project/.next/server/app/[locale]/dashboard/page.js:5082:1)
    at __webpack_require__ (/Users/Fabio/Desktop/test/project/.next/server/webpack-runtime.js:33:43)
    at eval (webpack-internal:///(sc_server)/./node_modules/@visx/tooltip/lib/index.js:8:50)
    at Object.(sc_server)/./node_modules/@visx/tooltip/lib/index.js (/Users/Fabio/Desktop/test/project/.next/server/app/[locale]/dashboard/page.js:5093:1) {
  digest: undefined
}

To give you the full picture, here’s the list of Visx dependencies that I’m currently using (and fun fact, in the project I’m not using any Portal component, just the TooltipWithBounds one and the useTooltip hook):

"@visx/axis": "^3.1.0",
"@visx/curve": "^3.0.0",
"@visx/event": "^3.0.1",
"@visx/glyph": "^3.0.0",
"@visx/grid": "^3.0.1",
"@visx/group": "^3.0.0",
"@visx/responsive": "^3.0.0",
"@visx/scale": "^3.0.0",
"@visx/shape": "^3.0.0",
"@visx/tooltip": "^3.1.2",
"d3-array": "^3.2.3",
"d3-time-format": "^4.1.0"

I gave a look at the /node_modules/@visx/tooltip/lib/Portal.js file, but I’m not quite sure on how to mitigate this issue. Did you guys have encounter the same problem and maybe found any solution?

Okay, so the problem was that I totally forgot to include the "use client" string at the top of the component that uses Visx utilities. Such a rookie mistake 🙈

I’m on Next.js 13.2.0 within the App router and I was also able to fix this issue by setting esmExternals to loose within the next.config.js.

const nextConfig = {
     ...,
     experimental: {
	appDir: true,
	esmExternals: 'loose',
     },
}

module.exports = nextConfig;

Found this fix within their docs here

Thanks @alexmalev , I have been looking for a solution for this as well, and your fix is the only thing I found that did the trick!

Hi @williaster, thank you for your message.

I tried the following in next.config.js for next@13.1 to no avail:

const nextConfig = {
  transpilePackages: ['d3-scale'],
};

In next@13.1 module transpilation is built-in. Still got the same error.

Had to switch to @visx/xychart v2.18.0 to get it working.