next.js: Rust WebAssembly module in an ES module wrapper from wasm-pack fails to load in Next.js

What version of Next.js are you using?

11.1.2

What version of Node.js are you using?

14.17.6

What browser are you using?

Firefox, Chrome

What operating system are you using?

macOS

How are you deploying your application?

Not yet deployed

Describe the Bug

A simple Rust WebAssembly module packaged with its glue code into an ES module with wasm-pack (patched as in https://github.com/rustwasm/wasm-pack/pull/1061) loads and works just fine under webpack, as illustrated in https://github.com/webpack/webpack/pull/14313, but fails to import under Next.js. This is apparently because Next.js generates the .wasm generated at one path but then tries to load it from a different path.

Expected Behavior

I expected the npm run dev to successfully run the application and render a page with the greeting from inside the WebAssembly module.

I expected npm run build to successfully build a production distribution that would do the same.

To Reproduce

This is demonstrated in https://github.com/gthb/try-to-use-wasm-in-next.js/ — the README there recounts my circuitous path of trying to get this to work, but the current state serves to illustrate the problem I’m reporting here.

In that repo, first setup:

yarn install
yarn run build-wasm # or just copy the `pkg` from https://github.com/webpack/webpack/pull/14313 into `hi-wasm/pkg`
yarn run link-wasm

Then try yarn run dev — it fails like this:

yarn run v1.22.11
$ next dev
ready - started server on 0.0.0.0:3000, url: http://localhost:3000
info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

event - compiled successfully
event - build page: /next/dist/pages/_error
wait  - compiling...
event - compiled successfully
event - build page: /
wait  - compiling...
event - compiled successfully
error - pages/index.js (22:51) @ Home
TypeError: (0 , hi_wasm__WEBPACK_IMPORTED_MODULE_4__.greeting) is not a function
  20 |         <p className={styles.description}>
  21 |           Greeting:
> 22 |           <code className={styles.code}>{ greeting("Bob") }</code>
     |                                                   ^
  23 |         </p>
  24 |
  25 |         <div className={styles.grid}>

Then try yarn run build — it fails like this:

yarn run v1.22.11
$ next build
info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

info  - Checking validity of types
info  - Creating an optimized production build
info  - Compiled successfully

> Build error occurred
[Error: ENOENT: no such file or directory, open '/Users/gthb/git/try-to-use-wasm-in-next.js/.next/server/static/wasm/1905d306caca373cb9a6.wasm'] {
  type: 'Error',
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/Users/gthb/git/try-to-use-wasm-in-next.js/.next/server/static/wasm/1905d306caca373cb9a6.wasm'
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Note the path to the .wasm chunk. The actual generated .wasm chunks are:

$ find .next -iname '*.wasm'
.next/server/chunks/static/wasm/1905d306caca373cb9a6.wasm
.next/static/wasm/b9a4f4672eb576798896.wasm

So apparently the failure is that Next.js generates the .wasm chunk on the server side with an extra chunks/ path element (which presumably should not be there) and then looks for it at a path without that path element and fails to find it.

I’m guessing that the same problem is the cause of the npm run dev failure (i.e. the module imports as empty).

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 18
  • Comments: 25 (16 by maintainers)

Commits related to this issue

Most upvoted comments

The above workaround breaks when using experiments.layers = true because even though the module ID is named, a chunk hash is still appended to the output assets. The only way the above fix can work is if the client and server output the same filename for both the client and the sever.

I’ve created a more reliable workaround in the form of an embedded webpack plugin. This behaviour is pretty close to how Next.js internally handles the chunks directory and then walking back up with ../

// next.config.js

module.exports = {
  webpack(config, { isServer, dev }) {
    config.experiments = {
      asyncWebAssembly: true,
      layers: true,
    };

    if (!dev && isServer) {
      config.output.webassemblyModuleFilename = "chunks/[id].wasm";
      config.plugins.push(new WasmChunksFixPlugin());
    }

    return config;
  },
};

class WasmChunksFixPlugin {
  apply(compiler) {
    compiler.hooks.thisCompilation.tap("WasmChunksFixPlugin", (compilation) => {
      compilation.hooks.processAssets.tap(
        { name: "WasmChunksFixPlugin" },
        (assets) =>
          Object.entries(assets).forEach(([pathname, source]) => {
            if (!pathname.match(/\.wasm$/)) return;
            compilation.deleteAsset(pathname);

            const name = pathname.split("/")[1];
            const info = compilation.assetsInfo.get(pathname);
            compilation.emitAsset(name, source, info);
          })
      );
    });
  }
}

@gthb Lord forgive me, but I’ve created a workaround:

// next.config.js
module.exports = {
  webpack(config, { isServer, dev }) {
    // Enable webassembly
    config.experiments = { asyncWebAssembly: true };

    // In prod mode and in the server bundle (the place where this "chunks" bug
    // appears), use the client static directory for the same .wasm bundle
    config.output.webassemblyModuleFilename =
      isServer && !dev ? "../static/wasm/[id].wasm" : "static/wasm/[id].wasm";

    // Ensure the filename for the .wasm bundle is the same on both the client
    // and the server (as in any other mode the ID's won't match)
    config.optimization.moduleIds = "named";

    return config;
  },
};

Then to ensure the .wasm file is always included in the client bundle (you don’t need to do this if you’re using it ONLY on the client, or in both the server and the client), place this somewhere in the root of your _app.tsx:

(function () {
  import("wasm/add.wasm");
  // Import which .wasm files you need here
});

This won’t download the file onto the client, just ensure that it’s in the bundle

This is obviously far from an ideal solution, because it still generates the server bundle at .next/server/chunks/../static/wasm/[id].wasm (i.e. .next/server/static/wasm/[id].wasm) that webpack is trying to import for the server, but instead gets the static one because it’s looking for .next/server/../static/wasm/[id].wasm (i.e. .next/static/wasm/[id].wasm)

I am using Nextjs13 with wasm. This thread has been a real help. 🤗

Though I would prefer a more official fix.

Hi has there been any progress on a fix for the deployment on Vercel? I have the same exact issue as @icyJoseph

@calclavia I was able to get WASM imports to work for regular API routes and on the frontend with the config below. Note that I had to enable Node 20 in the runtime settings (in Node 20 WASM support no longer requires a flag).

const CopyPlugin = require("copy-webpack-plugin");

const nextConfig = {
  webpack: (config, { isServer, dev }) => {
    config.experiments = {
      layers: true,
      asyncWebAssembly: true
    };

    if (!dev && isServer) {
      webassemblyModuleFilename = "./../server/chunks/[modulehash].wasm";

      const patterns = [];

      const destinations = [
        "../static/wasm/[name][ext]", // -> .next/static/wasm
        "./static/wasm/[name][ext]",  // -> .next/server/static/wasm
        "."                           // -> .next/server/chunks (for some reason this is necessary)
      ];
      for (const dest of destinations) {
        patterns.push({
          context: ".next/server/chunks",
          from: ".",
          to: dest,
          filter: (resourcePath) => resourcePath.endsWith(".wasm"),
          noErrorOnMissing: true
        });
      }

      config.plugins.push(new CopyPlugin({ patterns }));
    }

    return config;
  }
};

Oh wow thanks for the pointer @sam3d – I have missed that comment somehow. I just tried your workaround in hasharchives/wasm-ts-esm-in-node-jest-and-nextjs and it did it’s job! 🎉 Awesome stuff!

(p.s. you don’t need the following by the way, I posted my patch hastily and forgot to remove this part from my own Next.js config:)

config.module.rules.push({
  test: /\.svg$/,
  use: ["@svgr/webpack"],
});

🚨 This comment is partially out of date – see below


I managed to get import * as wasm from "./my.wasm" working during next dev / next build / next start, thanks to a workaround by @ctrespoo.

Here is my next.config.js for next@12.3.2-canary.27 (UPD: and next@13.0.2):

/** @type {import("next").NextConfig} */
export default {
  webpack: (webpackConfig, { isServer }) => {
    // WASM imports are not supported by default. Workaround inspired by:
    // https://github.com/vercel/next.js/issues/29362#issuecomment-1149903338
    // https://github.com/vercel/next.js/issues/32612#issuecomment-1082704675
    return {
      ...webpackConfig,
      experiments: {
        asyncWebAssembly: true,
        layers: true,
      },
      optimization: {
        ...webpackConfig.optimization,
        moduleIds: "named",
      },
      output: {
        ...webpackConfig.output,
        webassemblyModuleFilename: isServer
          ? "./../static/wasm/[modulehash].wasm"
          : "static/wasm/[modulehash].wasm",
      },
    };
  },
};

WASM modules load fine in an SSR page, but an API route returns Internal Server Error. Server logs contain this:

[Error: ENOENT: no such file or directory, open '/path/to/project/.next/static/wasm/7137689891e3e4e4.wasm'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/path/to/project/.next/static/wasm/7137689891e3e4e4.wasm'
}

Both Next page and API handler works correctly in next dev.

Repo with reproduction: hasharchives/wasm-ts-esm-in-node-jest-and-nextjs

I believe it is #22697 that’s causing this conflict.

On a production build with webpack 5 the output configuration is as follows:

{
  publicPath: '/_next/',
  path: '/Users/sam/Desktop/with-webassembly-app/.next/server/chunks',
  filename: '../[name].js',
  library: undefined,
  libraryTarget: 'commonjs2',
  hotUpdateChunkFilename: 'static/webpack/[id].[fullhash].hot-update.js',
  hotUpdateMainFilename: 'static/webpack/[fullhash].[runtime].hot-update.json',
  chunkFilename: '[name].js',
  strictModuleExceptionHandling: true,
  crossOriginLoading: undefined,
  webassemblyModuleFilename: 'static/wasm/[modulehash].wasm'
}

It seems as though webassemblyModuleFilename interprets this to output to .next/server/chunks/static/wasm/[modulehash].wasm, or alternatively {output.path}/{webassemblyModuleFilename}. Chunks are outputted to .next/server/chunks but ordinary files are outputted to the parent directory with ../[name].js. webassemblyModuleFilename doesn’t handle this parent recursion correctly, and so ends up in chunks. When importing, it doesn’t appear to look for chunks because it’s looking for webassemblyModuleFilename at the root of the webpack config, or .next/server

As per #29485, I’m actually not sure this is wasm-pack’s (or wasm-bindgen’s fault). I’m seeing this behaviour with just importing the vanilla .wasm file when building the Next.js application for production. Out of interest, are you able to replicate the problem in my issue?