transformers.js: [Bug] Problem working with Next.js

Describe the bug I am trying to use transformers.js in my Next.js project, but I have problem using it in the new app directory.

How to reproduce Import any model from @xenova/transformers and initialize it. I have a test repository for this error.

app/page.tsx

import { PreTrainedTokenizer } from '@xenova/transformers'

export default function Home() {
  const tokenizer = new PreTrainedTokenizer({}, {})
  /* extra stuff */
}

Expected behavior The application starts without any problem.

Logs/screenshots

- error ./node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)

Import trace for requested module:
./node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node
./node_modules/onnxruntime-node/bin/napi-v3/ sync ^\.\/.*\/.*\/onnxruntime_binding\.node$
./node_modules/onnxruntime-node/dist/binding.js
./node_modules/onnxruntime-node/dist/backend.js
./node_modules/onnxruntime-node/dist/index.js
./node_modules/@xenova/transformers/src/backends/onnx.js
./node_modules/@xenova/transformers/src/env.js
./node_modules/@xenova/transformers/src/transformers.js
./app/page.tsx

Environment

Transformers.js version: 2.4.1
Browser (if applicable): Safari
Operating system (if applicable): MacOS Ventura (13.3.1)
Other:

Additional context

What I’ve tried

The solution from #98 (Ignore .node files) I’ve tried making webpack to ignore .node files but as pointed out in #183, Cannot find module 'onnxruntime-node' error appears. I’ve also tried installing onnxruntime-node separately. Following is the config used: next.config.js

module.exports = {
    reactStrictMode: true,
    webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
        // Modify the webpack configuration
        config.plugins.push(
            // Ignore node-specific modules when bundling for the browser
            new webpack.IgnorePlugin({
                resourceRegExp: /^onnxruntime-node$|^node:/,
            })
        );

        return config;
    },
};

Use node-loader I have added a config to next.config.js for web pack to handle .node files. However, it seems like webpack is having trouble finding the path to the .node file.

- error node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node (5:0) @ eval
- error Error: node-loader:
Error: dlopen(/path/to/project/.next/server/app//_next/332b2ce9fd36cb77421d6f263b9d702f.node, 0x0001): tried: '/path/to/project/.next/server/app//_next/332b2ce9fd36cb77421d6f263b9d702f.node' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/path/to/project/.next/server/app//_next/332b2ce9fd36cb77421d6f263b9d702f.node' (no such file), '/path/to/project/.next/server/app//_next/332b2ce9fd36cb77421d6f263b9d702f.node' (no such file)
    at eval (webpack-internal:///(rsc)/./node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node:6:9)
    at (rsc)/./node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node (/path/to/project/.next/server/app/page.js:2667:1)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at webpackContext (/path/to/project/.next/server/app/page.js:33:9)
    at eval (webpack-internal:///(rsc)/./node_modules/onnxruntime-node/dist/binding.js:10:133)
    at (rsc)/./node_modules/onnxruntime-node/dist/binding.js (/path/to/project/.next/server/app/page.js:2540:1)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./node_modules/onnxruntime-node/dist/backend.js:20:19)
    at (rsc)/./node_modules/onnxruntime-node/dist/backend.js (/path/to/project/.next/server/app/page.js:2529:1)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42) {
  digest: undefined
}
null

Following is the config used: next.config.js

module.exports = {
    reactStrictMode: true,
    webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
        config.node = {
            __dirname: false,
        };
        config.module.rules.push({
            test: /\.node$/,
            loader: 'node-loader',
            options: {
                name: "[contenthash].[ext]"
            },
        });
        return config;
    },
};

I have checked out #183 however the issue closed with a solution not using the app directory. It would be great to have a working example using the app directory, since the future updates in Next will be recommending that way. I’ve noticed you are working on a Next.js example. I would be happy to work on it together if you have any trouble.

About this issue

Original URL
State: closed
Created a year ago
Reactions: 3
Comments: 19 (7 by maintainers)

Commits related to this issue

Create example next.js application (Closes #210) (#211) * Create example next app * Link to example app * Update next configs * Create tutorial for next.js application * Update next.js tu... — committed to xenova/transformers.js by xenova a year ago

Most upvoted comments

Hi there 👋 Yeah, I agree… I should make a Next.js example 😅 I read through some of the webpack documentation, and after a bit of testing, it looks like the setting we need is config.resolve.alias

Does something like this work for you?

/** @type {import('next').NextConfig} */
const nextConfig = {
    webpack: (config) => {
        // Ignore node-specific modules when bundling for the browser
        // https://webpack.js.org/configuration/resolve/#resolvealias
        config.resolve.alias = {
            ...config.resolve.alias,
            "sharp$": false,
            "onnxruntime-node$": false,
        }
        return config;
    },
};

module.exports = nextConfig;

^ note: the $ at the end of the name is important

xenova on Jul 23, 2023

Its worth noting that currently in Next.js 14 you cannot use this library in both client side and server side. You have to pick one or else you will get those WASM / binary errors.

For server side you have to have:

//next.config.js
  experimental: {
    serverComponentsExternalPackages: ['sharp', 'onnxruntime-node'],
  },

If you are doing client side you have to have:

//next.config.js
  webpack: (config) => {
    config.resolve.alias = {
      ...config.resolve.alias,
      sharp$: false,
      'onnxruntime-node$': false,
    };

    return config;
  },

You cannot have both configs, chose the one based on where your doing your transforming

cwooldridge1 on Jan 11, 2024

@blechatellier Unfortunately not (I also don’t have much experience with pnpm). That said, this does look like a pnpm issue, so I would recommend opening up an issue in their repo instead. Alternatively, this may be an issue with next.js, since serverComponentsExternalPackages is marked as experimental.

See here for a related issue.

xenova on Oct 2, 2023

@xenova Thanks so much! somehow using npm instead of pnpm resolved the issue. I’m not sure why that would be… if it works it works I guess though 😄

raunakdoesdev on Aug 20, 2023

Here’s the demo app for those interested: https://huggingface.co/spaces/Xenova/next-server-example-app

xenova on Jul 25, 2023

@xenova it seems to be working as expected server side (both locally and when deployed). i guess the part i was missing is serverComponentsExternalPackages in next.config. Nice catch!

callmephilip on Jul 25, 2023

I’ve checked your PR and it works great! My issue also seems to be solved so I think it will be okay to close this issue. However, I added the next.config.js part you uploaded and encountered the following error:

- error node_modules/@xenova/transformers/src/env.js (60:0) @ eval
- error TypeError: Cannot read properties of undefined (reading 'wasm')
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at eval (./app/tokenizer/page.tsx:7:78)
    at (sc_client)/./app/tokenizer/page.tsx (/path/to/project/.next/server/app/tokenizer/page.js:391:1)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
null

I moved all my transformers.js related code to a client side rendered page and it seems to be working fine. (But I keep getting the error message) It seems like it’s a problem caused by ignored the onnxruntime_node package on the server side, so I don’t think it is 100% fixable.

james0248 on Jul 24, 2023

I’ve checked your PR and it works great! My issue also seems to be solved so I think it will be okay to close this issue. However, I added the next.config.js part you uploaded and encountered the following error:
- error node_modules/@xenova/transformers/src/env.js (60:0) @ eval
- error TypeError: Cannot read properties of undefined (reading 'wasm')
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
    at eval (./app/tokenizer/page.tsx:7:78)
    at (sc_client)/./app/tokenizer/page.tsx (/path/to/project/.next/server/app/tokenizer/page.js:391:1)
    at __webpack_require__ (/path/to/project/.next/server/webpack-runtime.js:33:42)
null
I moved all my transformers.js related code to a client side rendered page and it seems to be working fine. (But I keep getting the error message) It seems like it’s a problem caused by ignored the onnxruntime_node package on the server side, so I don’t think it is 100% fixable.

Did you get this working at all?

thomasgrob on Dec 5, 2023

Its worth noting that currently in Next.js 14 you cannot use this library in both client side and server side. You have to pick one or else you will get those WASM / binary errors.

For server side you have to have:
//next.config.js
  experimental: {
    serverComponentsExternalPackages: ['sharp', 'onnxruntime-node'],
  },
If you are doing client side you have to have:
//next.config.js
  webpack: (config) => {
    config.resolve.alias = {
      ...config.resolve.alias,
      sharp$: false,
      'onnxruntime-node$': false,
    };

    return config;
  },
You cannot have both configs, chose the one based on where your doing your transforming

Thanks for this insight. It solved this issue for me.

tobiasodion on Jan 12, 2024

yeah, i am also curious how to get this to work server side

callmephilip on Jul 24, 2023