o1js: Can't use `PrivateKey` on server in a NextJS 14 app

I got most of the way towards removing TLA here: https://github.com/o1-labs/o1js/pull/1583 will finish tomorrow

Managed to get the node version of o1js used in NextJS by removing the main from package.json.

  "main": "./dist/web/index.js", // <-remove this
  "exports": {
    "types": "./dist/node/index.d.ts",
    "browser": "./dist/web/index.js",
    "node": {
      "import": "./dist/node/index.js",
      "require": "./dist/node/index.cjs"
    },
    "default": "./dist/web/index.js"
  },

Still hanging but at least running the right version now I think (not getting navigator error.

Hi @sam-goodwin! Thanks for all your feedback and clear descriptions of the problems you are facing. I agree that there are opportunities to remove the friction to create an app. The current NextJS scaffold in the zkApp-CLI was originally developed with versions that utilized the pages folder structure and an older version of webpack. We will update the scaffold to utilize Next14 as well the app router and turboPack. With these updates, we can simplify configurations like this.

 reactStrictMode: false,
 webpack(config) {
   config.resolve.alias = {
     ...config.resolve.alias,
     o1js: require('path').resolve('node_modules/o1js')
   };
   config.experiments = { ...config.experiments, topLevelAwait: true };
   return config;
 },

Opened a PR to update the node-backend.js and web-backend.js: https://github.com/o1-labs/o1js-bindings/pull/267

I believe this is required as a first step.

Is there a reason why that code warrants a separate repo vs just being in this repo for simplicity? How can I make changes to both repos in 1 PR? Or is that not possible?

Managed to get the node version of o1js used in NextJS by removing the main from package.json.

Nice catch!! 😮

Is this what you mean?

  async runAndCheck(f: (() => Promise<void>) | (() => void)) {
    await initO1(); // call it here?
    await generateWitness(f, { checkConstraints: true });
  },

And the plan to get rid of it is to:

• Refactor that function so that it won’t repeat its work when called a second time
• call that function in a selected set of places that depend on it
• make a dummy Nodejs version of it so we can do the above universally
• Also export the init function, so that people can trigger the initialization work explicitly. Note that doing this should be optional, because we also guarantee that it’s called when needed

Thanks for the responses, @mitschabaude.

Btw @sam-goodwin this “bizarre code” is just about loading the library lazily, to reduce initial loading time. Actually I think it’s a common pattern to use dynamic import for that 😅

Bundlers will optimize the bundled code and if there’s expensive initialization code, we can move that out of the module import path and put it in a function. Give control to the user through explicit functions instead of through the await import mechanism.

Sticking to ordinary practices is going to have far less bugs and also scare less people off. I don’t think I’ve ever seen an await import inside a useEffect.

Is this the only place where top-level await is required? For the bindings?

https://github.com/o1-labs/o1js/blob/main/src/snarky.js

Could we instead defer this evaluation by placing it in an init function:

const Mina = await initMina();

Avoiding global state and expensive async processing when importing a module is generally good practice.

To allow use of SharedArrayBuffer, browsers require the COEP and COOP headers which you saw in that nextjs config.

I think this is fine. Seems unavoidable.

This bit scares me:

  reactStrictMode: false,
  webpack(config) {
    config.resolve.alias = {
      ...config.resolve.alias,
      o1js: require('path').resolve('node_modules/o1js')
    };
    config.experiments = { ...config.experiments, topLevelAwait: true };
    return config;
  },

Anything we can do to remove that would be a win.

The main problem here, as far as I can tell, is that NextJS doesn’t “just work” when importing modern ES modules. It has trouble with top level await, which we use in our web export and which is supported in all major browser as since 3 years.

I appreciate that top-level await has been supported by browsers, but for Mina to succeed, I think prioritizing smooth integration with the popular web frameworks is more important than using a less supported, modern feature.

RE: https://github.com/o1-labs/o1js/issues/1205 - glad to see there is a plan to remove top-level await. Anything I can do to help? I’d very much like to be able to use o1js just like any other library in my NextJS’s client and server side code.