turso: Limbo package does not load on WebContainers

I changed the Next.js starter template on StackBlitz to following:

import Image from 'next/image';
import { Database } from 'limbo-wasm';

export default function Home() {
  const db = new Database('hello.db');
  return (
    <main><p>Hello, Limbo!</p></main>
  );
}

but there’s something wrong wit the limbo-wasm package because it fails to load:

Error: Cannot find module './snippets/limbo-wasm-a4104707d9a4f276/vfs.js'
Require stack:
- /home/projects/stackblitz-starters-pn7p8k/.next/server/vendor-chunks/limbo-wasm.js
- /home/projects/stackblitz-starters-pn7p8k/.next/server/webpack-runtime.js
- /home/projects/stackblitz-starters-pn7p8k/.next/server/app/not-found.js
- /home/projects/stackblitz-starters-pn7p8k/node_modules/next/dist/server/require.js
- /home/projects/stackblitz-starters-pn7p8k/node_modules/next/dist/server/load-components.js
- /home/projects/stackblitz-starters-pn7p8k/node_modules/next/dist/build/utils.js
- /home/projects/stackblitz-starters-pn7p8k/node_modules/next/dist/server/dev/hot-middleware.js
- /home/projects/stackblitz-starters-pn7p8k/node_modules/next/dist/server/dev/hot-reloader-webpack.js
- /home/projects/stackblitz-starters-pn7p8k/node_modules/next/dist/server/lib/router-utils/setup-dev.js
- /home/projects/stackblitz-starters-pn7p8k/node_modules/next/dist/server/lib/router-server.js
- /home/projects/stackblitz-starters-pn7p8k/node_modules/next/dist/server/lib/start-server.js
- ```

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 1
  • Comments: 27 (21 by maintainers)

Most upvoted comments

However, having to polyfill the NodeJS variables doesn’t sound like the correct way to me. I believe for Next and others like it, the correct dependency should be the web version. I’m going to look into how to correct that.

@LtdJorge @elijahmorg btw, the creator of napi-rs says that they support Wasm out of the box: https://x.com/Brooooook_lyn/status/1881265313138041168 I was already considering using napi-rs for the native bindings, but if it support Wasm out of the box, then maybe that’s the solution going forward?

I was correct. next dev is picking up the Node version, and bundling the limbo-wasm.js file but not picking up the other JS and .wasm files. If I delete the node version from node_modules and all references from the package.json, it bundles the web version with the .wasm file into .next but fails at runtime.

I added this to next.config.ts:

import type { NextConfig } from "next";
import { type Configuration } from "webpack"

const nextConfig: NextConfig = {
    webpack: (config: Configuration, options) => {
        config.node = {
            __dirname: true,
            __filename: true,
            global: true
        }
        return config;
    },
};

export default nextConfig;

And now it correctly picks up the .wasm file and the other JavaScript from node_modules

import { Database } from 'limbo-wasm';

export default function Home() {
  const db = new Database('/home/jorge/projects/limbo/testing/testing.db');
  const stmt = db.prepare("SELECT * FROM users;");
  const results = stmt.all();
  return (
      results.map((object, i) => {
          return <div key={i} style={{paddingLeft: '6em'}}>{JSON.stringify(object)}</div>
          })
  )
}

Image

next build + next start work too.

Interesting. I think my 2 cents (but I am not an expert in this area) is that while commonjs is the lingua franca of nodejs limbo-wasm should use that and once wasm-pack supports node esm then it could convert to that.

We can have the npm package work for web and nodejs and then for deno publish to the deno package registry.

My implementation is getting a bit complex, and I don’t like it. So I think we should just review and merge your PR, and if I find a better way, I’ll apply some changes on top of your version later.

I have been also working on making the WASM version fully asynchronous with completions based on callbacks (for Node), and adapting your OPFS implementation for it too. So I’d prefer to apply these changes to a working version of both Node bindings and web bindings, aka your PR.

So, with “type”: “commonjs” image With “type”: “module” image However, this requires to use CommonJS style require("") in your JS, or configuring the TS compiler to transform imports into requires. Seems like wasm-bindgen has an experimental mode for ESM in NodeJS, but wasm-pack doesn’t export it.

You can generate esm/commonjs using wasm-pack target options. See https://github.com/tursodatabase/limbo/blob/main/bindings/wasm/scripts/build#L8

and for reference. https://rustwasm.github.io/docs/wasm-pack/commands/build.html#target

Now I don’t know if the esm output works for nodejs.

Sadly, it doesn’t, but it is because of the difference in APIs, no because of ESM. ESM is fine with Node, with the experimental version. I tried it and it works. The problem lies in how the .wasm files is loaded. If you just use the web option in wasm-pack, it uses fetch() to get the file, but we’re in Node, not on the frontend, so we have to use node’s fs. And in the frontend there is no fs.

So, I’ve been cooking up something, with dynamic import(), so that the module that gets loaded is selected at runtime. I’ve been also reworking the bundling and the build script, to make it more straightforward.

Il’ll try to finish it tomorrow morning (I’m UTC+1) and send a PR for you guys to check. We can pick and choose then.

Ouch, perhaps switching to CommonJS is the sane choice for now… 😕