libsql-client-ts: "Runtime.ImportModuleError: Error: Cannot find module '@libsql/linux-x64-gnu" when bundling with esbuild for aws lambda

I was switching my database from planetscale to turso but I’m getting the following error

 {
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Error: Cannot find module '@libsql/linux-x64-gnu'\nRequire stack:\n- /var/task/index.js\n- /var/runtime/index.mjs",
  "trace": [
    "Runtime.ImportModuleError: Error: Cannot find module '@libsql/linux-x64-gnu'",
    "Require stack:",
    "- /var/task/index.js",
    "- /var/runtime/index.mjs",
    "    at _loadUserApp (file:///var/runtime/index.mjs:1061:17)",
    "    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1093:21)",
    "    at async start (file:///var/runtime/index.mjs:1256:23)",
    "    at async file:///var/runtime/index.mjs:1262:1"
  ]
}

I use the aws cdk to build and deploy my aws lambda function internally it uses esbuild here is a simple reproduction step

the aws lambda function code /nodejs/index.ts

import { drizzle } from 'drizzle-orm/libsql';
import { createClient } from '@libsql/client';
const client = createClient({ url: 'DATABASE_URL', authToken: 'DATABASE_AUTH_TOKEN' });

const db = drizzle(client);

export const handler = async (event) => {
  console.log('event', event);
}

the aws cdk code to deploy the function:

import {Construct} from 'constructs';
import {NodejsFunction} from 'aws-cdk-lib/aws-lambda-nodejs';
import {Architecture, Runtime} from 'aws-cdk-lib/aws-lambda';

export class SendUserToDynamo extends Construct {
  function: NodejsFunction;
  constructor(scope: Construct, id: string) {
    super(scope, id);

    // lambda function that triggers on aws lambda user created event
    this.function = new NodejsFunction(this, 'sendUserToDynamo', {
      entry: __dirname + '/nodejs/index.ts',
      handler: 'handler',
      architecture: Architecture.X86_64,
      runtime: Runtime.NODEJS_18_X,
    });
  }
}

Everything works when I’m using the PlanetScale driver, so the error must come from libsql and not the CDK build step. I tried using a Dockerfile without TypeScript and ESBuild, and it did work

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 27
  • Comments: 23

Commits related to this issue

Most upvoted comments

I ended up using the web/HTTP client to be able to run the latest version:

import { createClient } from '@libsql/client/web';

Confirming I’m seeing this too, also switching from Planetscale to Turso. Interestingly the error doesn’t happen in local env, only when deployed to Netlify. Running node 16. Pulling @libsql/client down to 0.3.2 from 0.5.3 fixed things for me.

fyi this is still an active issue. @libsql/client/web is also not a feasible workaround as it doesn’t support embedded replicas.

Just ran into this. The @libsql/client/web workaround also doesn’t seem to work. I’m deploying to Lambda using Serverless Framework. Does anyone have a workaround?

I ended up using the web/HTTP client to be able to run the latest version:

import { createClient } from '@libsql/client/web';

this works

I was able to solve this by adding the @libsql/client as a nodejs dependency which allows SST to do an NPM install of them in the function bundle:

import { usersTable } from "./database";
import { TursoAuthToken, TursoDatabaseUrl } from "./database";

export const api = new sst.aws.Function("Api", {
  url: true,
  link: [usersTable, TursoAuthToken, TursoDatabaseUrl],
  handler: "./packages/functions/api/index.handler",
  architecture: "arm64",   
  nodejs: {
    install: ["@libsql/client"]
  }        
});

0.3.3 also works

The import { createClient } from '@libsql/client/web' workaround only works when there’s no transitive dependencies on libsql. It’s possible to get transitive dependencies to work as well by adding an esbuild alias: --alias:'@libsql/client=@libsql/client/web' in the esbuild command (and importing it as import { createClient } from '@libsql/client' ).

I ended up using the web/HTTP client to be able to run the latest version:

import { createClient } from '@libsql/client/web';

this works

This worked for me too! Thanks!

If you want to use the native client that uses the libsql binary under the hood in Lambda, you’ll have to build the function in a similar arch/environment or use a Lambda layer, also built under the same conditions (i.e. Docker).

A bundler like esbuild won’t be able to include the correct version (i.e. @libsql-linux-arm64-gnu) unless it’s itself running on that same hardware.

This worked for me:

import { Stack, type StackProps, CfnOutput } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Architecture, Code, LayerVersion, Runtime } from 'aws-cdk-lib/aws-lambda';
import { NodejsFunction, OutputFormat } from 'aws-cdk-lib/aws-lambda-nodejs';

export class LibsqlStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const libsqlLayer = new LayerVersion(this, 'LibsqlLayer', {
      compatibleArchitectures: [Architecture.ARM_64],
      compatibleRuntimes: [Runtime.NODEJS_20_X],
      code: Code.fromAsset('./layers/libsql/nodejs', {
        bundling: {
          image: Runtime.NODEJS_20_X.bundlingImage,
          environment: {
            npm_config_cache: "/tmp/npm_cache",
            npm_config_update_notifier: "false",
          },
          command: [
            'bash',
            '-xc',
            [
              'cd $(mktemp -d)',
              'cp /asset-input/package* .',
              'npm --prefix . i @libsql/client',
              'cp -r node_modules /asset-output/'
            ].join(' && '),
          ]
        }
      }),
    });

    const fn = new NodejsFunction(this, 'MyFunction', {
      runtime: Runtime.NODEJS_20_X,
      architecture: Architecture.ARM_64,
      entry: './src/index.ts',
      handler: 'handler',
      bundling: {
        minify: true,
        mainFields: ['module', 'main'],
        sourceMap: true,
        format: OutputFormat.ESM,
        externalModules: ['@libsql/client'],
      },
      layers: [libsqlLayer],
    });

    new CfnOutput(this, 'FunctionArn', {
      value: fn.functionArn,
    });
  }
}

then in my function I use @libsql/client as normal:

import { Logger } from "@aws-lambda-powertools/logger";
import { createClient } from "@libsql/client";

const logger = new Logger();
const db = createClient({
  url: "file:/tmp/sqlite.db",
});

export const handler = async () => {

  await db.execute("CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT)");

  await db.execute({ sql: "INSERT INTO users (name) VALUES (?)", args:  ["Alice"]});

  const rows = await db.execute("SELECT * FROM users");

  logger.info("Rows: ", {rows});

  return {
    statusCode: 200,
    body: JSON.stringify("Hello, World!"),
  };
};

which when invoked prints this:

{
    "level": "INFO",
    "message": "Rows: ",
    "sampling_rate": 0,
    "service": "service_undefined",
    "timestamp": "2024-06-27T18:14:43.388Z",
    "xray_trace_id": "1-667dac12-2624c2867735f80f34586f58",
    "rows": {
        "columns": [
            "id",
            "name"
        ],
        "columnTypes": [
            "INTEGER",
            "TEXT"
        ],
        "rows": [
            [
                1,
                "Alice"
            ]
        ],
        "rowsAffected": 0,
        "lastInsertRowid": null
    }
}

The advantage of using a Lambda layer like that is that you can still continue using esbuild for the function(s) - the only caveat there is to make sure to specify externalModules: ['@libsql/client'], so that the dependency is not included in the bundle - this is because it comes from the Layer.

it does say in the docs that you should use it for vercel functions and the like. the docs could be a bit clearer here. nothing mentions what to use for lambda

Ran into this too and got it working with 0.3.3.