aws-cdk: @aws-cdk/aws-lambda-nodejs doesn't build functions with native modules

I’m trying to get a lambda set up with @aws-cdk/aws-lambda-nodejs.

I applied the change mentioned in #6274, and now I have a build error:

Error: Failed to build file at [my lambda function]: Error: 🚨 [project path]/node_modules/pg/lib/native/client.js:11:21: Cannot resolve dependency ‘pg-native’

Reproduction Steps

Use "pg": "7.18.1", then in the source of the lambda:

import * as pg from 'pg';

Error Log

Error: Failed to build file at [my lambda function]: Error: 🚨 [project path]/node_modules/pg/lib/native/client.js:11:21: Cannot resolve dependency ‘pg-native’

Environment

  • CLI Version : 1.24.0
  • Framework Version: 1.24.0
  • OS : OS X 10.15.3
  • Language : Typescript

Other

I’m familiar with how to work around this error in Webpack, but given that Parcel is a zero config tool, I’m not sure what to do about this error.

I think connecting to RDS databases from Node Lambdas will be a common use case, so this should be supported out of the box if possible, or at least have a documented solution.


This is 🐛 Bug Report

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 4
  • Comments: 19 (8 by maintainers)

Commits related to this issue

Most upvoted comments

FWIW, let me share my workaround of this issue for future reference. I didn’t intend to use pg-native, so set it to the externalModules settings as follows.

    const lambda = new NodejsFunction(this, 'Function', {
      entry: 'lambda/src/index.ts',
      handler: 'handler',
      allowAllOutbound: true,
      bundling: {
        // pg-native is not available and won't be used. This is letting the
        // bundler (esbuild) know pg-native won't be included in the bundled JS
        // file.
        externalModules: ['pg-native']
      }
    });

@jogold I think we should add support for building the function inside a Lambda docker image (like/with “sam build”). What do you think?

I’m with @thekevinbrown here.

I think:

  • parcel is too primitive and lacks configuration. It is great if you are building a simple web app, but it falls apart on large, backend-oriented projects.
  • using docker is way too heavy handed and is extremely slow.
    • It is also a pain to use, if you don’t have Docker installed, e.g. in a CI environment.
    • I am not even sure if GH Actions supports running arbitrary Docker containers. It might, but I couldn’t find any info on that.
    • Trying to test the infra via Jest basically runs all of the Docker bundling every time, which is extremely slow.

Overall DX here is very sluggish and full of errors.

I think a better approach would be to go for Webpack or Rollup, with a sane default configuration but allow consumers to supply config overrides.

You can see this being used successfully in projects like Nuxt. Where nuxt.config.ts file exposes props (simple) and hooks (advanced) for configuration of Webpack internals, but it is not necessary to touch them for most projects, and everything just works.

Likewise @aws-cdk/aws-lambda-nodejs could automatically fix many issues, like exclusion of aws-sdk, and ignoring pg-native module.

FWIW, let me share my workaround of this issue for future reference. I didn’t intend to use pg-native, so set it to the externalModules settings as follows.

    const lambda = new NodejsFunction(this, 'Function', {
      entry: 'lambda/src/index.ts',
      handler: 'handler',
      allowAllOutbound: true,
      bundling: {
        // pg-native is not available and won't be used. This is letting the
        // bundler (esbuild) know pg-native won't be included in the bundled JS
        // file.
        externalModules: ['pg-native']
      }
    });

I use a slightly different project structure provided by the sam cli and ran into this issue as well. I have a template.yaml file. Is it somehow possible to add this externalModules option there?

My template.yaml:

Resources:
  onSignup:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: onSignup/
      Handler: app.lambdaHandler
      Runtime: nodejs14.x
      Architectures:
        - x86_64
    Metadata: # Manage esbuild properties
      BuildMethod: esbuild
      BuildProperties:
        Minify: true
        Target: "es2020"
        Sourcemap: true
        EntryPoints: 
        - app.ts