serverless-webpack: Error when packager is yarn

This is a Bug Report

Description

When using nodeExternals in webpack config, deployment throws an error. The issue is only reproduced with includeModules: true + packager: 'yarn'.

 Error --------------------------------------------------
 
  Command failed: yarn install --frozen-lockfile --non-interactive
error An unexpected error occurred: "Your lockfile needs to be updated, but yarn was run with `--frozen-lockfile`.".

It may help because the deployment only include production packages, which causes a difference in yarn lockfile.

Additional Data

  • Serverless-Webpack: 5.1.1
  • Webpack version: 4.5.0
  • Serverless Framework: 1.26.1
  • webpack-node-externals: 1.7.2

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 1
  • Comments: 17 (4 by maintainers)

Most upvoted comments

I’m getting this same issue, but with no local modules and with a yarn.lock file that’s in sync with package.json. I tried deleting yarn.lock and running yarn install to generate a new one, but it generated an identical file to the one I was already using.

If I remove nodeExternals() from externals in webpack.config.js then I no longer get the problem.

FWIW i’ve also noticed this if I’m using a dependency that a module included but I didn’t explicitly add as a dependency. E.g. I have in one of my files:

import { DataSource } from 'apollo-datasource'

it isn’t in my package.json, but I get this ok because it comes via apollo-server-lambda -> apollo-server-core -> apollo-datasource. I use serverless offline to run the code locally.

On serverless deploy, I see:

Serverless: WARNING: Could not determine version of module apollo-datasource
Serverless: Adding explicit peers for dependency apollo-server-lambda
...
  yarn install --frozen-lockfile --non-interactive failed with code 1
  error Your lockfile needs to be updated, but yarn was run with `--frozen-lockfile`.

but if i do yarn add apollo-datasource, it deploys ok.

I suspect it is good practice to have as package dependencies the modules you explicitly use in your package, e.g. you shouldn’t be able to import ‘lodash’ just because some module you use has it as a dependency. However, why serverless deploy doesn’t warn or fail outright because the dependency isn’t explicitly added is a a puzzle to me.

Command failed: yarn install --frozen-lockfile --non-interactive

I was having the same problem ☝️

Something that I noticed, is that the Packing external modules was outputting babel-runtime, without a version:

Serverless: Packing external modules: babel-runtime, @aws/dynamodb-data-mapper@0.4.0, serverless-dynamodb-client@^0.0.2, nodemailer@^4.4.2, wkhtmltopdf@^0.3.4, memorystream@^0.3.1, joi@12.0.0, bson-objectid@^1.2.2

all other modules have a version.

Adding babel-runtime in my package.json “fixed” the problem.

But adding babel-runtime didn’t feel right, so I ended removing all my dependencies, then added one-by-one back, to find out that babel-plugin-transform-object-entries was causing the problem, but since I no longer need it, I removed it and the problem was gone 🙏

In my case, I’m not using local references. But while reading the output from serverless deployment, I just found out what was happening. In the list of modules in Packing external modules stage, there was a peer dependency trying to be packaged that was actually missing from my package.json. Then it was throwing the error.

Regarding the discussion above, the --frozen-lockfile completely makes sense to me.

Thank you @mzmiric5 and @HyperBrain.

When you got error, open .serverless/dependencies/package.json you can see one of package has no version. EX: "aws-sdk: "".

RESOLVED To resolve this, you just run yarn add --dev aws-sdk or without --dev for some packages. Then run deploy again.

I’m running into a similar issue, so I checked my .webpacl/dependencies/package.json VS my package.json and there are some differences:

{
  "name": "sib-helpers-server",
  "version": "1.0.0",
  "description": "Packaged externals for sib-helpers-server",
  "private": true,
  "scripts": {},
  "dependencies": {
    "source-map-support": "",
    "babel-runtime": "",
    "axios": "0.18.0",
    "dotenv": "6.0.0",
    "moment": "2.22.2",
    "@google-cloud/storage": "2.1.0",
    "aws-sdk": "2.334.0"
  }
}

Here is the package.json:

"devDependencies": {
    "@jimdo/serverless-dotenv": "1.0.0",
    "babel-cli": "6.26.0",
    "babel-core": "6.26.3",
    "babel-loader": "7.1.5",
    "babel-plugin-source-map-support": "2.0.1",
    "babel-plugin-transform-runtime": "6.23.0",
    "babel-preset-env": "1.7.0",
    "babel-preset-stage-2": "6.24.1",
    "nodemon": "1.18.4",
    "serverless": "1.32.0",
    "serverless-offline": "3.27.0",
    "serverless-plugin-aws-alerts": "1.2.4",
    "serverless-webpack": "5.2.0",
    "webpack": "4.20.2",
    "webpack-node-externals": "1.7.2"
  },
  "dependencies": {
    "@google-cloud/storage": "2.1.0",
    "adm-zip": "0.4.11",
    "aws-sdk": "2.334.0",
    "axios": "0.18.0",
    "copy-webpack-plugin": "4.5.4",
    "dotenv": "6.0.0",
    "fast-crc32c": "1.0.4",
    "jest": "23.6.0",
    "json2csv": "4.2.1",
    "jszip": "3.1.5",
    "lodash.map": "4.6.0",
    "lodash.uniq": "4.5.0",
    "moment": "2.22.2",
    "yargs": "12.0.2"
  }

I don’t understand why source-map-support and transform-runtime are empty.

Here is my .babelrc:

{
  "plugins": [
    "source-map-support",
    "transform-runtime"
  ],
  "presets": [
    "env",
    "stage-2"
  ]
}

Any idea? I tried deleting/recreating the yarn.lock as well but didn’t fix anything.

Edit: We fixed it by using babel-runtime and source-map-support as dependencies, I don’t really understand why it works this way, it’s very confusing to say the least.

  "devDependencies": {
    "aws-sdk": "2.354.0",
    "babel-cli": "6.26.0",
    "babel-core": "6.26.3",
    "babel-loader": "7.1.5",
    "babel-plugin-source-map-support": "1.0.0",
    "babel-plugin-transform-runtime": "6.23.0",
    "babel-preset-env": "1.7.0",
    "babel-preset-stage-2": "6.24.1",
    "nodemon": "1.18.4",
    "serverless": "1.32.0",
    "serverless-offline": "3.27.0",
    "serverless-plugin-aws-alerts": "1.2.4",
    "serverless-webpack": "5.2.0",
    "webpack": "4.20.2",
    "webpack-node-externals": "1.7.2"
  },
  "dependencies": {
    "@google-cloud/storage": "2.1.0",
    "@jimdo/serverless-dotenv": "1.0.0",
    "adm-zip": "0.4.11",
    "axios": "0.18.0",
    "babel-runtime": "6.26.0",
    "copy-webpack-plugin": "4.5.4",
    "dotenv": "6.1.0",
    "fast-crc32c": "1.0.4",
    "jest": "23.6.0",
    "json2csv": "4.2.1",
    "jszip": "3.1.5",
    "lodash.map": "4.6.0",
    "lodash.uniq": "4.5.0",
    "moment": "2.22.2",
    "source-map-support": "0.4.18",
    "yargs": "12.0.2"
  }

@bazaglia @mzmiric5 This is intended behavior. The plugin sets --frozen-lockfile to prevent the build/deployment server install any new versions on its own and so change the module dependencies. You have to make sure that you commit a yarn.lock file in your projects that matches the package.json.

You should check, if your yarn.lock file is up-to-date and matches everything that is in your package.json. If you change your package.json manually, they are out of sync and the packaging will fail.

As long as you do a yarn install after changing the files and use yarn add to add linked of local (file) packages, it will generate a matching yarn.lock file and should work.

However, I can add an option like custom: webpack: packagerOptions: noFrozenLockFile: true that can be set in the serverless.yml. Imo it must default to freeze because of CI/CD stability but then you could disable the behavior easily for unstable testing environments.

@mzmiric5 :

but only when I use local packages with the file: reference […]

That might be a small different bug in the plugin because local references are rebased. I think the package lock file is not rebased properly in that case.

@clarsen Confirmed I just ran into the same issue. Mine was a difference in:

import { gql } from 'apollo-server-core';
// vs.
import { gql } from 'apollo-server-lambda';

Where "apollo-server-lambda": "2.0.5" is my actual package dependency. VS code auto-imported core and I didn’t even notice.

The noFrozenLockfile option would be very useful for a project I am currently working on. Do you know when it will be released?