serverless: Outage: Version 4.1.13 Breaking Changes (Deployments + Building)

Issue description

As of one hour ago, all of our SLS commands have broken. It seems like the inclusion/exclusion logic changes are causing issues.

image

the errors pop up for non aws-sdk things too (ex. “semver” or “datadog-lambda-js”)

All of our CI is essentially broken 😦

Two asks:

  1. Most important How can we pin to a specific version of serverless? This is the second time now where deployments broke from a bad version release and as far as I can tell, there’s no way to turn off the auto updating of serverless4. I already tried pinning to version 4.1.12 in our package.json but it still installs 4.1.13 under the hood which makes it impossible for us to mitigate this issue ourselves right now. Please tell me if I’m missing something!
  2. Request to help mitigate the underlying issues in 4.1.13 so we can eventually roll forward

Context

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 39 (18 by maintainers)

Most upvoted comments

@NicoHinderling A new version has finally been released. If you want to run a serverless update and try it again?

Thanks @NicoHinderling sorry this took so long, wanted to verify everything before publishing a release. A new release is coming shortly.

@NicoHinderling aws-sdk/* is not the module namespace, it is @aws-sdk/*

@NicoHinderling Wanted to circle back on this, We release 4.1.17 today that includes a change that will keep @aws-sdk/* in the exclude/external lists if you explicitly define it.

As for version pinning, we are putting the finishing touches on this and testing it internally, but expect a release with that functionality in the next few days.

@NicoHinderling Yeah I’m still working on this. Sorry for the confusion here, the ticket was moved in our internal Linear and it closed when it shouldn’t have.

Well, I think it is an issue with us not with you. For an immediate fix you could remove that as a dependency and it should work without the work around from last night.

Basically what we do, which I think is a bit erroneous, is if any @aws-sdk/ modules are in the package.json we include those and then remove the wildcard one. I need to think through how to address this so we support both cases, will provide an update here when I come up with a solution.

happy monday 😂 I saw that linearbot closed this task, just want to make sure in addition to version pinning, this ^ is still in the works to be resolved as well, right? 🙏

Version pinning is now in the works. Tracking it here: https://github.com/serverless/serverless/issues/12692

sounds great, looking fwd to help test!

Also just curious, have you and the team discussed the ability to disable auto-update? We’d like to make sure we can self-mitigate a situation like this in the future 🙏

Well, I think it is an issue with us not with you. For an immediate fix you could remove that as a dependency and it should work without the work around from last night.

Basically what we do, which I think is a bit erroneous, is if any @aws-sdk/ modules are in the package.json we include those and then remove the wildcard one. I need to think through how to address this so we support both cases, will provide an update here when I come up with a solution.

@noahsmartin you should be good if you are on version 4.1.15 or higher.

Glad that that is at least a workaround for you for now @NicoHinderling. I did go ahead and publish 4.1.16 with some extra debug logs. At some point when you are able to if you could run your deploy command with the --debug flag and without the dependencies all in the external & exclude block, and then send me any log lines that start with the prefix s:esbuild, might make it possible to track down what specifically is happening to just that one service of yours.

So what should be happening is that the pattern @aws-sdk/* gets auto added to your external & exclude block if you are using node18 or higher as your runtime. For some reason though it seems like that isn’t happening for you.

What you could try doing to see if it is works is adding @aws-sdk/* to your excludes block in your esbuild config.

@NicoHinderling are you able to share what your esbuild configuration is? I believe I know the issue but want to confirm based on what you are doing.

We’re looking into this right now…