aws-cdk: ‼️ NOTICE: 'Error: ENOTDIR invalid cwd' on `cdk deploy` in 1.90.0

Please add your +1 👍 to let us know you have encountered this

Update

Version 1.90.1 which includes a fix is now released.


There is an issue with the way the CDK assets mechanism identifies already .zip file. Currently it will try to zip the already zipped file that will throw an error. We are working on a fix.

Workaround

TL;DR: delete the cdk.out folder before executing cdk deploy

The issue occurs when executing cdk deploy when there is a .zip file in the cdk.out folder. This happens when executing cdk synth - which creates the cdk.out folder, followed by cdk deploy (which is a completely normal cdk flow). To workaround this, delete the cdk.out folder before executing cdk deploy.

Repro

Any application which includes an asset with a .zip file will fail to deploy:

new lambda.Function(this, 'MyLambda', {
      code: lambda.Code.fromAsset(path.join(__dirname, '../my-zip.zip')),
      handler: 'index.handler',
      runtime: lambda.Runtime.NODEJS_12_X
 })

important: execute cdk synth before executing cdk deploy

cdk deploy will throw:

UnhandledPromiseRejectionWarning: Error: ENOTDIR invalid cwd 

Status: Working on a patch release

Overview:

When running cdk deploy in version 1.90.0 on a Stack that contains assets, you might see an error like 'Error: ENOTDIR invalid cwd.

Complete Error Message:

$ npx cdk deploy --require-approval=never 
Bug13131Stack: deploying...
[0%] start: Publishing e9882ab123687399f934da0d45effe675ecc8ce13b40cb946f3e1d6141fe8d68:current
(node:81012) UnhandledPromiseRejectionWarning: Error: ENOTDIR invalid cwd /Users/adamruka/workplace/cdk/on-call/bug-13131/cdk.out/asset.e9882ab123687399f934da0d45effe675ecc8ce13b40cb946f3e1d6141fe8d68.zip
    at GlobSync._readdirError (/Users/adamruka/workplace/cdk/on-call/bug-13131/node_modules/aws-cdk/node_modules/glob/sync.js:324:21)
    at GlobSync._readdir (/Users/adamruka/workplace/cdk/on-call/bug-13131/node_modules/aws-cdk/node_modules/glob/sync.js:290:10)
    at GlobSync._processGlobStar (/Users/adamruka/workplace/cdk/on-call/bug-13131/node_modules/aws-cdk/node_modules/glob/sync.js:350:22)
    at GlobSync._process (/Users/adamruka/workplace/cdk/on-call/bug-13131/node_modules/aws-cdk/node_modules/glob/sync.js:130:10)
    at new GlobSync (/Users/adamruka/workplace/cdk/on-call/bug-13131/node_modules/aws-cdk/node_modules/glob/sync.js:48:10)
    at Function.globSync [as sync] (/Users/adamruka/workplace/cdk/on-call/bug-13131/node_modules/aws-cdk/node_modules/glob/sync.js:26:10)
    at Promise (/Users/adamruka/workplace/cdk/on-call/bug-13131/node_modules/aws-cdk/node_modules/cdk-assets/lib/private/archive.ts:17:24)
    at new Promise (<anonymous>)
    at Object.zipDirectory (/Users/adamruka/workplace/cdk/on-call/bug-13131/node_modules/aws-cdk/node_modules/cdk-assets/lib/private/archive.ts:7:10)
    at FileAssetHandler.packageFile (/Users/adamruka/workplace/cdk/on-call/bug-13131/node_modules/aws-cdk/node_modules/cdk-assets/lib/private/handlers/files.ts:78:13)
(node:81012) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:81012) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Original bug report

My CDK script was all right, but was failing with the latest 1.90.0 version

Reproduction Steps

Run the CDK script via gitlab pipeline

What did you expect to happen?

No errors

What actually happened?

See the error message as below:

xxxx- infra-Dev: deploying...
[0%] start: Publishing 45f6bfc57a7695260616f2916fbef2dd77200af18743d956a59c8db9c3816b3f:current
(node:55) UnhandledPromiseRejectionWarning: Error: ENOTDIR invalid cwd /builds/xxxx/cdk.out/asset.45f6bfc57a7695260616f2916fbef2dd77200af18743d956a59c8db9c3816b3f.sh
    at GlobSync._readdirError (/usr/local/lib/node_modules/aws-cdk/node_modules/glob/sync.js:324:21)
    at GlobSync._readdir (/usr/local/lib/node_modules/aws-cdk/node_modules/glob/sync.js:290:10)
    at GlobSync._processGlobStar (/usr/local/lib/node_modules/aws-cdk/node_modules/glob/sync.js:350:22)
    at GlobSync._process (/usr/local/lib/node_modules/aws-cdk/node_modules/glob/sync.js:130:10)
    at new GlobSync (/usr/local/lib/node_modules/aws-cdk/node_modules/glob/sync.js:48:10)
    at Function.globSync [as sync] (/usr/local/lib/node_modules/aws-cdk/node_modules/glob/sync.js:26:10)
    at /usr/local/lib/node_modules/aws-cdk/node_modules/cdk-assets/lib/private/archive.ts:17:24
    at new Promise (<anonymous>)
    at Object.zipDirectory (/usr/local/lib/node_modules/aws-cdk/node_modules/cdk-assets/lib/private/archive.ts:7:10)
    at FileAssetHandler.packageFile (/usr/local/lib/node_modules/aws-cdk/node_modules/cdk-assets/lib/private/handlers/files.ts:78:13)
(node:55) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:55) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Environment
  • CDK CLI Version : 1.90.0
  • **Framework Version: **1.90.0
  • **Node.js Version: **12
  • OS : Docker
  • Language (Version): Python (3.8)
Other

Rollback to 1.89.0 fixed the problem


This is 🐛 Bug Report

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 30
  • Comments: 35 (14 by maintainers)

Commits related to this issue

Most upvoted comments

Added a workaround to the issue description, TL;DR: delete the cdk.out folder before executing cdk deploy.

A patch is on the way!

Confirming I was able to reproduce this. We’re working on a fix.

Thanks everyone, we found the problem and we are working on a fix. The problem was introduced in version 1.90.0 and affect all node version. We are on it

Node v14.15.5 for us, which is the latest default node version on GitHub Actions Linux Agents and Azure DevOps Linux Agents (per https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu2004-README.md).

Thanks @jogold. No rush. We reverted the other commit just because there were some merge conflicts.

@NetaNir @eladb found the bug and running synth two times will also reproduce it.

After the first synth the packaging is correctly set to file in the manifest.json. But then the asset cache comes into play and the second time it runs packaging is now set to zip in the manifest.json.

This is because staging doesn’t rerun: https://github.com/aws/aws-cdk/blob/cd7a3ed333570a3b26446e1e3a054ca886cd3906/packages/%40aws-cdk/core/lib/asset-staging.ts#L191 and the FileAssetPackaging is set in the staging private methods not in the constructor.

On it for a redo.

This is only #13076.

@NetaNir , I can confirm my deployment worked with 1.89

I am using node v14.15.0

Are we missing some test cases to prevent this from happening again?