aws-cdk: [@aws-cdk/assets] docs issue - Negate exclude pattern doesn't work

I’m unable to use a negated exclude glob to create an asset for a Lambda function with a single source code file.

Reproduction Steps

new Function(this, "DeployAction", {
  runtime: Runtime.NODEJS_12_X,
  code: new AssetCode(__dirname, {
    exclude: ["!deploy.js"]
  }),
  handler: "deploy.handler",
});

Error Log

No error - the asset ZIP doesn’t respect the negated exclude (it contains deploy.js + the files I was trying to exclude).

The same exclude, but without negation (deploy.js) does work (deploy.js is excluded from the asset zip).

Environment

CLI Version : N/A (I don’t have a global CDK installation)
Framework Version: 1.39.0 (build 5d727c1)
Node.js Version: v12.13.1
OS : Mac OS 10.13.6
Language (Version): TypeScript 3.7.5

This is 🐛 Bug Report

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 18 (10 by maintainers)

Commits related to this issue

Update S3 deployment glob pattern to correctly exclude all files According to https://github.com/aws/aws-cdk/issues/9146#issuecomment-660469797, an exclude pattern should be preceded by a `'*'` glo... — committed to Tehnix/aws-cdk by Tehnix 4 years ago
Update S3 deployment glob pattern to correctly exclude all files According to https://github.com/aws/aws-cdk/issues/9146#issuecomment-660469797, an exclude pattern should be preceded by a `'*'` glo... — committed to Tehnix/aws-cdk by Tehnix 4 years ago
chore(s3-deployment): Exclude pattern in README example for prune was wrong (#9352) The documentation at https://docs.aws.amazon.com/cdk/api/latest/docs/aws-s3-deployment-readme.html#prune currently... — committed to aws/aws-cdk by Tehnix 4 years ago
chore(s3-deployment): Exclude pattern in README example for prune was wrong (#9352) The documentation at https://docs.aws.amazon.com/cdk/api/latest/docs/aws-s3-deployment-readme.html#prune currently... — committed to curtiseppel/aws-cdk by Tehnix 4 years ago
fix: Don't ignore any STAC repo files Works around <https://github.com/aws/aws-cdk/issues/9146>, which resulted in the entire stac-spec directory being missing from the Docker image. — committed to linz/geostore by l0b0 3 years ago
fix: Don't ignore any STAC repo files Works around <https://github.com/aws/aws-cdk/issues/9146>, which resulted in the entire stac-spec directory being missing from the Docker image. — committed to linz/geostore by l0b0 3 years ago

Most upvoted comments

@danielsharvey We’ve been trying with both GLOB and GIT with almost exactly the same outcomes. We still come up against the issue @Tehnix points out. ['*.*', '!*.html']

The closest we’ve come is using that method but with two issues:

It doesn’t exclude files WITHOUT an extension (files that appear to be like directories since they have no ext)
It leaves all the empty directories intact. Not sure if this would even be an issue or would S3 just ignore since there is no such thing as a directory in s3?

REALLY, need some direction here. More than willing to help work thru testing and troubleshooting if anyone has any ideas.

iDVB on Jun 11, 2021

Just a note that the CDK now supports IgnoreMode as part of Source.asset().

You get more standard behaviour with gitignore or dockerignore.

danielsharvey on Jan 8, 2021

I don’t know if I would say this is necessarily a “docs issue”. The file matching makes it rather difficult to match globs recursively. It’s great that we can use *.* to try to match files but not directories, but what happens if you have a folder that has a dot in it? As uncommon as that is, that’s a pretty big caveat to recursive pattern matching.

Perhaps you could use a trailing / as an indicator that you’re trying to match against directories, and otherwise only match files? So, I could use ["*", "!*.html"] to match only *.html files recursively; if I did ["*/", "!*.html"], it would only match *.html files in the root directory. This way, you could even exclude lists of specific folders: ["(folder1|folder2)/"]

The array configuration is misleading, as well. For example, you wouldn’t want to do ["*.*", "!*.html", "!*.css"], because it would only ever match *.css files, since that’s the last one in the array. A couple possibilities that I could see for handling that:

short-circuit the shouldExclude() loop when a negated match is found https://github.com/aws/aws-cdk/blob/cc2600f4f2e43ebe8cc7b57e58e9ed75a210dfc4/packages/%40aws-cdk/core/lib/fs/utils.ts#L18-L29
indicate somewhere that minimatch is used for glob matching. That way, people know they can do !*(.html|.css), or something similar. That still isn’t perfect, but it adds more flexibility, especially if combined with short-circuiting based on negated matches.

Js-Brecht on Oct 21, 2020

And !deploy.js means exclude everything not named deploy.js, right? Why would I exclude everything and then exclude everything again (except the file I want to include)?

On Mon, Jul 27, 2020, 8:12 AM Elad Ben-Israel notifications@github.com wrote:

I am confused:

exclude: [ ‘*’ ]

Means exclude all files.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aws/aws-cdk/issues/9146#issuecomment-664455944, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFOU4B65ROYYGSE5LBXPV3R5WKMLANCNFSM4O66OE2Q .

aecollver on Jul 27, 2020

Shouldn’t it be exclude: ['*', '!deploy.js']?

jogold on Jul 18, 2020