aws-cdk: @aws-cdk/aws-lambda-python-alpha: Bundling lambdas with Poetry is broken

Describe the bug

Our CI started breaking yesterday when trying to synthesise stacks using the PythonFunction construct. I was able to reproduce the failure locally using both aws-lambda-python-alpha in v2 and aws-lambda-python in v1.

The failure occurs when trying to setup the virtual env inside the bundling docker image. I assume something has changed in the latest aws/sam/build-python3.9 docker image that breaks the bundling command.

EDIT: we were using Python 3.9 lambdas, but I tested and builds also fail for 3.7 and 3.8 runtimes.

Expected Behavior

cdk synth should succeed in bundling the lambda function.

Current Behavior

The lambda bundling fails with an error from virtualenv: virtualenv: error: argument dest: the destination . is not write-able at /

Full error output

❯ cdk synth
[+] Building 1.1s (7/7) FINISHED
 => [internal] load build definition from Dockerfile                                                                                      0.0s
 => => transferring dockerfile: 559B                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                         0.0s
 => => transferring context: 2B                                                                                                           0.0s
 => [internal] load metadata for public.ecr.aws/sam/build-python3.9:latest                                                                1.0s
 => [1/3] FROM public.ecr.aws/sam/build-python3.9@sha256:aa96722319b3838e27f79b8c90a8e14352695669454172724b8619b1718d9b25                 0.0s
 => CACHED [2/3] RUN pip install --upgrade pip                                                                                            0.0s
 => CACHED [3/3] RUN pip install pipenv==2022.4.8 poetry                                                                                  0.0s
 => exporting to image                                                                                                                    0.0s
 => => exporting layers                                                                                                                   0.0s
 => => writing image sha256:3b54578b8b1e23782a5e30c7ef1c9fd2b69edecca9a2eed9a82ec87b09291b64                                              0.0s
 => => naming to docker.io/library/cdk-3610f113cfbf35f4b4e4218bc72d3b9bac4c71d7137512ba8d0302db2ba09a5b                                   0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Bundling asset Cdktestv2Stack/Lambda/Code/Stage...
Creating virtualenv dummy-Plo1f0Cp-py3.9 in /.cache/pypoetry/virtualenvs
usage: virtualenv [--version] [--with-traceback] [-v | -q] [--read-only-app-data] [--app-data APP_DATA] [--reset-app-data] [--upgrade-embed-wheels] [--discovery {builtin}] [-p py] [--try-first-with py_exe]
                  [--creator {builtin,cpython3-posix,venv}] [--seeder {app-data,pip}] [--no-seed] [--activators comma_sep_list] [--clear] [--no-vcs-ignore] [--system-site-packages] [--symlinks | --copies] [--no-download | --download]
                  [--extra-search-dir d [d ...]] [--pip version] [--setuptools version] [--wheel version] [--no-pip] [--no-setuptools] [--no-wheel] [--no-periodic-update] [--symlink-app-data] [--prompt prompt] [-h]
                  dest
virtualenv: error: argument dest: the destination . is not write-able at /
/Users/weird/scratch/cdktestv2/node_modules/aws-cdk-lib/core/lib/asset-staging.js:2
`),localBundling=options.local?.tryBundle(bundleDir,options),!localBundling){let user;if(options.user)user=options.user;else{const userInfo=os.userInfo();user=userInfo.uid!==-1?`${userInfo.uid}:${userInfo.gid}`:"1000:1000"}options.image.run({command:options.command,user,volumes,environment:options.environment,entrypoint:options.entrypoint,workingDirectory:options.workingDirectory??AssetStaging.BUNDLING_INPUT_DIR,securityOpt:options.securityOpt??""})}}catch(err){const bundleErrorDir=bundleDir+"-error";throw fs.existsSync(bundleErrorDir)&&fs.removeSync(bundleErrorDir),fs.renameSync(bundleDir,bundleErrorDir),new Error(`Failed to bundle asset ${this.node.path}, bundle output is located at ${bundleErrorDir}: ${err}`)}if(fs_1.FileSystem.isEmpty(bundleDir)){const outputDir=localBundling?bundleDir:AssetStaging.BUNDLING_OUTPUT_DIR;throw new Error(`Bundling did not produce any output. Check that content is written to ${outputDir}.`)}}calculateHash(hashType,bundling,outputDir){if(hashType==assets_1.AssetHashType.CUSTOM||hashType==assets_1.AssetHashType.SOURCE&&bundling){const hash=crypto.createHash("sha256");return hash.update(this.customSourceFingerprint??fs_1.FileSystem.fingerprint(this.sourcePath,this.fingerprintOptions)),bundling&&hash.update(JSON.stringify(bundling)),hash.digest("hex")}switch(hashType){case assets_1.AssetHashType.SOURCE:return fs_1.FileSystem.fingerprint(this.sourcePath,this.fingerprintOptions);case assets_1.AssetHashType.BUNDLE:case assets_1.AssetHashType.OUTPUT:if(!outputDir)throw new Error(`Cannot use \`${hashType}\` hash type when \`bundling\` is not specified.`);return fs_1.FileSystem.fingerprint(outputDir,this.fingerprintOptions);default:throw new Error("Unknown asset hash type.")}}}exports.AssetStaging=AssetStaging,_a=JSII_RTTI_SYMBOL_1,AssetStaging[_a]={fqn:"aws-cdk-lib.AssetStaging",version:"2.39.0"},AssetStaging.BUNDLING_INPUT_DIR="/asset-input",AssetStaging.BUNDLING_OUTPUT_DIR="/asset-output",AssetStaging.assetCache=new cache_1.Cache;function renderAssetFilename(assetHash,extension=""){return`asset.${assetHash}${extension}`}function determineHashType(assetHashType,customSourceFingerprint){const hashType=customSourceFingerprint?assetHashType??assets_1.AssetHashType.CUSTOM:assetHashType??assets_1.AssetHashType.SOURCE;if(customSourceFingerprint&&hashType!==assets_1.AssetHashType.CUSTOM)throw new Error(`Cannot specify \`${assetHashType}\` for \`assetHashType\` when \`assetHash\` is specified. Use \`CUSTOM\` or leave \`undefined\`.`);if(hashType===assets_1.AssetHashType.CUSTOM&&!customSourceFingerprint)throw new Error("`assetHash` must be specified when `assetHashType` is set to `AssetHashType.CUSTOM`.");return hashType}function calculateCacheKey(props){return crypto.createHash("sha256").update(JSON.stringify(sortObject(props))).digest("hex")}function sortObject(object){if(typeof object!="object"||object instanceof Array)return object;const ret={};for(const key of Object.keys(object).sort())ret[key]=sortObject(object[key]);return ret}function singleArchiveFile(directory){if(!fs.existsSync(directory))throw new Error(`Directory ${directory} does not exist.`);if(!fs.statSync(directory).isDirectory())throw new Error(`${directory} is not a directory.`);const content=fs.readdirSync(directory);if(content.length===1){const file=path.join(directory,content[0]),extension=getExtension(content[0]).toLowerCase();if(fs.statSync(file).isFile()&&ARCHIVE_EXTENSIONS.includes(extension))return file}}function determineBundledAsset(bundleDir,outputType){const archiveFile=singleArchiveFile(bundleDir);switch(outputType===bundling_1.BundlingOutput.AUTO_DISCOVER&&(outputType=archiveFile?bundling_1.BundlingOutput.ARCHIVED:bundling_1.BundlingOutput.NOT_ARCHIVED),outputType){case bundling_1.BundlingOutput.NOT_ARCHIVED:return{path:bundleDir,packaging:assets_1.FileAssetPackaging.ZIP_DIRECTORY};case bundling_1.BundlingOutput.ARCHIVED:if(!archiveFile)throw new Error("Bundling output directory is expected to include only a single archive file when `output` is set to `ARCHIVED`");return{path:archiveFile,packaging:assets_1.FileAssetPackaging.FILE,extension:getExtension(archiveFile)}}}function getExtension(source){for(const ext of ARCHIVE_EXTENSIONS)if(source.toLowerCase().endsWith(ext))return ext;return path.extname(source)}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     ^
Error: Failed to bundle asset Cdktestv2Stack/Lambda/Code/Stage, bundle output is located at /Users/weird/scratch/cdktestv2/cdk.out/asset.52019e0cc4e1d6c178c5252bcb442b7407ccbc065d9f9470adcfc1fd279d930c-error: Error: docker exited with status 2
    at AssetStaging.bundle (/Users/weird/scratch/cdktestv2/node_modules/aws-cdk-lib/core/lib/asset-staging.js:2:614)
    at AssetStaging.stageByBundling (/Users/weird/scratch/cdktestv2/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:4314)
    at stageThisAsset (/Users/weird/scratch/cdktestv2/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:1675)
    at Cache.obtain (/Users/weird/scratch/cdktestv2/node_modules/aws-cdk-lib/core/lib/private/cache.js:1:242)
    at new AssetStaging (/Users/weird/scratch/cdktestv2/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:2070)
    at new Asset (/Users/weird/scratch/cdktestv2/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:736)
    at AssetCode.bind (/Users/weird/scratch/cdktestv2/node_modules/aws-cdk-lib/aws-lambda/lib/code.js:1:4628)
    at new Function (/Users/weird/scratch/cdktestv2/node_modules/aws-cdk-lib/aws-lambda/lib/function.js:1:2803)
    at new PythonFunction (/Users/weird/scratch/cdktestv2/node_modules/@aws-cdk/aws-lambda-python-alpha/lib/function.ts:73:5)
    at new Cdktestv2Stack (/Users/weird/scratch/cdktestv2/lib/cdktestv2-stack.ts:10:15)

Subprocess exited with error 1

Reproduction Steps

lib/cdktestv2-stack.ts:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { PythonFunction } from '@aws-cdk/aws-lambda-python-alpha';
import { Runtime } from 'aws-cdk-lib/aws-lambda';

export class Cdktestv2Stack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const f = new PythonFunction(this, 'Lambda', {
      runtime: Runtime.PYTHON_3_9,
      entry: './lambda',
    })
  }
}

bin/cdktestb2.ts:

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { Cdktestv2Stack } from '../lib/cdktestv2-stack';

const app = new cdk.App();
new Cdktestv2Stack(app, 'Cdktestv2Stack');

lambda/pyproject.toml:

[tool.poetry]
name = "dummy"
version = "1.0.0"
description = "Some lambda function"
authors = ["Daryl"]

[tool.poetry.dependencies]
python = "~3.9"
httpx = "==0.23.0"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

lambda/index.py:

import httpx

def handler(event, context):
    r = httpx.get("https://www.example.org")
    print(r.status_code)

and poetry install in lambda to create a lock file.

Then run cdk synth and watch it fall over.

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.39.0

Framework Version

No response

Node.js Version

16.14.0

OS

MacOS 10.15.7

Language

Typescript, Python

Language Version

Typescript 3.9.7, Python 3.9.1

Other information

No response