aws-cdk: (custom-resources): StateNotFoundError: State functionActiveV2 not found

Please add your +1 👍 to let us know you have encountered this

Status: RESOLVED

Overview:

Any customer using custom resources may encounter the error when the custom resource handler lambda becomes INACTIVE. Root cause: Lambda installs the SDK at 2.1055.0, but functionActiveV2 doesn’t exist until 2.1080.0. It was reported in ap-south-2 but can happen in any region.

The error occurs anytime the custom resource provider framework fails to invoke the custom resource handler lambda. In that event, the framework will use functionActiveV2 to wait for the lambda to become active again. However, the call to functionActiveV2 will fail in the provider lambda because of root cause described above.

Complete Error Message:

Received response status [FAILED] from custom resource. Message returned: StateNotFoundError: State functionActiveV2 not found.

Workaround:

Solution:

use functionActive instead.

https://github.com/aws/aws-cdk/pull/25228

Related Issues:

https://github.com/aws/aws-cdk/issues/23862#issuecomment-1445756934

Original Issue

Describe the bug

When deploying 10+ custom resources in ap-south-2, it fails with StateNotFoundError: State functionActiveV2 not found error as below:

1. It’s fine to deploy 10+ custom resources in other AWS regions such as us-east-1 or ap-northeast-1. Only ap-south-2 fails in this case.
2. When deploying with <10 custom resources, ap-south-2 will be fine with no error.

Expected Behavior

The provided code above should deploy in ap-south-2.

Current Behavior

Reproduction Steps

import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import * as lambda from "aws-cdk-lib/aws-lambda";
import { CustomResource } from "aws-cdk-lib";
import * as cr from "aws-cdk-lib/custom-resources";

export class DummyLambdaStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const onEventHandler = new lambda.Function(this, 'OnEventHandler', {
      runtime: lambda.Runtime.NODEJS_18_X,
      code: lambda.Code.fromInline(`exports.onEvent = () => { return }`),
      handler: 'index.onEvent',
    })

    const provider = new cr.Provider(this, "Provider", {
      onEventHandler,
    });

    const numResources = 10
    for (let i = 0; i < numResources; i++) {
      new CustomResource(this, `CR${i}`, {
        serviceToken: provider.serviceToken,
      });
    }
  }
}

Possible Solution

There might be some restrictions in ap-south-2.

Additional Information/Context

No response

CDK CLI Version

2.66.1 (build 539d036)

Framework Version

No response

Node.js Version

v16.17.0

OS

Linux

Language

Typescript

Language Version

No response

Other information

No response