aws-cdk: aws-ecs: Resource handler returned message: "Invalid request provided: The target group with targetGroupArn does not have an associated load balancer."

Describe the bug

With this workaround for redirecting HTTP to HTTPS on an ALB, I indeterminately experience the following:

CoreStackDev | 28/70 | 5:02:27 PM | UPDATE_FAILED        | AWS::ECS::Service                             | WebAppSvc/FlaskAppService/ControlPlane/ApplicationLoadBalancerFargateSvc/AlbFargateSvc/Service/Service (WebAppSvcFlaskAppServiceControlPlaneApplicationLoadBalancerFargateSvcAlbFargateSvcServiceADE54F9D) Resource handler returned message: "Invalid request provided: The target group with targetGroupArn <redacted> does not have an associated load balancer. (Service: Ecs, Status Code: 400, Request ID: 91fd6146-d2f3-4534-bbdb-893469be2a1d) (SDK Attempt Count: 1)" (RequestToken: 25bc3e51-6110-4b81-8256-f4c02c60a863, HandlerErrorCode: InvalidRequest)

I tested redeploying multiple times (and changing the ECS Service, ALB, Task definition logicalids to recreate the resources) to no avail.

It looks like there’s either:

  1. A race condition (regression)? somewhere. What’s strange is this was working fine for a few days with multiple successful redeployments in between the original deployment and experiencing the error.
  2. Some other reason why cloudformation thinks an old ALB target group exists when it does not

After removing the listener redirection (lb.ListenerAction.redirect) the deployment succeeded again.

My ticket is related.

Regression Issue

  • Select this option if this issue appears to be a regression.

Last Known Working CDK Version

No response

Expected Behavior

Successful deployment

Current Behavior

Indeterminate Invalid request provided: The target group with targetGroupArn <arn> does not have an associated load balancer.

Reproduction Steps

  1. Deploy an ALB with ecs_patterns.ApplicationLoadBalancedFargateService with protocol=lb.ApplicationProtocol.HTTP,
  2. Use this workaround to redirect HTTP to HTTPSand deploy
  3. Deploy a few times…
  4. Maybe experience error?

Possible Solution

No response

Additional Information/Context

"userAgent": "cloudformation.amazonaws.com",
    "errorCode": "InvalidParameterException",
    "errorMessage": "The target group with <redacted>/CoreSt-WebAp-OBAY5UVSDTFW/5f79f6f39969fa92 does not have an associated load balancer.",
    "requestParameters": {
        "capacityProviderStrategy": [
            {
                "capacityProvider": "FARGATE",
                "weight": 1,
                "base": 1
            }
        ],
        "cluster": "CoreStackDev-WebAppSvcFlaskAppServiceControlPlaneEcsClusterSvcEcsClusterEcsClusterSvcB291F355-xtUAqFpdBbkj",
        "deploymentConfiguration": {
            "deploymentCircuitBreaker": {
                "enable": true,
                "rollback": true
            },
            "maximumPercent": 200,
            "minimumHealthyPercent": 100,
            "alarms": {
                "rollback": false,
                "enable": false
            }
        },
        "desiredCount": 1,
        "enableECSManagedTags": true,
        "enableExecuteCommand": true,
        "availabilityZoneRebalancing": "DISABLED",
        "forceNewDeployment": false,
        "healthCheckGracePeriodSeconds": 60,
        "loadBalancers": [
            {
                "targetGroupArn": "<redacted>/CoreSt-WebAp-OBAY5UVSDTFW/5f79f6f39969fa92",
                "containerName": "EcsTaskDefContainerSvcEcsContainerDef",
                "containerPort": 80
            },
            {
                "targetGroupArn": "<redacted>/CoreSt-WebAp-XB77YZBVGDH8/559a5660e756f528",
                "containerName": "EcsTaskDefContainerSvcEcsContainerDef",
                "containerPort": 80
            }
        ],

CDK CLI Version

2.180.0

Framework Version

No response

Node.js Version

v22.12.0

OS

Mac

Language

Python

Language Version

No response

Other information

No response

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 16 (8 by maintainers)

Most upvoted comments

@BwL1289 thank you patience while we dive into this issue. I was successfully able to reproduce the “The target group with targetGroupArn does not have an associated load balancer” error. Here’s a detailed reproduction and explanation of the issue:

Reproduction Code:

// Create HTTP listener with target group first
const httpListener = loadBalancer.addListener('HttpListener', {
  port: 80,
  open: true,
  defaultTargetGroups: [targetGroup], // Associate target group with listener
});

// Create HTTPS listener with the same target group
const httpsListener = loadBalancer.addListener('HttpsListener', {
  port: 443,
  certificates: [certificate],
  open: true,
  defaultTargetGroups: [targetGroup],
});

// Create Fargate service
const fargateService = new ecs.FargateService(this, 'Service', {
  cluster,
  taskDefinition,
  desiredCount: 1,
  assignPublicIp: false,
});

// Register service with target group
targetGroup.addTarget(fargateService);

// PROBLEM TRIGGER: Modifying HTTP listener to redirect AFTER the target group 
// has been associated and AFTER the service has been registered
httpListener.addAction('HttpRedirect', {
  action: lb.ListenerAction.redirect({
    port: '443',
    protocol: 'HTTPS',
    permanent: true,
  }),
});

During deployment, we get this warning:

[Warning at /MyProjectStack/ALB/HttpListener] A default Action already existed on this Listener and was replaced.`

And we get the error:

Resource handler returned message: "Invalid request provided: The target group with targetGroupArn does not have an associated load balancer. (Service: Ecs, Status Code: 400)"

Console testing:

  1. Create a ALB with 2 listeners HTTP and HTTPS. Added them to the same target group.
  2. Once ALB is created. Select the HTTP:80 listener(checkbox)
  3. click, Manage Listener -> Edit listener

Image 4. save changes.

No errors observed on console

Root Cause The issue occurs due to the sequence of operations:

  1. We create a target group and associate it with an HTTP listener
  2. When we later modify the HTTP listener to redirect to HTTPS, we’re replacing its default action
  3. This disrupts the dependency tracking between the target group and load balancer in CDK
  4. When deploying, CloudFormation’s ordering of operations can sometimes cause the ECS service to be created/updated before the target group is fully associated with a load balancer

The warning about replacing a default action does appear to be the key indicator for this issue. But similar to https://github.com/aws/aws-cdk/issues/34235 modifying listeners after creation, appears to be causing this error. Marking as P1 Bug