terraform-provider-aws: ECS Service can't update desired replicas when Blue Green deployment is enabled

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave “+1” or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.26

  • provider.aws v2.65.0
  • provider.template v2.1.2

Affected Resource(s)

  • resource aws_ecs_service

Terraform Configuration Files

Debug Output

Expected Behavior

Should be able to update ECS Service desired_replicas even when deployment_controller.type is set to CODE_DEPLOY

Actual Behavior

  • API call fails with the following message:
Error: Error updating ECS Service (arn:aws:ecs:us-east-1:708286315144:service/test/policies-service-v2): InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment.

Steps to Reproduce

  • Create new ECS Service with deployment_controller.type = CODE_DEPLOY
  • Change Service desired_count and attempt to apply changes again

Important Factoids

  • Despite having no changes being made to network_configuration property, TF included it in the payload to update-service request
2020/06/08 04:05:32 [DEBUG] module.ecs.module.policies_service-v2.aws_ecs_service.service: applying the planned Update change
2020-06-08T04:05:32.935Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4: 2020/06/08 04:05:32 [DEBUG] Updating ECS Service (arn:aws:ecs:us-east-1:708286315144:service/test/policies-service-v2): {
2020-06-08T04:05:32.935Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:   Cluster: "arn:aws:ecs:us-east-1:708286315144:cluster/test",
2020-06-08T04:05:32.935Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:   DesiredCount: 0,
2020-06-08T04:05:32.935Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:   ForceNewDeployment: false,
2020-06-08T04:05:32.935Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:   NetworkConfiguration: {
2020-06-08T04:05:32.935Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:     AwsvpcConfiguration: {
2020-06-08T04:05:32.935Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:       AssignPublicIp: "DISABLED",
2020-06-08T04:05:32.935Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:       SecurityGroups: ["sg-02848434002f782f9"],
2020-06-08T04:05:32.935Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:       Subnets: ["subnet-014f47760b389f8db","subnet-03cdd43d8265511be","subnet-0f67bb732709250c6"]
2020-06-08T04:05:32.935Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:     }
2020-06-08T04:05:32.936Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:   },
2020-06-08T04:05:32.936Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4:   Service: "arn:aws:ecs:us-east-1:708286315144:service/test/policies-service-v2"
2020-06-08T04:05:32.936Z [DEBUG] plugin.terraform-provider-aws_v2.65.0_x4: }
  • TF Plan doesn’t detect any change to network_configuration
  • This problem isn’t only triggered by changes to desired_count and can by reproduced by making changes to task_placement_constaints too

References

For services using the blue/green (CODE_DEPLOY ) deployment controller, only the desired count, deployment configuration, task placement constraints and strategies, and health check grace period can be updated using this API. If the network configuration, platform version, or task definition need to be updated, a new AWS CodeDeploy deployment should be created. For more information, see CreateDeployment in the AWS CodeDeploy API Reference .

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 103
  • Comments: 19 (7 by maintainers)

Most upvoted comments

Having the same issues while trying to update the desired count of tasks, any updates on this?

Hi all,

what about updates on this issue ?

This is very important for us. We are running terraform-pipelines in production with ecs_service and CODE_DEPLOY deployment controller. Updates on the service with terraform are not possible over this way !

Having the same issue when trying to update the platform version.

I’m having the same issue, trying to add tags to a service.

In the plan only appears the tags being changed.

During the apply I get this error: “InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment.”

Including network_configuration as ignore_changes is not helping.

Could the PR made earlier this year, which references this issue, be reviewed? It’s a fairly small change. We believe this to be a good way to address this, having resorted to using our own fork with that PR to address this for now and can confirm this resolves it for us.

For visibility @bflad @gdavison

https://github.com/hashicorp/terraform-plugin-sdk/pull/711

FWIW, the way I was able to control desired_count is by adding an autoscaling group. I’m not using it to autoscale, since I have the min/max set to the same number and the cpu target_value 99. It just acts as a way to modify desired_count which seems to play well with terraform and the CODE_DEPLOY deployment_controller type:

resource "aws_iam_service_linked_role" "ecs_application_autoscaling" {
  aws_service_name = "ecs.application-autoscaling.amazonaws.com"
  description      = "Allows Application Auto Scaling to call ECS and CloudWatch on your behalf."
}

resource "aws_appautoscaling_target" "target" {
  # THIS CONTROLS THE DESIRED_COUNT
  max_capacity       = var.running_max
  min_capacity       = var.running_min
  resource_id        = "service/${aws_ecs_cluster.cluster.name}/${aws_ecs_service.app.name}"
  role_arn           = aws_iam_service_linked_role.ecs_application_autoscaling.arn
  scalable_dimension = "ecs:service:DesiredCount"
  service_namespace  = "ecs"
}

resource "aws_appautoscaling_policy" "cpu_tracking" {
  name               = "${aws_ecs_cluster.cluster.name}_${aws_ecs_service.app.name}_cpu_tracking"
  policy_type        = "TargetTrackingScaling"
  resource_id        = "service/${aws_ecs_cluster.cluster.name}/${aws_ecs_service.app.name}"
  scalable_dimension = aws_appautoscaling_target.target.scalable_dimension
  service_namespace  = aws_appautoscaling_target.target.service_namespace

  target_tracking_scaling_policy_configuration {
    predefined_metric_specification {
      predefined_metric_type = "ECSServiceAverageCPUUtilization"
    }

    target_value       = 99
    scale_in_cooldown  = 300
    scale_out_cooldown = 60
  }
}

I encountered the same problem. Including network_configuration as ignore_changes doesn’t work - network_configuration change doesn’t appear in the diff view, but it is included in the API call request when apply.

Looking into the debug log, I found the order of the subnets in the update request is different from the value of the current state.

networkConfiguration of current state (ecs/DescribeServices)
{"awsvpcConfiguration":{"assignPublicIp":"ENABLED","securityGroups":["sg-09b7377097c0c97fd"],"subnets":["subnet-012c6b3edd7abd4d9","subnet-0f5c632fc7e9db0ab","subnet-041643c250580e1dc"]}}
networkConfiguration of update request (ecs/UpdateService)
{"awsvpcConfiguration":{"assignPublicIp":"ENABLED","securityGroups":["sg-09b7377097c0c97fd"],"subnets":["subnet-041643c250580e1dc","subnet-012c6b3edd7abd4d9","subnet-0f5c632fc7e9db0ab"]}}

The attribute type of subnets is TypeSet, so the two values should be the same.

https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_ecs_service.go#L238

However, it seems d.hasChange(“network_configuration”) returns true somehow since the request includes the networkConfiguration attribute.

https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_ecs_service.go#L1027

I’m not familiar with the codebase so I might look at unrelated place though.

Hi @breathingdust, I’ve written the test case that replicates the issue. When I try to update the desired_count, the function resourceServiceUpdate says I’m also changing the NetworkConfiguration. Like @calimonk says, the solution to this issue is the release of the version v2.9.0 of the terraform-sdk-plugin. So I can open the PR with the test case that replicates the bug and update the go.mod when the terraform-plugin-sdk is released.

Cheers

Hi all 👋 Just letting you know that this is issue is featured on this quarters roadmap. If a PR exists to close the issue a maintainer will review and either make changes directly, or work with the original author to get the contribution merged. If you have written a PR to resolve the issue please ensure the “Allow edits from maintainers” box is checked. Thanks for your patience and we are looking forward to getting this merged soon!

I tried to use CODE_DEPLOY with terraform but got either unable to update task definition or unable to update network configuration. I can’t deploy anything. This is nightmare.

@ewbankkit let me update the branch and add further tests to cover all the fields allowed by the APIs