terraform-provider-aws: aws_wafv2_web_acl - Error: Provider produced inconsistent final plan

Community Note

  • Please vote on this issue by adding a πŸ‘ reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave β€œ+1” or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Related:

Terraform CLI and Terraform AWS Provider Version

Terraform version 1.1.7 provider registry.terraform.io/hashicorp/aws v4.8.0

Affected Resource(s)

  • aws_wafv2_web_acl

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_wafv2_ip_set" "ip_whitelist" {
  name = "IP-whitelist"
  provider  = aws.virginia
  scope = "CLOUDFRONT"
  ip_address_version = "IPV4"
  addresses = var.stage == "prod" || var.stage == "cons" ? ["xxx.xxx.xxx.xxx/32"] : ["0.0.0.0/1", "128.0.0.0/1"]
}

resource "aws_wafv2_web_acl" "whitelist_waf_acls" {
  name = "Whitelist-WAF-ACLs"
  provider = aws.virginia
  scope = "CLOUDFRONT"

  default_action {
    block {}
  }

  rule {
    name = "Whitelist-WAF-ACLs-rule"
    priority = 0

    action {
      allow {}
    }

    statement {
      ip_set_reference_statement { 
        arn = aws_wafv2_ip_set.ip_whitelist.arn
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name = "Whitelist-WAF-ACLs-rule"
      sampled_requests_enabled = true
    }
  }

  rule {
    name     = "AWSManagedRulesBotControlRuleSet"
    priority = 1

    override_action {
      count {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesBotControlRuleSet"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "Whitelist-WAF-ACLs-menaged-rule"
      sampled_requests_enabled   = true
    }
  }

  rule {
    name     = "AWSManagedRulesAdminProtectionRuleSet"
    priority = 2

    override_action {
      count {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesAdminProtectionRuleSet"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "Whitelist-WAF-ACLs-menaged-rule"
      sampled_requests_enabled   = true
    }
  }

  rule {
    name     = "AWSManagedRulesAmazonIpReputationList"
    priority = 3

    override_action {
      count {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesAmazonIpReputationList"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "Whitelist-WAF-ACLs-menaged-rule"
      sampled_requests_enabled   = true
    }
  }

  rule {
    name     = "AWSManagedRulesAnonymousIpList"
    priority = 4

    override_action {
      count {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesAnonymousIpList"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "Whitelist-WAF-ACLs-menaged-rule"
      sampled_requests_enabled   = true
    }
  }

  rule {
    name     = "AWSManagedRulesCommonRuleSet"
    priority = 5

    override_action {
      count {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesCommonRuleSet"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "Whitelist-WAF-ACLs-menaged-rule"
      sampled_requests_enabled   = true
    }
  }

  rule {
    name     = "AWSManagedRulesKnownBadInputsRuleSet"
    priority = 6

    override_action {
      count {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesKnownBadInputsRuleSet"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "Whitelist-WAF-ACLs-menaged-rule"
      sampled_requests_enabled   = true
    }
  }

  rule {
    name     = "AWSManagedRulesSQLiRuleSet"
    priority = 7

    override_action {
      count {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesSQLiRuleSet"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "Whitelist-WAF-ACLs-menaged-rule"
      sampled_requests_enabled   = true
    }
  }

  visibility_config {
    cloudwatch_metrics_enabled = true
    metric_name = "Whitelist-WAF-ACLs-metric"
    sampled_requests_enabled = true
  }

  tags = {
    Name = "whitelist_waf_acls"
  }

  depends_on = [aws_wafv2_ip_set.ip_whitelist]
}

Debug Output

Panic Output

Expected Behavior

I am trying to make a update to the config, this particular change is for removing tags

Terraform will perform the following actions:

  # aws_wafv2_web_acl.whitelist_waf_acls will be updated in-place
  ~ resource "aws_wafv2_web_acl" "whitelist_waf_acls" {

        id         = "xxxxxx-xxxxxx-xxxxx-xxxxxx"

        name       = "Whitelist-WAF-ACLs"

      ~ tags       = {
          - "Application" = "xxxx" -> null
          - "Cost Center" = "xxxx" -> null
          - "Environment" = "dev" -> null
          - "Owner"       = "xxxxx@xxx.xx" -> null
          - "Project"     = "xxxxx" -> null
          - "TeamName"    = "xxxxx" -> null
          - "Entity" = "xxx" -> null
            # (1 unchanged element hidden)
        }
      ~ tags_all   = {
          - "Application" = "xxxx" -> null
          - "Cost Center" = "xxx" -> null
          - "Environment" = "dev" -> null
          - "Owner"       = "xxxx@xx.xx" -> null
          - "Project"     = "xxxx" -> null
          - "TeamName"    = "xxxx" -> null
          - "Entity" = "xxx" -> null
            # (1 unchanged element hidden)
        }

Actual Behavior

It does not deploy, it throws the error;

When expanding the plan for aws_wafv2_web_acl.whitelist_waf_acls to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .rule: planned set element

cty.ObjectVal(map[string]cty.Value{"action":cty.ListValEmpty(cty.Object(map[string]cty.Type{"allow":cty.List(cty.Object(map[string]cty.Type{"custom_request_handling":cty.List(cty.Object(map[string]cty.Type{"insert_header":cty.Set(cty.Object(map[string]cty.Type{"name":cty.String, "value":cty.String}))}))})), "block":cty.List(cty.Object(map[string]cty.Type{"custom_response":cty.List(cty.Object(map[string]cty.Type{"custom_response_body_key":cty.String, "response_code":cty.Number,
.
.
.
"text_transformation":cty.Set(cty.Object(map[string]cty.Type{"priority":cty.Number, "type":cty.String}))}))})}), "visibility_config":cty.ListVal([]cty.Value{cty.ObjectVal(map[string]cty.Value{"cloudwatch_metrics_enabled":cty.True, "metric_name":cty.StringVal("Whitelist-WAF-ACLs-menaged-rule"), "sampled_requests_enabled":cty.True})})}) does not correlate with any element in actual.

 This is a bug in the provider, which should be reported in the provider's own issue tracker.

Steps to Reproduce

  1. terraform apply

Important Factoids

References

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 120
  • Comments: 25 (6 by maintainers)

Most upvoted comments

when can this be fixed?

FYI: This issues with WAFv2 has been reported multiple times:

this is happening to me on the latest 4.23.0 aws provider version.

Bumping this to keep the bots at bay.

Running into this issue on multiple environments. Current workaround for me is to just not modify any tags, which is only a short-term solution. Ideally, it looks like this just needs fixed.

Every time we apply, we update a timestamp tag which is in the provider defaults tags, if there are changes to the WAF rules, we can apply without issues, but if the only change is a tag being updated we get this panic. I noticed this after upgrading from this provider from v3 to v4, we haven’t changed the use of aws_wafv2_web_acl but now we get a HUGE error message (30+ pages of unhelpful error text) when ever we try to apply our resource without any changes (other than a minor tag update)

Upgrading terraform to 1.4.x fixed this for me

I also face this issue when I try to add more rule by using the dynamic block

Error: Provider produced inconsistent final plan
β”‚
β”‚ When expanding the plan for module.waf_alb.aws_wafv2_web_acl.this to include new values learned so far during apply, provider
β”‚ "registry.terraform.io/hashicorp/aws" produced an invalid new value for .rule: planned set element
β”‚ cty.ObjectVal(map[string]cty.Value{"action":cty.ListValEmpty(cty.Object(map[string]cty.Type{"allow":cty.List(cty.Object(map[string]cty.Type{"custom_request_handling":cty.List(cty.Object(map[string]cty.Type{"insert_header":cty.Set(cty.Object(map[string]cty.Type{"name":cty.String,
β”‚ "value":cty.String}))}))})),
β”‚ "block":cty.List(cty.Object(map[string]cty.Type{"custom_response":cty.List(cty.Object(map[string]cty.Type{"custom_response_body_key":cty.String,
β”‚ "response_code":cty.Number, "response_header":cty.Set(cty.Object(map[string]cty.Type{"name":cty.String, "value":cty.String}))}))})),
β”‚ "captcha":cty.List(cty.Object(map[string]cty.Type{"custom_request_handling":cty.List(cty.Object(map[string]cty.Type{"insert_header":cty.Set(cty.Object(map[string]cty.Type{"name":cty.String,
β”‚ "value":cty.String}))}))})),
β”‚ "challenge":cty.List(cty.Object(map[string]cty.Type{"custom_request_handling":cty.List(cty.Object(map[string]cty.Type{"insert_header":cty.Set(cty.Object(map[string]cty.Type{"name":cty.String,
β”‚ "value":cty.String}))}))})),
β”‚ "count":cty.List(cty.Object(map[string]cty.Type{"custom_request_handling":cty.List(cty.Object(map[string]cty.Type{"insert_header":cty.Set(cty.Object(map[string]cty.Type{"name":cty.String,```

I’m getting this exact error when I change the description instead of the tags so this doesn’t seem to be limited to updating tags.

I also was removing tags, I had moved some to the default tags for the provider

provider "aws" {
  region = "us-east-1"
  default_tags {
    tags = {
      stack   = terraform.workspace
      purpose = "example"    
    }
  }
}

I found that if I do only that, it fails with the error you mentioned. But if I also make some other arbitrary change to the web acl rules, such as changing a rule priority slightly, then it does not fail.

Also impacted by this with v4.29.0, in this case β€œonly” changing the default action from β€œblock” to β€œallow”:

Terraform will perform the following actions:

  # module.cloudfront_wafv2.aws_wafv2_web_acl.main will be updated in-place
  ~ resource "aws_wafv2_web_acl" "main" {
        id          = "3a01400a-0c69-443a-a1b2-629a3156587c"
        name        = "frontend-production-cloudfront-app-waf"
        tags        = {}
        # (6 unchanged attributes hidden)

      ~ default_action {
          + allow {
            }

          - block {
            }
        }


        # (10 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Hi all πŸ‘‹ As was mentioned above, this issue appears to be fixed when using a minimum Terraform version of 1.4.2 and a minimum AWS Provider version of 4.67.0 (preferably Terraform 1.5.3 or later and AWS Provider 5.8.0 or later). If you experience additional unexpected behaviors with versions that meet these parameters, please open a new issue so that we can investigate further.

@88lexd Thank you for feedback. See my response here.

Based on a combination of changes in Terraform core and the AWS provider, we believe these issues have been resolved. But, the fix requires that you upgrade to Terraform v1.4+ and the AWS provider v5.7.0+.

I upgraded to 1.4 last week, but my errors persist

On Tue, 28 Mar 2023, 16:10 Antonio Pagliara, @.***> wrote:

Upgrading terraform to 1.4.x fixed this for me

β€” Reply to this email directly, view it on GitHub https://github.com/hashicorp/terraform-provider-aws/issues/23992#issuecomment-1486968380, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB626UTPYXDWTUAXWVLF6ELW6LWL5ANCNFSM5SIAOXHQ . You are receiving this because you commented.Message ID: @.***>

As a workaround, I’ve just changed priority of rules, so let terraform also modify rules with tags. I hope this problem will be fixed soon.