terraform-provider-aws: [Bug]: aws_iam_role role_last_used has changed outside of Terraform

Terraform Core Version

1.1.4

AWS Provider Version

4.64.0

Affected Resource(s)

  • aws_iam_role

Expected Behavior

We expect role_last_used to change frequently. The value is not controlled by Terraform so I don’t think it should be reported as a change to the resource when generating the plan.

Actual Behavior

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply":

  # aws_iam_role.state_machine_terminate has changed
  ~ resource "aws_iam_role" "state_machine_terminate" {
        id                    = "state_machine_terminate-jburgess3"
        name                  = "state_machine_terminate-jburgess3"
      ~ role_last_used        = [
          ~ {
              ~ last_used_date = "2023-04-21T10:24:09Z" -> "2023-04-21T10:24:11Z"
              ~ region         = "us-east-1" -> "us-west-2"
            },
        ]
        tags                  = {}
        # (10 unchanged attributes hidden)

        # (5 unchanged blocks hidden)
    }

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

It looks like this would appear for any IAM role created by Terraform

Steps to Reproduce

Apply terraform plan which creates an AWS IAM role Use the role Might need to wait some time for AWS to update the role usage metadata Run plan again

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

role_last_used is a new feature in the 4.64 release: #30750

Would you like to implement a fix?

None

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 68
  • Comments: 34 (13 by maintainers)

Most upvoted comments

After consideration of community feedback, we’ve decided to remove the role_last_used attribute from the aws_iam_role resource, but leave it in the corresponding data source. While technically representing a breaking change, we’re tentatively planning to release this in the next minor version, v5.1.0. We believe a breaking change is acceptable in this context for the following reasons:

  1. The attribute, while functioning correctly, is causing a negative user experience, evidenced by the number of upvotes and comments on this issue in a relatively short time since its release.
  2. The attribute is relatively new (release with v4.64.0, from April 2023), read-only, and unlikely to be used as an input to any other resource.
  3. The corresponding data source provides an alternative source for this information.

Please let us know if you have any questions or concerns with this proposed change. Thanks!

When I first saw this behavior, I obviously thought of it as a feature similar to the detection of external changes in many other resources/attributes. E.g. those who don’t want to see it, should be able to ignore that part. However, as @ollytheninja mentioned above, this doesn’t work by design for read-only attributes:

│ Warning: Redundant ignore_changes element
│ 
│   on ../../../../modules/my_module/my_module.tf line 58, in resource "aws_iam_role" "user_role":
│   58: resource "aws_iam_role" "user_role" {
│ 
│ Adding an attribute name to ignore_changes tells Terraform to ignore future
│ changes to the argument in configuration after the object has been created,
│ retaining the value originally configured.
│ 
│ The attribute role_last_used is decided by the provider alone and therefore
│ there can be no configured value to compare with. Including this attribute
│ in ignore_changes has no effect. Remove the attribute from ignore_changes
│ to quiet this warning.

After stumbling upon this discussion, I’m more inclined to consider this a “bug” as well for the following reasons:

  • Cannot be silenced/ignored through configuration (!)
  • Even if it was possible to ignore it, the lifecycle block would need to be added to all existing (and future) aws_iam_role (and other) resources. “Opt-out” doesn’t look like a good way to manage this.
  • I do agree that for many other resources/attributes, drift detection (changes outside of Terraform) is a neat feature to have a quick overview whether external changes are legit or not. But this one is completely useless (if somebody needs to track last used times of roles, Terraform plan is one of the worst places to do that. A proper reporting solution would be more appropriate for this use-case.)

Hi there, I am currently working on a fix, PR should be ready to review soon

This is a super annoying bug as this will constantly update the terraform state when that role is being used which is constantly… Please give this the proper priority if you can

@jar-b - @yurrriq reported this issue with Terraform 1.4.0 running in Terraform Enterprise (showing up in TFE drift detection), so I don’t think its just an issue with < 1.2.0

Would someone be able to share their lifecycle policies used to ignore changes to the IAM role role_last_used and last_used_date values in state?

As others have mentioned, this causes a lot of false alarms and noise in our Terraform drift detection alerts.

Adding another variant: This always shows as a change when using terraform plan --refresh=false, resulting in a role_last_used = (known after apply), while a terraform plan shows no changes. Very noisy and frustrating 😄

@dpowley / @jar-b - can we revert https://github.com/hashicorp/terraform-provider-aws/pull/30750 - it doesn’t seem like it was a highly requested feature and its causing more harm than good currently. I’m not sure of the value of this attribute in Terraform either

and all computed values that change show up as drift as the feature is currently implemented

Which I somewhat understand, thats fundamentally how Terraform works - this is why I raised the question of reverting the changes made in #30750. It doesn’t appear that it was a highly requested feature addition and it doesn’t fit within the Terraform model. If we know that a computed value, that is not controlled by Terraform, is always going to change and keep changing, why is that a valuable addition to the provider?

Hi everyone - Thanks for your reports, and apologies for any frustration or confusion this addition has caused.

To restate my understanding of the issue, the roles_last_used attribute is causing pre-v1.2.0 versions of Terraform (more on this later) to display a note indicating changes were made outside of Terraform. While no changes are actually planned, the note, and its resemblance to a plan/apply output when changes actually are necessary, is causing confusion and logging noise.

Under the hood, Terraform core is handling what changes are displayed at plan time. A core change introduced in v1.2.0 (specifically hashicorp/terraform#30486) should help this case:

When showing a plan, Terraform CLI will now only show “Changes outside of Terraform” if they relate to resources and resource attributes that contributed to the changes Terraform is proposing to make.

A previous comment in this thread documents how changes to this computed attribute are correctly hidden in later versions of the Terraform CLI. A similar reproduction on a different computed attribute can be found in this issue comment, which explicitly shows the differences between plan outputs across Terraform CLI versions.

In summary, upgrading to Terraform CLI >=v1.2.0 should prevent excess noise caused by changes to this computed attribute.


Edit: For anyone experiencing plan differences (not just a note about changes outside Terraform on a plan with no changes), or still observing displayed differences on Terraform versions >v1.2.0, please open a new issue linking back to this one. Refresh-only plans may be a separate case that we need to handle regardless of Terraform CLI version, so a separate issue report with more details would be greatly appreciated!

The below does not appear to work @eduardoasantana, I’d love to know how to ignore it too!

lifecycle {
  ignore_changes = [
    role_last_used.0.last_used_date,
  ]
}

I’ve also tried just role_last_used but that also doesn’t work. Not sure if I’m missing something or if the ignore flag isn’t working properly?