terraform-provider-aws: aws_instance change to user_data no longer results in a destroy and re-create
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave “+1” or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Terraform CLI and Terraform AWS Provider Version
Affected Resource(s)
- aws_instance
Terraform Configuration Files
Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.
resource "aws_instance" "tfe" {
count = 2
ami = data.aws_ami.ubuntu.id
instance_type = "m5.xlarge"
key_name = aws_key_pair.main.key_name
availability_zone = data.aws_availability_zones.available.names[count.index]
iam_instance_profile = aws_iam_instance_profile.tfe.id
network_interface {
network_interface_id = aws_network_interface.tfe[count.index].id
device_index = 0
}
root_block_device {
volume_size = 100
tags = {
Name = "${var.friendly_name_prefix}-tfe-server-ebs-root-${count.index + 1}"
}
}
user_data = (count.index == 0 && var.install_type != "tfe_manual" && var.install_type != "apache_hello_world") ? "${local.final_tfe_script}${local.tfe_script_setup_admin_user}" : local.final_tfe_script
tags = {
Name = "${var.friendly_name_prefix}-tfe-server-${count.index + 1}"
}
}
Debug Output
Panic Output
Expected Behavior
Prior to version 4.0.0 a change to the user_data argument resulted in the resource being destroyed and recreated.
Actual Behavior
The resource is no longer destroyed and recreated when there is a change to user_data.
Steps to Reproduce
terraform apply
Important Factoids
The docs appear to have been updated to include this line ‘Updates to this field will trigger a stop/start of the EC2 instance’.
However there does not appear to be anything in the release notes about this being a breaking change, so thought I should flag it in case it is unintentional. This change causes significant impact on users that expect the previous behaviour. This is particularly the case for Terraform Cloud and Enterprise users with VCS integration who are unable to use the -replace command line flag.
Given that user_data scripts are only run the first time an EC2 instance is deployed, it seemed logical that a change would result in a recreation of the resource, otherwise user_data changes will not be applied which may cause confusion.
References
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 20
- Comments: 17 (6 by maintainers)
We are planning to work on a solution for this. We plan to add a new argument called
user_data_replace_on_change. If set to true, then a change touser_dataoruser_data_base64will trigger a destroy and recreate of the resource as per the previous behaviour.The new argument will default to false meaning that users wanting to keep the previous behaviour will need to proactively set this argument.
I will update this issue once I have a better idea of a release date.
This change has has significant impact for our automated deployments using terraform within CI/CD. We relied on terraform replacing the instance when it saw a userdata change, so that the new userdata could run when the new instance was launched. Is there a workaround where we can force terraform to replace the instance if it sees a user data change?
I agree with @jaredfholgate; this is an unexpected and breaking change. It’s not just that it changes existing behavior, it’s also that it seems incorrect for the average (
cloud-init) use case where (as the OP points out) user-data is often a script that only gets run when the instance is launched and not when it reboots. So in these cases, with the new (reboot-only) behavior, changing the user data won’t even result in that script being re-executed, which seems counter-intuitive to say the least. I’m not saying they don’t exist, but I’d be willing to bet that a majority of EC2 instances (including those based on AWS’ own amazon-linux) are using cloud-init (or similar) mechanisms to handle user-data (and, more specifically, for instance-launch scripts) and not treating it as some sort of generic data blob.There are definitely valid reasons for allowing user-data changes to not recreate the instance, and it should be supported by terraform. It just doesn’t feel like that warrants a change to the default behavior given the average (maybe wrong on this) use case and long history, as people (and tools) have come to expect it. I see there is a proposal to add an
user_data_replace_on_changeattribute, but that would be “opt-out”. Significant changes to default behavior should be opt-in in my opinion.we have had to pin our cicd pipes to a specific aws provider which behaves as we expect. Wr risk being forced to update due to security concerns. Can we have a resolution to this issue please? Can terraform aws once again terminate and destroy EC2s when user data hash has changed?