terraform-provider-aws: `Cannot delete` error occurs when `aws_batch_compute_environment` used in `aws_batch_job_queue` is recreated
Hi there,
I found that Cannot delete, found existing JobQueue relationship error occurs when aws_batch_compute_environment used in aws_batch_job_queue is recreated.
Do you have any solutions in regards to this?
Terraform Version
- Terraform v0.10.7
- Terraform-provider-aws v1.1.0
Affected Resource(s)
- aws_batch_compute_environment
- aws_batch_job_queue
Terraform Configuration Files
resource "aws_batch_compute_environment" "test" {
compute_environment_name = "test_batch"
type = "MANAGED"
service_role = "arn:aws:iam::xxxxxxxx:role/xxxxxxxx"
compute_resources {
type = "EC2"
instance_role = "arn:aws:iam::xxxxxxxx:instance-profile/xxxxxxxx"
instance_type = ["c4.large"]
max_vcpus = 8
desired_vcpus = 0
min_vcpus = 0
security_group_ids = [
"sg-xxxxxxxx"
]
subnets = [
"subnet-xxxxxxxx", "subnet-xxxxxxxx"
]
}
}
resource "aws_batch_job_queue" "test" {
name = "test-batch-job-queue"
state = "ENABLED"
priority = 3
compute_environments = ["${aws_batch_compute_environment.test.arn}"]
}
Output
plan:
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
aws_batch_compute_environment.test: Refreshing state... (ID: test_batch)
aws_batch_job_queue.test: Refreshing state... (ID: arn:aws:batch:ap-northeast-1:xxxxxxxxxxxxx:job-queue/test-batch-job-queue)
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place
-/+ destroy and then create replacement
Terraform will perform the following actions:
-/+ aws_batch_compute_environment.test (new resource required)
id: "test_batch" => <computed> (forces new resource)
arn: "arn:aws:batch:ap-northeast-1:xxxxxxxxxxxxx:compute-environment/test_batch" => <computed>
compute_environment_name: "test_batch" => "test_batch"
compute_resources.#: "1" => "1"
compute_resources.0.desired_vcpus: "0" => "0"
compute_resources.0.instance_role: "arn:aws:iam::xxxxxxxxxxxxx:instance-profile/xxxxxxxxxxxxx" => "arn:aws:iam::xxxxxxxxxxxxx:instance-profile/xxxxxxxxxxxxx"
compute_resources.0.instance_type.#: "1" => "1"
compute_resources.0.instance_type.3819562017: "c4.large" => "c4.large"
compute_resources.0.max_vcpus: "8" => "8"
compute_resources.0.min_vcpus: "0" => "0"
compute_resources.0.security_group_ids.#: "1" => "2" (forces new resource)
compute_resources.0.security_group_ids.1377324769: "sg-xxxxxxxx" => "sg-xxxxxxxx"
compute_resources.0.security_group_ids.3516056991: "" => "sg-yyyyyyyy" (forces new resource)
compute_resources.0.subnets.#: "2" => "2"
compute_resources.0.subnets.796390534: "subnet-xxxxxxxxx" => "subnet-xxxxxxxxx"
compute_resources.0.subnets.877356347: "subnet-xxxxxxxxx" => "subnet-xxxxxxxxx"
compute_resources.0.type: "EC2" => "EC2"
ecc_cluster_arn: "arn:aws:ecs:ap-northeast-1:xxxxxxxxxxxxx:cluster/test_batch_Batch_e2eb0db4-1f83-3935-94af-e38f529d6480" => <computed>
ecs_cluster_arn: "arn:aws:ecs:ap-northeast-1:xxxxxxxxxxxxx:cluster/test_batch_Batch_e2eb0db4-1f83-3935-94af-e38f529d6480" => <computed>
service_role: "arn:aws:iam::xxxxxxxxxxxxx:role/xxxxxxxxx" => "arn:aws:iam::xxxxxxxxxxxxx:role/xxxxxxxxx"
state: "DISABLED" => "ENABLED"
status: "VALID" => <computed>
status_reason: "ComputeEnvironment Healthy" => <computed>
type: "MANAGED" => "MANAGED"
~ aws_batch_job_queue.test
compute_environments.#: "1" => <computed>
Plan: 1 to add, 1 to change, 1 to destroy.
apply:
aws_batch_compute_environment.test: Refreshing state... (ID: test_batch)
aws_batch_job_queue.test: Refreshing state... (ID: arn:aws:batch:ap-northeast-1:xxxx:job-queue/test-batch-job-queue)
aws_batch_compute_environment.test: Destroying... (ID: test_batch)
Error applying plan:
1 error(s) occurred:
* aws_batch_compute_environment.test (destroy): 1 error(s) occurred:
* aws_batch_compute_environment.test: : Cannot delete, found existing JobQueue relationship
status code: 400, request id: xxx
Panic Output
None
Expected Behavior
- delete
aws_batch_job_queue - delete and create
aws_batch_compute_environment - create
aws_batch_job_queue
Actual Behavior
- recreate compute environment
- update job queue
Steps to Reproduce
terraform apply- Add a security group id to
aws_batch_compute_environment.test. terraform apply- The error occurs.
Important Factoids
No
References
I could not find any issues related this.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 15
- Comments: 25 (7 by maintainers)
Commits related to this issue
- Fix #2044 — committed to shibataka000/terraform-provider-aws by shibataka000 7 years ago
I can confirm the delete is still broken in AWS 1.5.0.
@radeksimko Do you mind re-opening this issue?
Hi @radeksimko
Would it be possible to re-open this bus?
I have faced this problem using the latest version of terraform and AWS provider:
#2044 is still a valid fix but it is not ideal to perform this task everytime the compute environment is changed.
Just ran into this myself. For me, I initially added the lifecycle rule
create_before_destroyso the queue relationships would get moved before the compute environment was destroyed. That works if I change the name for the compute environment. So it seems to me that adding something likecompute_environment_name_prefix(similar to thename_prefixon a launch config) would be an easy way to resolve most of the issues here?If someone could confirm I’d be happy to take a stab at a pull request for that.
I have this problem too, I’m not quite sure why this issue is closed, since it clearly is a problem and it’s still there.
It should definitely delete the Job Queue before deleting the Compute Environment, and the recreate them, or just modify the Compute Environment directly when possible.
Having to intervene manually is a real problem, and at the same time it proves that it can work like that.
It there a PR for that issue right now?
I would argue that it is a Terraform provider issue. Making entirely reasonable changes to Terraform config can leave your Terraform resources in an inconsistent state which requires manual intervention to fix. That seems like something that the provider should be dealing with.
@shibataka000 are you still actively working on this module? I’d like to help work out how to get this issue resolved if you are. Thanks,
Andy
A fix for me in the meantime:
terraform state rmto remove the job queue from terraform’s stateterraform applySolved by tainting the job queue, took around 1,5 minutes for it to be destroyed though and almost 20min for the rest:
Although I guess it’s a AWS Batch backend/API issue, not really about terraform?
@shibataka000 wrote:
IIUC this workaround only work once, as the random resource is created only once.
The name then become fixed and further changes to the compute_resources that would require resource recreation (i-e all the parameters marked “replacement” in https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-batch-computeenvironment-computeresources.html) would return the error “Object already exists” as per #3207.
You would then need to force an update of the random name (by marking it as
taintedor by changing its value).If my understanding is correct then this issue should still be open.
However at this stage it feels to me that the prefix solution from #3207 (which was also described by @radeksimko in https://github.com/terraform-providers/terraform-provider-aws/pull/2347#issuecomment-345292890) would fix this and other related issues in a clean way (especially given that behind the scene it all ends up with LC/ASGs anyways), so maybe it should be marked as duplicate?
Like @Ludonope, I just ran into this exact same issue.
@andylockran It’s bug reported at https://github.com/terraform-providers/terraform-provider-aws/pull/1710#discussion_r142713542 . I create https://github.com/terraform-providers/terraform-provider-aws/pull/2322 to fix it.
@mia-0032 Another bug caused it. I will create PR after https://github.com/terraform-providers/terraform-provider-aws/pull/2322 merged.