terraform-provider-aws: Invalid Index Error when using element and count.index

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave “+1” or “me too” comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.6
+ provider.aws v2.15.0
+ provider.null v2.1.2
+ provider.template v2.1.1

Terraform Configuration Files

variable "worker_instance_count" {
  default = 3
}

variable "worker_ebs_storage_count" {
  default = 3
}

resource "aws_instance" "data_cluster_worker" {
  ami = "ami-0a313d6098716f372"
  instance_type = "t2.micro"
  count = var.worker_instance_count
}

resource "aws_ebs_volume" "data-ebs-volumes" {
  count             = "${var.worker_instance_count * var.worker_ebs_storage_count}"
  availability_zone = var.availability_zone
  size              = var.worker_ebs_storage_size
  type              = "gp2"
}

resource "aws_volume_attachment" "data-ebs-volumes-attach" {
  count       = var.worker_instance_count * var.worker_ebs_storage_count
  volume_id = aws_ebs_volume.data-ebs-volumes.*.id[count.index]
  device_name = element(var.device_names, count.index)
  instance_id = element(aws_instance.data_cluster_worker.*.id, count.index)
}

variable "device_names" {
    default = ["/dev/xvdf",
  "/dev/xvdg",
  "/dev/xvdh",
  "/dev/xvdi",
  "/dev/xvdj",
  "/dev/xvdk",
  "/dev/xvdl",
  "/dev/xvdm"]
  }

Debug Output

module.data-cluster.aws_instance.data_cluster_worker[1]: Refreshing state... [id=i-XXXXXXXXX]
module.data-cluster.aws_instance.data_cluster_worker[2]: Refreshing state... [id=i-XXXXXXXXX]
module.data-cluster.aws_instance.data_cluster_worker[0]: Refreshing state... [id=i-XXXXXXXXX]
module.data-cluster.aws_ebs_volume.data-ebs-volumes[2]: Refreshing state... [id=vol-XXXXXX]
module.data-cluster.aws_ebs_volume.data-ebs-volumes[8]: Refreshing state... [id=vol-XXXXXX]
module.data-cluster.aws_ebs_volume.data-ebs-volumes[3]: Refreshing state... [id=vol-XXXXXX]
module.data-cluster.aws_ebs_volume.data-ebs-volumes[1]: Refreshing state... [id=vol-XXXXXX]
module.data-cluster.aws_ebs_volume.data-ebs-volumes[6]: Refreshing state... [id=vol-XXXXXX]
module.data-cluster.aws_ebs_volume.data-ebs-volumes[0]: Refreshing state... [id=vol-XXXXXX]
module.data-cluster.aws_ebs_volume.data-ebs-volumes[5]: Refreshing state... [id=vol-XXXXXX]
module.data-cluster.aws_ebs_volume.data-ebs-volumes[7]: Refreshing state... [id=vol-XXXXXX]
module.data-cluster.aws_ebs_volume.data-ebs-volumes[4]: Refreshing state... [id=vol-XXXXXX]
Error: Invalid index

  on ../modules/data-cluster/main.tf line 127, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 127:   volume_id = aws_ebs_volume.data-ebs-volumes.*.id[count.index]
    |----------------
    | aws_ebs_volume.data-ebs-volumes is empty tuple
    | count.index is 6

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/data-cluster/main.tf line 127, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 127:   volume_id = aws_ebs_volume.data-ebs-volumes.*.id[count.index]
    |----------------
    | aws_ebs_volume.data-ebs-volumes is empty tuple
    | count.index is 1

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/data-cluster/main.tf line 127, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 127:   volume_id = aws_ebs_volume.data-ebs-volumes.*.id[count.index]
    |----------------
    | aws_ebs_volume.data-ebs-volumes is empty tuple
    | count.index is 4

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/data-cluster/main.tf line 127, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 127:   volume_id = aws_ebs_volume.data-ebs-volumes.*.id[count.index]
    |----------------
    | aws_ebs_volume.data-ebs-volumes is empty tuple
    | count.index is 5

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/data-cluster/main.tf line 127, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 127:   volume_id = aws_ebs_volume.data-ebs-volumes.*.id[count.index]
    |----------------
    | aws_ebs_volume.data-ebs-volumes is empty tuple
    | count.index is 2

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/data-cluster/main.tf line 127, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 127:   volume_id = aws_ebs_volume.data-ebs-volumes.*.id[count.index]
    |----------------
    | aws_ebs_volume.data-ebs-volumes is empty tuple
    | count.index is 7

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/data-cluster/main.tf line 127, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 127:   volume_id = aws_ebs_volume.data-ebs-volumes.*.id[count.index]
    |----------------
    | aws_ebs_volume.data-ebs-volumes is empty tuple
    | count.index is 0

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/data-cluster/main.tf line 127, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 127:   volume_id = aws_ebs_volume.data-ebs-volumes.*.id[count.index]
    |----------------
    | aws_ebs_volume.data-ebs-volumes is empty tuple
    | count.index is 8

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/data-cluster/main.tf line 127, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 127:   volume_id = aws_ebs_volume.data-ebs-volumes.*.id[count.index]
    |----------------
    | aws_ebs_volume.data-ebs-volumes is empty tuple
    | count.index is 3

The given key does not identify an element in this collection value.


Error: Error in function call

  on ../modules/data-cluster/main.tf line 129, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 129:   instance_id = element(aws_instance.data_cluster_worker.*.id,count.index)
    |----------------
    | aws_instance.data_cluster_worker is empty tuple
    | count.index is 6

Call to function "element" failed: cannot use element function with an empty
list.


Error: Error in function call

  on ../modules/data-cluster/main.tf line 129, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 129:   instance_id = element(aws_instance.data_cluster_worker.*.id,count.index)
    |----------------
    | aws_instance.data_cluster_worker is empty tuple
    | count.index is 1

Call to function "element" failed: cannot use element function with an empty
list.


Error: Error in function call

  on ../modules/data-cluster/main.tf line 129, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 129:   instance_id = element(aws_instance.data_cluster_worker.*.id,count.index)
    |----------------
    | aws_instance.data_cluster_worker is empty tuple
    | count.index is 4

Call to function "element" failed: cannot use element function with an empty
list.


Error: Error in function call

  on ../modules/data-cluster/main.tf line 129, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 129:   instance_id = element(aws_instance.data_cluster_worker.*.id,count.index)
    |----------------
    | aws_instance.data_cluster_worker is empty tuple
    | count.index is 5

Call to function "element" failed: cannot use element function with an empty
list.


Error: Error in function call

  on ../modules/data-cluster/main.tf line 129, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 129:   instance_id = element(aws_instance.data_cluster_worker.*.id,count.index)
    |----------------
    | aws_instance.data_cluster_worker is empty tuple
    | count.index is 2

Call to function "element" failed: cannot use element function with an empty
list.


Error: Error in function call

  on ../modules/data-cluster/main.tf line 129, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 129:   instance_id = element(aws_instance.data_cluster_worker.*.id,count.index)
    |----------------
    | aws_instance.data_cluster_worker is empty tuple
    | count.index is 7

Call to function "element" failed: cannot use element function with an empty
list.


Error: Error in function call

  on ../modules/data-cluster/main.tf line 129, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 129:   instance_id = element(aws_instance.data_cluster_worker.*.id,count.index)
    |----------------
    | aws_instance.data_cluster_worker is empty tuple
    | count.index is 0

Call to function "element" failed: cannot use element function with an empty
list.


Error: Error in function call

  on ../modules/data-cluster/main.tf line 129, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 129:   instance_id = element(aws_instance.data_cluster_worker.*.id,count.index)
    |----------------
    | aws_instance.data_cluster_worker is empty tuple
    | count.index is 8

Call to function "element" failed: cannot use element function with an empty
list.


Error: Error in function call

  on ../modules/data-cluster/main.tf line 129, in resource "aws_volume_attachment" "data-ebs-volumes-attach":
 129:   instance_id = element(aws_instance.data_cluster_worker.*.id,count.index)
    |----------------
    | aws_instance.data_cluster_worker is empty tuple
    | count.index is 3

Call to function "element" failed: cannot use element function with an empty
list.

Expected Behavior

It should have created 3 instances, 3 volumes per instance and attached 3 volumes to each instance.

Actual Behavior

Instead it throws an error, see the output above.

Steps to Reproduce

  1. terraform plan

Important Factoids

None.

References

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 92
  • Comments: 24 (3 by maintainers)

Commits related to this issue

Most upvoted comments

I came here because I had a similar error. First of all, I’m quite sure that this is a Terraform bug.

Here is the short-term fix I did to unbreak my Terraform deployment. I’m describing what I did to my environment, using the values from your example. I hope this will work for you, but of course, I could not test this, since I don’t have access to your deployment.

One error message tries to tell us that Terraform thinks aws_ebs_volume.data-ebs-volumes is an empty list (or tuple, or whatever collection). We know that aws_ebs_volume.data-ebs-volumes is not empty.

I think I got into this state by deleting resources out of band (without Terraform). Terraform should be able to detect and handle this drift, so I’m still confident that this is a bug.

Here is how I recovered:

For simplicity, I’m only describing the fix for aws_ebs_volume.data-ebs-volumes. You have to apply the same trick to aws_instance.data_cluster_worker simultaneously (not sequentially).

  1. Make sure to reference the resource which Terraform thinks is empty (here: aws_ebs_volume.data-ebs-volumes) in the count field. For example count = min(var.worker_instance_count * var.worker_ebs_storage_count, length(aws_ebs_volume.data-ebs-volumes)).
  2. DO NOT APPLY THIS, this may just tear down stuff! However, Terraform may now be able to recover. You should be able to run terraform plan again. Do NOT run apply, since my change most likely changes the count and has bad consequences.
  3. Now that Terraform is able to run again, refresh the state by running terraform refresh. The bug should disappear now.
  4. Revert the changes to the count field.
  5. Done.

Note: this may be the same issue as https://github.com/hashicorp/terraform/issues/21917#issuecomment-506948754

+23
diekmann on Aug 13, 2019

Can confirm this, @diekmann 's count fix got me out of a bad state.

This does seem to be a core bug with 0.12.

+7
bleggett on Aug 16, 2019

The only way to bypass that error or the workaround that I have found is to use the method try, example:

output "example" {
  value = try(aws_instance.ec2_instance[0].private_ip, "")
}

Hope it’s useful for someone else.

https://www.terraform.io/docs/configuration/functions/try.html

+3
kainlite on May 8, 2020

on 12.24 if we destroy an instance in AWS outside of terraform(via console or some other script), and rerun terraform to recreate that instance we are now seeing this behavior when running terraform plan. Previously this was NOT and issue on 12.07 or 11.07. Expected behavior a fresh redeploy of the below resources without error.

steps in AWS console that created this error for terraform: terminated instance in AWS destroyed 3 EBS volumes attached to instance destroyed ENI for instance

aws_volume_attachment.job01_u01: Refreshing state... [id=vai-1800376936]
aws_volume_attachment.job01_u80: Refreshing state... [id=vai-845612090]
aws_volume_attachment.job01_u05: Refreshing state... [id=vai-1388828358]

Error: Invalid index

  on resources.tf line 455, in resource "aws_volume_attachment" "job01_u80":
 455:   instance_id = aws_instance.job01[0].id
    |----------------
    | aws_instance.job01 is empty tuple

The given key does not identify an element in this collection value.

Additional notes: Terraform instance configuration block is using count = 1, and always has been.

For anyone else, below conditional to null wont work because instance ID is a required filed:

resource "aws_volume_attachment" "job01_u80" {
  device_name = "job01_u80"
  volume_id   = aws_ebs_volume.job01_u80.id
  instance_id = length(aws_instance.job01) > 0 ? aws_instance.job01[0].id : null
}


Error: "instance_id": required field is not set

  on resources.tf line 451, in resource "aws_volume_attachment" "job01_u80":
 451: resource "aws_volume_attachment" "job01_u80" {

TEMP RESOLUTION: setting conditional false val to an empty string will allow you complete terraform plan without error, and subsequently running terraform apply to deploy the instance.

resource "aws_volume_attachment" "job01_u80" {
  device_name = "job01_u80"
  volume_id   = aws_ebs_volume.job01_u80.id
  instance_id = length(aws_instance.job01) > 0 ? aws_instance.job01[0].id : ""
}

I would be careful here, as in this allows you to get around plans refresh phase, but if youre CI/CD your IaC with auto-approve flag on apply, imo this could cause additional errors if condition is evaluated an empty. which then sounds like deploying volumes with no instance to attach to or erroring out on apply.

+1
hotdawg789 on May 11, 2020

One more thing: I see this bug is tagged as service/ec2. I encountered the error with a GCP resource and with a resource of a custom provider. All resources were counted. → This is probably an error in Terraform core.

+1
diekmann on Aug 13, 2019
Read more comments on GitHub
← terraform-provider-aws: Invalid AWS Region
terraform-provider-aws: InvalidClientTokenId: Terraform 0.9.2 →