terraform-provider-aws: Datasource aws_opensearch_domain fails when advanced_security_options is enabled

Terraform CLI and Terraform AWS Provider Version

Terraform v1.2.8
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.29.0

Affected Resource(s)

Datasource aws_opensearch_domain

Terraform Configuration Files

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "4.29.0"
    }
  }
}

provider "aws" {
  region = "eu-west-3"
}

resource "aws_opensearch_domain" "bug" {
  domain_name = "bug"

  cluster_config {
    instance_type            = "t3.small.search"
    instance_count           = 1
    dedicated_master_enabled = false
  }
  ebs_options {
    ebs_enabled = true
    volume_size = 10
    volume_type = "gp2"
  }
  node_to_node_encryption {
    enabled = true
  }
  encrypt_at_rest {
    enabled = true
  }
  domain_endpoint_options {
    enforce_https       = true
    tls_security_policy = "Policy-Min-TLS-1-2-2019-07"
  }
  advanced_security_options {
    enabled                        = true
    internal_user_database_enabled = true
    master_user_options {
      master_user_name     = "elastic"
      master_user_password = "Eel@st1c"
    }
  }
}

data "aws_opensearch_domain" "my_domain" {
  domain_name = aws_opensearch_domain.bug.domain_name
}

Debug Output

...
aws_opensearch_domain.bug: Creation complete after 16m16s [id=arn:aws:es:eu-west-3:560417158718:domain/bug]
data.aws_opensearch_domain.my_domain: Reading...
╷
│ Error: error setting advanced_security_options: Invalid address to set: []string{"advanced_security_options", "0", "anonymous_auth_enabled"}
│ 
│   with data.aws_opensearch_domain.my_domain,
│   on main.tf line 47, in data "aws_opensearch_domain" "my_domain":
│   47: data "aws_opensearch_domain" "my_domain" {
│ 
╵

See full log in attachment

References

This configuration is perfectly working with provider version 4.28.0

Maybe related to this PR?

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 27
Comments: 19 (1 by maintainers)

Most upvoted comments

Updates? I can confirm that 4.28 works fine. But 4.48 fails with the following error:

│ Error: error setting advanced_security_options: Invalid address to set: []string{"advanced_security_options", "0", "anonymous_auth_enabled"}

Other than downgrading, I’m not sure how else to proceed.

kevinashaw on Dec 28, 2022

This issue is a regression because it’s preventing to reference an opensearch domain that has been created by terraform in another configuration. That is our case, and it prevent us to update, we are stuck to 4.28.0 since 2 months.

As this bug is a regression and the PR available (#26746 which is quite small), would be possible to prioritize it? It’s really really annoying not be able to update, we are missing functionnalities that are in newer realeases

xfournet on Nov 3, 2022

Any news regarding this fix? 😢

fmartinou on Oct 24, 2022

We encountered the same problem and are stuck with an old provider version - a merge and release of the PR would really be appreciated!

neovatar on Jan 19, 2023

any news on this? creates an issue in our setup too.

tgeo-cambrian on Nov 7, 2022

Hi @fmartinou, you are right. The PR #26503 is related to this issue. The function flattenAdvancedSecurityOptions now tries to set anonymous_auth_enabled to a boolean value but the field is missing in the data resource model. I’m going to create a PR to fix this.

dpattmann on Sep 9, 2022

@YakDriver, it seems this has fallen on deaf ears. This is affecting more and more each day, now including my company.

spliskamatyshak-w3 on Feb 16, 2023

@justinretzolk help would be appreciated, thanks.

barthel on Mar 8, 2023

Best thing you can do is give the PR #26746 a like, and hopefully Terraform finds it worthwhile to fix this broken aspect of their tool.

Also impacting my organization

kai-ten on Mar 7, 2023

We are about to hit the wall in few days… We are stuck to v4.28.0 due to this regression. Since we are also deploying WAF rules with Terraform we will be blocked at the end of the month since at that time AWS will require WAF rules to specify Oversize Handling which is only possible starting from v4.52.0 (see #25545)

It’s very questioning to me why Terraform, unlike the vast majority of open source project, don’t prioritize regressions over bugs and features. Prioritization just using ‘like’ vote is quite worrying, do i need to write a bot to get attention on this 6 month old regression ? Community help didn’t work too, since the proposed PR has not been merged since months, multiple comments about the impact of this regression didn’t resonate too.

@YakDriver help would be appreciated, thanks.

xfournet on Mar 6, 2023

Downgrading isn’t an option for my situation. What I needed to look up, I’ve just stored programmatically in parameter store. It would be nice if the data call just worked properly with the latest provider, though.

spliskamatyshak-w3 on Mar 1, 2023

I also encountered this issue today on 4.54.0. I got past it by downgrading to 4.28.0.

mmcmaster-seek on Mar 1, 2023

Any updates? It is a real drawback, but there is a PR ready to be merged and fix the issue. How come this issue has persisted for so long?

kaiohenricunha on Feb 28, 2023

Any updates? I had same problem here. In really this is affecting more and more each day. Now, I’m blocked too in a old version.

leonardolacerdaatlantico on Feb 28, 2023

Any updates?

barthel on Feb 14, 2023

Any update for this?

Wyifei on Nov 30, 2022

We just upgraded and got impacted. Thanks so much for working on fixing this!

axi43 on Nov 19, 2022