terraform-provider-tfe: Sensitive values is missing

Hello guys

I’m facing an uncommon issue with Terraform that I think could be an edge case or a bug with tfe_output

We have several workspaces (one per env - test, staging, uat, and prod), but due to a costs issue, we have some more critical test resources under the staging workspace, like the Kubernetes cluster, key vault, container registry, etc…

But now, we have a data lake (basically a storage account with a bunch of blob containers) created under the test workspace but we need to create a secret on the test key vault with the storage account name and primary access key.

Once the data lake is created under the test workspace, but the test key vault is created under the staging workspace, we are trying to use outputs and tfe_outputs to use these values on different workspaces, so that, we added those as outputs on the test workspace

output "data_lake_storage_account_name" {
  value = module.data_lake.storage_account_name
}

output "data_lake_primary_access_key" {
  value = module.data_lake.primary_access_key
  sensitive = true
}

and on staging terraform files we set up tfe_outputs to get those values

data "tfe_outputs" "test_outputs" {
  organization = "my-org"
  workspace = "test"
}

and create the following azure key vault secrets (on staging):

resource "azurerm_key_vault_secret" "dev_data_lake_storage_account_name" {
  key_vault_id = module.akv-dev.id
  name         = "DataLakeAccountName"
  value        = data.tfe_outputs.test_outputs.values.data_lake_storage_account_name
}

resource "azurerm_key_vault_secret" "dev_data_lake_primary_access_key" {
  key_vault_id = module.akv-dev.id
  name         = "DataLakePrimaryAccessKey"
  value        = data.tfe_outputs.test_outputs.values.data_lake_primary_access_key
}

however, dev_data_lake_storage_account_name secret is created sucessfully, but dev_data_lake_primary_access_key secret is always giving this error:

╷
│ Error: Missing required argument
│ 
│   with azurerm_key_vault_secret.dev_data_lake_primary_access_key,
│   on secrets-dev.tf line 46, in resource "azurerm_key_vault_secret" "dev_data_lake_primary_access_key":
│   46:   value        = data.tfe_outputs.test_outputs.values.data_lake_primary_access_key
│ 
│ The argument "value" is required, but no definition was found.
╵

Not sure if this is because the output is marked as sensitive on the test workspace, but I can’t find any documentation related to this. Can anyone help me with this problem? Am I the first person with this issue? I can’t remove the sensitive flag from that output, because it’s a access key and Terraform raises an error.

Thanks for your time and help guys

Terraform version

Terraform Cloud
1.1.7

Terraform Configuration Files

terraform {
  required_providers {
    azuread = {
      source = "hashicorp/azuread"
      version = "=1.5.1"
    }
    azurerm = {
      source = "hashicorp/azurerm"
      version         = "=2.80.0"
    }
......
    tfe = {
      source = "hashicorp/tfe"
      version = "0.28.1"
    }

  }
  required_version = ">= 1.0.0"
}

Debug Output

https://gist.github.com/luis-serra-ki/ddab4de4217674240f66689c0a4cffb8

Expected Behavior

resource “azurerm_key_vault_secret” “dev_data_lake_storage_account_name” created with success resource “azurerm_key_vault_secret” “dev_data_lake_primary_access_key” created with success

Actual Behavior

resource “azurerm_key_vault_secret” “dev_data_lake_storage_account_name” created with success resource “azurerm_key_vault_secret” "dev_data_lake_primary_access_key"error

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 32
  • Comments: 21

Most upvoted comments

Hi, any update on this bug ? It is typically the kind of issue that will prevent the adoption of this provider…

+9
panasonixme on Jun 2, 2022

May or may not be relevant but I was having what I think is the same problem.

╷ │ Error: Unsupported attribute │ │ on main.tf line 15, in locals: │ 15: thing = data.tfe_outputs.outputter.values.anotherthing │ │ This object does not have an attribute named "anotherthing".

It turned out I needed to add the TFE_TOKEN environment variable to the workspace consuming the output as the tfe provider needs authentication to do work within terraform cloud. It is in the docs but I’d missed this because I’d been linked direct to the tfe_outputs page from wherever I’d been reading about using outputs in cloud. It was only when I was looking to use the tfe_provider to create a bootstrap workspace to build all other workspaces that I found the information.

Like I say it might not be the same issue everyone else is experiencing but it feels like it is.

The error message could have been more precise - pointing to an authentication issue would have pointed me in the right direction or maybe a change to the docs to add an information box on each page mentioning the need for auhentication.

Anyway hope this helps

Phill

+3
phill-lewis on Oct 12, 2022

Same issue here, I needed to export the sensitive value as nonsensitive using https://www.terraform.io/language/functions/nonsensitive

+3
galvarado on May 24, 2022

Internally, the “tfe_outputs” resources uses the “Show Workspace” API call with the “outputs” include.

The documentation does not specifically state if this API call should also return the sensitive values, but in reality it definitely does not, leading to this issue.

+3
qwerty1979bg on May 18, 2022

@serrovsky-pt @jmwri FYI, you may work with the terraform_remote_state data source instead of the tfe_outputs as a workaround, until the issue is solved:

data "terraform_remote_state" "infra_state" {
  backend = "remote"
  config = {
    organization = "xxx"
    workspaces = {
      name    = "yyy"
    }
   }
}
redis_password = data.terraform_remote_state.infra_state.outputs.redis_password
+3
asaf-upstream on Apr 18, 2022

Same issue here, all not sensitive values gets retrieved from the remote_state, but all sensitive values returns empty.

+3
asaf-upstream on Apr 14, 2022

Thanks for update @MXfive! Yep, workspace state sharing makes sense and I’m using it. I’m more concerned about sensitive output values. Thx for your input on that. I have also opened support ticket to put more urgency on the issue. I’m currently working around the issue using unsensitive function, but that is not sustainable and not secure.

+2
alexeyatbluescape on Jul 29, 2022

I am also experiencing the same issue. The funny thing is it was working before, I created a whole setup using output. Now out of a sudden, I am having that issue with all the outputs

+2
tiagocborg on Jul 7, 2022

Although it sounds related, the error occurs to me in TF cloud, so I can rule out that it is a token issue

It’s a similar problem, but this actually helped me find the issue! I had this working in one workspace but not another; It turns out the difference is that the workspace that did have the outputs working had Remote State Sharing in the workspace settings enabled for the target workspaces that needed it. I did the same for the workspace that wasn’t working, and it started working for that one too.

(Well, actually it was enabled for all workspaces in the org, but it defaults to targeted workspaces with an empty list)

Correct, there are two completely seperate things:

  1. A bug in this provider that was fixed in #565
  2. A breaking change made on the TFC API which now requires the source workspace to allow state sharing with the consumer workspace

This ticket is only around no 1. and so should be close now IMO.

+1
MXfive on Dec 8, 2022

@alexeyatbluescape Yes, through debugging with support it turned out that their development team made a breaking API that now requires consuming workspaces to be whitelisted as remote state consumers (or the source workspace sharing org wide to all workspaces).

They are apparently working on the docs and also a public release notes for the API. But I’ve not been following this as I’ve moved onto other tasks.

For the topic of this issue here where the provider does not support reading the sensitive outputs, I started working on a PR for this that I plan to test either this or next weekend. I’m not writing Go daily and it requires a bunch of refactoring to have a chance of being merged though. Will update here as I go.

+1
MXfive on Jul 29, 2022

I inadvertently filed the same bug under the main Terraform repo. Pasting it here for reference:

https://github.com/hashicorp/terraform/issues/31234

My workaround was to use nonsensitive() to get what I need working until this is fixed.

+1
spanktar on Jun 14, 2022
Read more comments on GitHub
← terraform-provider-null: Allow defining triggers that are sensitive information
terraform-provider-tfe: tfe_workspace import breaks when using terraform token in variable and remote execution →