terraform-provider-datadog: [bug] datadog_gcp_integration not working properly / incorrect documentation

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

0.12.18

Affected Resource(s)

datadog_integration_gcp

Terraform Configuration Files

resource "google_service_account" "datadog_insights_monitoring_account" {
  account_id   = "datadog-insights-monitoring"
  display_name = "Datadog Monitoring"
  description  = "A Service Account that is configured into Datadog to support GCP monitoring integrations"
}

resource "google_project_iam_member" "datadog_monitoring_account_metrics_viewer" {
  project = var.project
  role    = "roles/monitoring.viewer"
  member  = "serviceAccount:${google_service_account.datadog_insights_monitoring_account.email}"
}

resource "google_project_iam_member" "datadog_daemon_set_compute_instance_getter" {
  project = var.project
  role    = "roles/compute.viewer"
  member  = "serviceAccount:${google_service_account.datadog_insights_monitoring_account.email}"
}

resource "google_project_iam_member" "datadog_monitoring_account_cloud_asset_viewer" {
  project = var.project
  role    = "roles/cloudasset.viewer"
  member  = "serviceAccount:${google_service_account.datadog_insights_monitoring_account.email}"
}

## GCP/Datadog integration
resource "google_service_account_key" "datadog_insights_monitoring" {
  service_account_id = google_service_account.datadog_insights_monitoring_account.name
  public_key_type    = "TYPE_X509_PEM_FILE"
}

resource "datadog_integration_gcp" "gcp_project_integration" {
  project_id = var.project

  private_key_id = jsondecode(base64decode(google_service_account_key.datadog_insights_monitoring.private_key))["private_key_id"]
  private_key    = jsondecode(base64decode(google_service_account_key.datadog_insights_monitoring.private_key))["private_key"]

  client_email = google_service_account.datadog_insights_monitoring_account.email
  client_id    = google_service_account.datadog_insights_monitoring_account.unique_id
}

Debug Output

Please provider a link to a GitHub Gist containing the complete debug output: https://www.terraform.io/docs/internals/debugging.html. Please do NOT paste the debug output in the issue; just paste a link to the Gist.

The apply “succeeds”.

Panic Output

If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the crash.log.

Expected Behavior

What should have happened?

Integration is working

Actual Behavior

What actually happened?

Integration is not working. Integrations UI shows error message “- Datadog could not query this project properly. You need to reconfigure this project’s credentials/permissions”

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

terraform apply

Important Factoids

n/a

References

n/a

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 15
Comments: 17 (4 by maintainers)

Most upvoted comments

I’m having the same issue. Complete the google service account, upload file (success) then errors with “- Datadog could not query this project properly. You need to reconfigure this project’s credentials/permissions”

+20

jskowalski on Jun 4, 2020

I’m also having the same problem

Stopped working last week, I updated the provider to the latest version but still doesn’t work, I do get data back from google but it seems to be working intermittently.

jtafurth on Apr 6, 2021

I am also having the same error but not only with the Datadog Terraform provider. What I tested so far without success:

GCP project with billing, compute, monitoring, cloud asset services enabled
Service account that has viewer roles for compute, monitoring and cloud asset (one service account created with Terraform and the other created manually)
Credentials created with terraform and manually
Datadog integration keeps having the same error using Terraform resource, manually creating using Terraform service account and manually creating using the manually-created service account.

The versions I used for the provider are 3.4.0 and 3.2.0 (this was using for other projects)

Is it possible that the GCP project requires another service to be enabled?

sergio-guillen-exa on Oct 21, 2021

Hi we are facing the same issue! I just followed this configuration: https://github.com/nephosolutions/terraform-google-datadog-integration/blob/master/main.tf#L17

and using the terraform resource its not working, in fact, when I go to that service account and I export the json file manually and then I import it on datatadog manually as well, it works.

Really really weird, I am using the datadog provider 3.4.0

xavidop on Oct 21, 2021

Actually I came back to it the day after and it seemed to be working. Could be GCP took a while to make permissions work… Or maybe someone in my huge organisation fixed something with our config and I wasn’t aware.

Interesting. Was wondering if that was the issue, but waiting for the permissions didn’t seem to change anything for us.

However, I was able to get it working using the following roles for the service account:

resource "google_project_iam_member" "service_account" {
  for_each = toset([
    "roles/compute.viewer",
    "roles/container.viewer",
    "roles/monitoring.viewer",
  ])

  project = var.project
  role    = each.value
  member  = "serviceAccount:${google_service_account.datadog.email}"
}

I got those from this module.

Hopefully this can help someone, but very curious why this works even though its quite different than what is in the documentation.

davidtom on Dec 24, 2020