terraform-provider-google: Cloud Run: cannot reconcile service edited through console

Community Note

  • Please vote on this issue by adding a šŸ‘ reaction to the original issue to help the community and maintainers prioritize this request.
  • Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.
  • If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

terraform -v
Terraform v1.2.3
on darwin_arm64
+ provider registry.terraform.io/chainguard-dev/ko v0.0.4
+ provider registry.terraform.io/hashicorp/google v4.47.0
+ provider registry.terraform.io/hashicorp/google-beta v4.47.0
...

Affected Resource(s)

  • google_cloud_run_service

Terraform Configuration Files

This should affect virtually any Cloud Run service deployed through terraform.

Debug Output

N/A

Panic Output

N/A

Expected Behavior

terraform reconciles the service

Actual Behavior

After ~20 minutes it times out and prints an error with a 409 because the named revision already exists.

Steps to Reproduce

  1. Deploy a service via terraform,
  2. Edit it via the Console’s editor (not yaml),
  3. Deploy the service again via terraform.

Important Factoids

The Knative resource model used by Cloud Run supports ā€œbring your own revision nameā€ where you can use spec.template.metadata.name to name the revision that the Service will create. This is used by the Cloud Run console when edits are made.

If changes are made to the service without removing or updating this name, then things will fail to deploy.

cc @steren

References

b/272367711

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Reactions: 32
  • Comments: 39 (1 by maintainers)

Commits related to this issue

Most upvoted comments

Hey folks! Good news: As of today, the requisite features to resolve the core issue here have been rolled out everywhere.

Prerequisites:

  • Cloud Run Service v2 terraform provider
  • gcloud version >= 446.0

Specific workflow:

  • Define a service using the google_cloud_run_v2_service terraform provider (do not set a revision name), and apply.
  • Deploy an updated container using gcloud run deploy or the Cloud Console UI
  • Update the TF resource to make a change to the service definition. Apply it.

There should no longer appear any conflict with revision name.

It’s frustrating that v2 was introduced without considering this real-world scenario. We are still fighting this issue and it causes a lot of confusion / errors but at the same time we are forced to use v2 for new features… Any timeline on the gcloud fix?!

Hey folks, just wanted to update the thread on what we’re doing on the Cloud Run team to address this issue. As @steren mentioned above, this is an issue with our two major clients (gcloud CLI and the GCP Console UI) setting a ā€˜prettified’ revision name in the spec.

After evaluating, we’re going to change our clients to leave the revision name empty by default. We’re also updating the behavior of the control plane to use the ā€˜pretty’ name scheme if a revision name is not specified.

TL;DR - We’re updating our clients and control plane, that will address the root cause of this problem. We’ll keep this thread posted when there’s an update.

This problem also happens if you provision a cloud run service using terraform google_cloud_run_v2_service and then deploy revisions in CI/CD. Either Terraform will always detect and apply ā€œchangesā€ because the revision has changed or if you add the revision to lifecycle ignore_changes then you get the error Revision named ... with different configuration already exists.

Hi @justinmahood, I can’t speak for @FabianFrank but it seems like I’m experiencing the same issue as him using google_cloud_run_v2_service.

My resource definition does not include a revision field, even for the initial creation. my ignore_changes is set to ignore template[0].containers[0].image because that is the field I want to update outside of the context of terraform (via gcloud run deploy <service> --image <image>)

There are two issues I’m running into, neither of which result in the behavior I would ideally want:

with the revision field left out, and also not included in ignore_changes:

  • initial deployment works. subsequent changes on the TF side create revisions as expected
  • gcloud run deploy successfully deploys a new revision
  • now a tf plan sees that the revision is different, and will deploy a new revision just to remove the ā€œrevisionā€ field, even if no other changes were made to the service definition
~ template {
          - revision                         = "myservice-00004-bub" -> null

so, it seems like the solution to that would be to add this revision field to ignore_changes, and this does solve the problem of creating unnecessary revisions, however if I have to actually make a change to the TF definition after my gcloud run deploy (for example changing max_instance_count from 10 to 20) that is when I see this other error:

Error: Error updating Service "projects/.../services/myservice": googleapi: Error 409: Requested entity already exists

even though the plan action seems to make the correct plan.

so… the repro steps for my case would be:

  1. deploy service using google_cloud_run_v2_service. revision should not be present, and template[0].containers[0].image as well as template[0].revision should be in the lifecycle ignore_changes block
  2. deploy a new revision of the service using something like gcloud run deploy <service> --image <myimage>
  3. modify a value in the TF definition. plan -> successfully shows 1 to change, apply results in above mentioned 409

When i do a terraform state show google_cloud_run_v2_service.myservice, i can see that there is a revision in there:

 revision                         = "myservice-00004-bub"

and my guess is that a subsequent apply is trying to create another revision with that same name, which explains the 409, but what i want is for it to let google auto-generate a new revision name. is something like that possible?

Hello, any update ? 😃 Same trouble for me. Deploy CR V2 with Terraform and updating image with gcloud, and not able to apply new conf with terraform after that 😦

++

As I described above, the root cause comes from the 2018-ish design choice of having gcloud and Cloud Console name revisions by default because automatic names weren’t so nice. We should change that. Cloud Run API should just generate good names automatically and clients should not implicitly name revisions if they don’t want to.

autogenerate_revision_name = true in google_cloud_run_service was probably added to address this problem or to mimic the behavior of other clients. But it is a logic built into the Terraform resource, it basically ā€œpatchesā€ the root cause.

Others on the team could chime in, but I don’t think we want this feature added to google_cloud_run_v2_service, the idea with google_cloud_run_v2_service is that is exactly maps to the Cloud Run Admin API v2 resources. This enables the Cloud Run team to guarantee that any Cloud Run feature added to the Admin API v2 automatically appears in google_cloud_run_v2_service. Therefore, we want to avoid any hand-crafting f the behavior. I am not even sure if the infrastructure used allows it.

It’s great that autogenerate_revision_name = true exists in google_cloud_run_service, but as I said, we want to fix that at the root, in the Cloud Run API and CLI/UI clients

Yes, please see my comment

Any update on a fix for this that doesn’t involve deploying a new revision on every TF apply?

I had this same issue - I deployed a cloudrun service via Terraform (terraform cloud) and then subsequently deployed new revisions with updated image tags via this GitHub actions workflow.

I was getting the same error:

Error 409: Revision named 'service-name-00048-fts' with different configuration already exists.

I was able to circumvent this error with 2 code changes. The first - adding the random_uuid terraform generator, and providing a unique revision name to the terraform resource.

resource "random_uuid" "cloudrun_revision_id" {
  keepers = {
    first = timestamp()
  }
}

resource "google_cloud_run_v2_service" "service" {
  name         = var.cloudrun_name

  template {
    revision = "${var.cloudrun_name}-${random_uuid.cloudrun_revision_id.result}"
  }
}

Secondly, ignoring the following lifecycle changes:

  lifecycle {
    ignore_changes = [
      annotations,
      client_version,
      client,
      labels,
      template.0.annotations,
      template.0.labels,
    ]
  }

Not ideal, but a fairly simple workaround that allowed me manage the service from two angles. I hope this helps!

has this been progressed at all? the issue has been open for ~5 months now, I’ve had to upgrade to the v2 resource for other features and now completely stuck on this.

@trriplejay explained it perfectly, that is what I am experiencing!

While I agree this should be fixed, I am also wondering if the same issue occurs in the v2 resources.

I recommend switching to v2 as a workaround.