terraform-provider-google: Cloud Function deployment is Flaky: Error code 3, message: Failed to retrieve function source code

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v0.12.20

• provider.google v3.17.0
• provider.google-beta v3.17.0

Affected Resource(s)

• google_cloudfunctions_function

Terraform Configuration Files

//Version config
terraform {
  required_version = ">= 0.12.10"
}

provider "google" {
  version = "~> 3.17.0"
}

provider "random" {}

resource "random_id" "random_number" {
  byte_length = 2
}



data "archive_file" "archive_code" {
  type        = "zip"
  source_dir  = pathexpand("functions_code/")
  output_path = pathexpand("functions_code.zip")
}


module "project-factory" {
  source  = "terraform-google-modules/project-factory/google"
  version = "~> 7.1"

  name              = "test"
  random_project_id = "true"
  org_id            = "foo"
  billing_account   = "bar"
  activate_apis = [
    "storage-component.googleapis.com",
    "cloudfunctions.googleapis.com",
  ]
  skip_gcloud_download = true
}

resource "google_storage_bucket" "functions_bucket" {
  name               = "gcf_bucket-${random_id.random_number.dec}"
  location           = "US-CENTRAL1"
  storage_class      = "STANDARD"
  force_destroy      = "true"
  bucket_policy_only = "true"
  project            = module.project-factory.project_id
}

resource "google_storage_bucket_object" "functions_code_archive" {
  name                = "functions_code.zip"
  bucket              = google_storage_bucket.functions_bucket.name
  source              = data.archive_file.archive_code.output_path
  storage_class       = "STANDARD"
  content_disposition = "attachment"
  content_encoding    = "gzip"
  content_type        = "application/zip"
}


resource "google_cloudfunctions_function" "function" {
  name                  = "python_function"
  description           = "Pull code from GCS"
  available_memory_mb   = 256
  region                = "us-central1"
  runtime               = "python37"
  trigger_http          = true
  entry_point           = "test_function"
  timeout               = 60
  source_archive_bucket = google_storage_bucket.functions_bucket.name
  source_archive_object = google_storage_bucket_object.functions_code_archive.name
  project               = module.project-factory.project_id
}

Example Py function

def test_function(request):
    """Responds to any HTTP request.
    Args:
        request (flask.Request): HTTP request object.
    Returns:
        The response text or any set of values that can be turned into a
        Response object using
        `make_response <http://flask.pocoo.org/docs/1.0/api/#flask.Flask.make_response>`.
    """
    request_json = request.get_json()
    if request.args and 'message' in request.args:
        return request.args.get('message')
    elif request_json and 'message' in request_json:
        return request_json['message']
    else:
        return f'It works!'

Debug Output

https://gist.github.com/bharathkkb/23ac7ee3539419a0900351d5594b36a9

Panic Output

N/A

Expected Behavior

Cloud Function should be deployed consistently

Actual Behavior

Cloud Function gets deployment is flaky. Half the time it errors out with Error code 3, message: Failed to retrieve function source code

Steps to Reproduce

1. terraform apply

Important Factoids

It always works when apply is run again after failure. Some kind of race condition, maybe?

After failure, I can go to the failed cloud function in UI, copy the CF via UI, and deploy with same config and it works. This seems to indicate that I am not enforcing some explicit dependency or its a bug.

References

• #4932 - seems related to this, but different error code