terraform-provider-aws: aws_api_gateway_rest_api private API does not set VPC Endpoint association or policy until SECOND apply.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave “+1” or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.14 AWS provider 2.67.0

Affected Resource(s)

• aws_api_gateway_rest_api

Terraform Configuration Files

resource "aws_api_gateway_rest_api" "service_api_private_vpce" {
  count = "${local.private_vpce ? 1 : 0}"
  name = "${var.api_name}"

  body = "${var.api_openapi_yaml == "" ? data.template_file.openapi_yaml.rendered : var.api_openapi_yaml}"

  policy = "${data.template_file.policy.rendered}"

  endpoint_configuration {
  #
  # This will always be PRIVATE here
  #
    types = ["${local.endpoint_type}"]
    vpc_endpoint_ids = ["${module.environment_vpc_endpoint_id.ssm_parameter_value}"]
  }
  lifecycle {
      #
      # Plans have a permanent difference here!
      #
      ignore_changes = [ "binary_media_types" ]
  }
}

Expected Behavior

API should be created with endpoint type PRIVATE, associated with the passed VPCE ID and with the provided policy attached

Actual Behavior

API is created as PRIVATE but without the VPCE association or the policy. Then running terraform apply a second time, the policy and the VPCE association are set

Steps to Reproduce

• create the necessary infrastructure to have a VPCE associated with the API Gateway
• create a terraform config file with the contents as provided by the plan output below, substituting the VPCE ID in your config for the one currently in the content below
• terraform apply
• examine the API in the API Gateway console - no policy, no VPCE
• terraform apply
• examine the API in the API Gateway console - policy and VPCE exist

Important Factoids

While the above resource property settings use a lot of variables, the plan output shows that the values have been correctly resolved

  + module.main.module.console_service_api.module.api.aws_api_gateway_rest_api.service_api_private_vpce
      id:                                                    <computed>
      api_key_source:                                        "HEADER"
      arn:                                                   <computed>
      body:                                                  "---\nswagger: \"2.0\"\ninfo:\n  version: \"2019-02-19T16:23:40Z\"\n  title: \"Assure-Platform-Console-Service-private-1\"\nschemes:\n- \"https\"\npaths:\n  /:\n    get:\n      produces:\n      - \"application/json\"\n      responses:\n        200:\n          description: \"200 response\"\n          schema:\n            $ref: \"#/definitions/Empty\"\n      x-amazon-apigateway-integration:\n        uri: \"arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:665158502186:function:${stageVariables.integrationLambda}/invocations\"\n        credentials: \"arn:aws:iam::665158502186:role/${stageVariables.integrationLambdaExecRole}\"\n        responses:\n          default:\n            statusCode: \"200\"\n        passthroughBehavior: \"when_no_match\"\n        httpMethod: \"POST\"\n        contentHandling: \"CONVERT_TO_TEXT\"\n        type: \"aws_proxy\"\n    options:\n      consumes:\n      - \"application/json\"\n      produces:\n      - \"application/json\"\n      responses:\n        200:\n          description: \"200 response\"\n          schema:\n            $ref: \"#/definitions/Empty\"\n          headers:\n            Access-Control-Allow-Origin:\n              type: \"string\"\n            Access-Control-Allow-Methods:\n              type: \"string\"\n            Access-Control-Allow-Headers:\n              type: \"string\"\n      x-amazon-apigateway-integration:\n        responses:\n          default:\n            statusCode: \"200\"\n            responseParameters:\n              method.response.header.Access-Control-Allow-Methods: \"'POST,OPTIONS'\"\n              method.response.header.Access-Control-Allow-Headers: \"'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'\"\n              method.response.header.Access-Control-Allow-Origin: \"'*'\"\n        requestTemplates:\n          application/json: \"{\\\"statusCode\\\": 200}\"\n        passthroughBehavior: \"when_no_match\"\n        type: \"mock\"\ndefinitions:\n  Empty:\n    type: \"object\"\n    title: \"Empty Schema\"\n\n"
      created_date:                                          <computed>
      endpoint_configuration.#:                              "1"
      endpoint_configuration.0.types.#:                      "1"
      endpoint_configuration.0.types.0:                      "PRIVATE"
      endpoint_configuration.0.vpc_endpoint_ids.#:           "1"
      endpoint_configuration.0.vpc_endpoint_ids.1596079509:  "vpce-0efe2662207bd63a0"
      execution_arn:                                         <computed>
      minimum_compression_size:                              "-1"
      name:                                                  "Assure-Platform-Console-Service-private-1"
      policy:                                                "{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Deny\",\n            \"Principal\": \"*\",\n            \"Action\": \"execute-api:Invoke\",\n            \"Resource\": \"execute-api:/*\",\n            \"Condition\": {\n                \"StringNotEquals\": {\n                    \"aws:sourceVpce\": \"vpce-0efe2662207bd63a0\"\n                }\n            }\n        },\n        {\n            \"Effect\": \"Allow\",\n            \"Principal\": \"*\",\n            \"Action\": \"execute-api:Invoke\",\n            \"Resource\": \"execute-api:/*\"\n        }\n    ]\n}"
      root_resource_id:                                      <computed>

That VPCE ID has been confirmed to exist and be active at the time of the apply, and, when in the API Gateway console, is available for attachment to the API

On running terraform apply the second time (immediately after the first attempt that fails to set the policy and VPCE association), I get a diff for the VPCE and the policy.

  ~ module.main.module.console_service_api.module.api.aws_api_gateway_rest_api.service_api_private_vpce
      endpoint_configuration.0.vpc_endpoint_ids.#:           "0" => "1"
      endpoint_configuration.0.vpc_endpoint_ids.1596079509:  "" => "vpce-0efe2662207bd63a0"
      policy:                                                "" => "{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Deny\",\n            \"Principal\": \"*\",\n            \"Action\": \"execute-api:Invoke\",\n            \"Resource\": \"execute-api:/*\",\n            \"Condition\": {\n                \"StringNotEquals\": {\n                    \"aws:sourceVpce\": \"vpce-0efe2662207bd63a0\"\n                }\n            }\n        },\n        {\n            \"Effect\": \"Allow\",\n            \"Principal\": \"*\",\n            \"Action\": \"execute-api:Invoke\",\n            \"Resource\": \"execute-api:/*\"\n        }\n    ]\n}"

Letting THAT run through, adds the VPCE association and the policy.