terraform-provider-google: data.google_project.id breaks configs in 3.0.0-beta.1
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave “+1” or “me too” comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
- If an issue is assigned to the “modular-magician” user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to “hashibot”, a community member has claimed the issue already.
Terraform Version
Terraform v0.11.14
Affected Resource(s)
Every resource that needs to specify the project ID reference. This effects both google and google-beta provider both.
- google_…
Terraform Configuration Files
data "google_project" "project" {
project_id = "my-project-7jewd"
}
resource "google_compute_global_forwarding_rule" "ipv4-primary" {
name = "primary"
ip_address = "${data.google_compute_global_address.ipv4.address}"
ip_protocol = "TCP"
port_range = "80"
target = "${google_compute_target_tcp_proxy.default.self_link}"
project = "${data.google_project.project.id}"
}
The above is one of the effected resources. Anywhere I refer the project = "${data.google_project.project.id}" it fails.
So, instead I had to update it to project = "${data.google_project.project.project_id}" to get it to work. Also, Terraform no longer identifies old resources from the existing state after I did this change, it starts to recreate everything and even fail at some points. May be because I had to do that update I mentioned to those depending resources. This causes a chain re-action accross the entire deployment.
The main problem here is in the older 2.x.x provider versions the output of
${data.google_project.project.id} is something like 'my-project-7jewd`. But now the resulting output is something like ‘projects/my-project-7jewd’.
I think this is a clear bug that needs to be addressed asap. and make sure the output of ${data.google_project.project.id} remains consistent with the earlier versions.
Debug Output
Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "google" (3.0.0-beta.1)...
- Downloading plugin for provider "google-beta" (3.0.0-beta.1)...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
* provider.google: version = "~> 3.0"
* provider.google-beta: version = "~> 3.0"
In the above init output you can clearly see that providers are sourcing a beta release of the providers. This was released earlier today.
Panic Output
Causes this type of output…
1 error occurred:
* google_compute_target_tcp_proxy.default: 1 error occurred:
* google_compute_target_tcp_proxy.default: Error waiting to create TargetTcpProxy: Error waiting for Creating TargetTcpProxy: error while retrieving operation: googleapi: Error 400: Invalid value 'projects/my-project-7jewd'. Values must match the following regular expression: '(?:(?:[-a-z0-9]{1,63}\.)*(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?):)?(?:[0-9]{1,19}|(?:[a-z0-9](?:[-a-z0-9]{0,61}[a-z0-9])?))', invalidParameter
Expected Behavior
For new 3.x.x provider upgrades the data source google_project, attribute id should be the project ID instead of having a prefix projects/ along with the project ID.
data "google_project" "project" {
project_id = "my-project-7jewd"
}
output "gcp_project_id" {
value = "${data.google_project.project.id}"
}
Outputs:
gcp_project_id = my-project-7jewd
Actual Behavior
data "google_project" "project" {
project_id = "my-project-7jewd"
}
output "gcp_project_id" {
value = "${data.google_project.project.id}"
}
Outputs:
gcp_project_id = projects/my-project-7jewd
Steps to Reproduce
terraform apply
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 3
- Comments: 21 (7 by maintainers)
The issue is that the format of the
idfield changed from provider version 2.x -> 3.0.0. In 2.20 and below it was recommended to not use theidfield as an input to other resources because the format was not documented and it was likely to change (as it has here).We decided to move ids to be similar to the
self_linkfield on many resources, so that it can be used as an input for other resources in the future. To do this we had to update the id formats of many resources and datasources. Many resources changed from an id ofnameto for example:projects/{project-name}/zones/{zone}/instances/{name}This allows us to potentially send the id as the fully qualified path for the resource, and have standardization across most resources. But it does cause issues for people who were depending on the
idfield in configs in versions 2.x. The recommended fix is to update configs to not use theidfield until 3.0.0@dedamico That is a separate issue, looks like the
google_dns_managed_zonedatasource is currently broken. I filed an issue here: https://github.com/terraform-providers/terraform-provider-google/issues/4939and a fix will be coming soon via https://github.com/GoogleCloudPlatform/magic-modules/pull/2712
@universalvishwa it looks like you are using a module to manage the instance group manager? Can you either provide the config that you are using that causes this error or point me at the module that you are using?
I believe the issues are caused by a config using the
.idfield from some resource, which is causing issues now that id formats have been updated. There should be existing fields on the resources that contain the same information that should be used rather than theidfield@universalvishwa I’m concerned about the report of resources being recreated due to this. I’m unable to reproduce locally using a sample config with several resources including
google_compute_global_forwarding_rulethough.When upgrading providers I do see the diff:
but changing from
project.idtoproject.project_idcauses aterraform planto show no diff. Are you able to provide debug logs for this issue after changing all instances ofproject.id->project.project_id?