terraform-provider-azurerm: Intermittent Error: Unable to locate Storage Account when RAGRS/RAZGRS account kind is used
Community Note
- Please vote on this issue by adding a đź‘Ť reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave “+1” or “me too” comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Terraform (and AzureRM Provider) Version
2.29.0
Affected Resource(s)
azurerm_storage_account intermittently produces an error on plan for the already created resource Error: Unable to locate Storage Account
Terraform Configuration Files
resource "azurerm_resource_group" "example" {
name = "example-resources"
location = "West Europe"
}
resource "azurerm_storage_account" "example" {
name = "storageaccountname"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "RAZGRS"
}
Expected Behaviour
Should be no error for the RA and non RA accounts
Actual Behaviour
intermittently produces an error on plan for the already created resource Error: Unable to locate Storage Account
Steps to Reproduce
terraform applyterraform plan
Important Factoids
Only affecting RA storage accounts
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 22
- Comments: 37 (12 by maintainers)
When using GH Actions in combination with AzureRM and account_replication_type LRS you have the same problem. When I run the same terraform apply locally I don’t have any issues. Seems related to GH actions.
azrumrm version 3.12.0
@roehrijn / @andrey-dubnik - we’ve experienced this issue over the past month or so: We found that a call to the
/resources?%24filter=resourceType+eq+%27Microsoft.KeyVault%endpoint provided different results depending on the region that serviced that ARM API call. Essentially if theX-Ms-Routing-Request-Idvalue was the same as the location of the resource, the call returned the key vault - but if not it sporadically would return an empty array. This had a downstream effect on the rest of the TF deployment as it treated the resource as missing. It would be interesting for you to check theX-Ms-Routing-Request-Idof your calls to see if there’s a correlation between successful and failed ones.We have an open case with the ARM API team, and so far they’ve confirmed that it’s an issue with the ARM API cross-region Cache not being updated quick enough.
In terms of fixes / workarounds - they’re currently a bit limited:
cc: @stuartleeks
This is correct, terraform was able to obtain the keys and all the data for the account but the list api returned blanc hence the account can’t be found error.
There was another account in the sub which was having an issue originally so in total there are 2 accounts in there. After tagging at least one the second account also appeared in the api call.
Using azurerm 3.16.0, storage account type LRS, this happens fairly frequently.
This issue is also causing sleepless nights on our side here. What I can say so far: most likely this issue is caused by some race condition hitting sorts of ARM API limits. We’re experiencing this on a Terraform project with around 100 resources (1 storage account, 1 key vault, a lot of role assignments to them) and I’m not able to reproduce the issue in a smaller project. However, calling
terraform apply -parallelism=1seems to help.Furthermore: We’re experiencing the very same behaviour also on Key Vaults (is switches randomly from Storage Account to Key Vault, whatever Terraform tries to access first). Here is a log example from that:
Seem the workaround is to create a TAG on the storage account (via portal), after that the issue goes away and above API returns the value. More to it it started to return value for all the storage account in that sub which were having the issue…
suspect it is related to the call which comes back empty - https://docs.microsoft.com/en-us/rest/api/storagerp/storage-accounts/list