terraform-provider-azurerm: "Invalid index key specified" for Cosmos DB collections after running TF plan without any code changes.

After upgrading successfully to azure rm provider 2.20.0, on 14.08.2020, TF plan/apply pipeline ran successfully showing no infra changes. Starting today 17.08, any subsequent tf plans fails with following error even without any code changes. "Invalid index key specified" for Cosmos DB collections. See below for full error details. Any hints on why this might be happening ? Thank you in advance.

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave “+1” or “me too” comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.26 provider.azurerm v2.20.0

Affected Resource(s)

azurerm_cosmosdb_mongo_collection

Terraform Configuration Files

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key: https://keybase.io/hashicorp

Debug Output

Panic Output

Expected Behavior

TF plan should ran again successfully since last run.

Actual Behavior

TF plan fails for all of the cosmosDB collections with:

Error: Error reading Cosmos Mongo Collection "test" (Account: "cosmos-db-test", Database: "test"): documentdb.MongoDBResourcesClient#GetMongoDBCollection: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="Invalid index key specified.\r\nActivityId: 76c8c64d-cb9c-4862-90aa-2004b805b5ea, Microsoft.Azure.Documents.Common/2.11.0"

Steps to Reproduce

terraform plan

Important Factoids

References

#0000

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 10
Comments: 24 (5 by maintainers)

Commits related to this issue

Fix issues with mongodb update - explicitely enable mongo API through capabilities. Without this there is an account recreation on update - add required _id index. Some context for this change in htt... — committed to cloudfoundry/csb-brokerpak-azure by pivotal-marcela-campo 3 years ago
Fix issues with mongodb update - explicitely enable mongo API through capabilities. Without this there is an account recreation on update - add required _id index. Some context for this change in htt... — committed to cloudfoundry/csb-brokerpak-azure by pivotal-marcela-campo 3 years ago
Fix issues with mongodb update (#128) * Fix issues with mongodb update - explicitely enable mongo API through capabilities. Without this there is an account recreation on update - add required _... — committed to cloudfoundry/csb-brokerpak-azure by pivotal-marcela-campo 3 years ago

Most upvoted comments

hi @avdhut1990, just sharing the TF error and the link to this issue would be enough for starters.

hkailantzis on Aug 21, 2020

ok , so we got a reply from Azure support regarding our current DB collections: Please note that for Mongo version 3.6 accounts with API version 2020-04-01 and later, "_id" is a required index. An index with a single key, equal to _id should be present: "key": {"keys": ["_id"]} Please update your workload to ensure the collection contains this required index. We’re going to try this and see how it goes.

Hi @hkailantzis - Did you manage to resolve the issue using the feedback from support? We’re just discussing how to address this in the provider, since it’s likely to represent a breaking change there to compensate for the apparent behavioural change in the API. Options include:

Don’t fix, but add to documentation that this key is now required to be included
insert the new required key on the user’s behalf if missing (possibly silently)
make the index property Required and validate that the set includes the new required key in the set.

The last I think is the better option, despite representing a breaking change in the resource as it keeps all the settings in the open, rather than masking behaviour.

WDYT?

jackofallops on Aug 25, 2020

Hey @hkailantzis,

that is was they said and it works for me now. I do not know if it is somehow account specific, sorry!

I would suggest raising a support request with MS. There should be a free plan option if you do not have a paid plan.

Cheers

bbung on Aug 20, 2020

I actually have a newer CosmosDB Account that is not failing. It is exactly configured as the ones failing, was just created three weeks later. I did not recognize this before. So i think you are right. I’ve contacted Azure Support on this and keep you posted if i get any new findigns.

bbung on Aug 18, 2020