terraform-provider-azuread: Error: ODataId was nil when creating an azuread_group resource
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritise this request
- Please do not leave “+1” or “me too” comments, they generate extra noise for issue followers and do not help prioritise the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Terraform (and AzureAD Provider) Version
$ terraform -v
Terraform v1.0.7
on linux_amd64
+ provider registry.terraform.io/hashicorp/azuread v2.2.1
+ provider registry.terraform.io/hashicorp/azurerm v2.77.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
Affected Resource(s)
azuread_group
Terraform Configuration Files
provider "azurerm" {
features {}
}
data "azuread_client_config" "current" {}
resource "azuread_group" "example" {
display_name = "example"
owners = [data.azuread_client_config.current.object_id]
security_enabled = true
}
Debug Output
azuread_group.example: Creating...
│ Error: Could not retrieve owner principal object "00000000-0000-0000-0000-000000000000"
│
│ with azuread_group.example,
│ on main.tf line 41, in resource "azuread_group" "example":
│ 41: resource "azuread_group" "example" {
│
│ ODataId was nil
Expected Behavior
The group should have been successfully created
Actual Behavior
An error occurred mentioning ODataId was nil
Steps to Reproduce
export ARM_CLIENT_ID="..."
export ARM_CLIENT_SECRET="..."
export ARM_SUBSCRIPTION_ID="..."
export ARM_TENANT_ID="..."
export ARM_ENVIRONMENT=usgovernment
terraform apply
Important Factoids
We are attempting to create groups in our Azure Active Directory hosted in Azure US Government.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 101
- Comments: 107 (23 by maintainers)
Commits related to this issue
- Workaround for corrupted or missing `@odata.id` for directory objects We use the `@odata.id` field (`DirectoryObject.ODataId`) to reference objects for `@odata.bind` fields, e.g. when creating applic... — committed to hashicorp/terraform-provider-azuread by manicminer 3 years ago
- Workaround for corrupted or missing `@odata.id` for directory objects We use the `@odata.id` field (`DirectoryObject.ODataId`) to reference objects for `@odata.bind` fields, e.g. when creating applic... — committed to hashicorp/terraform-provider-azuread by manicminer 3 years ago
- Workaround for corrupted or missing `@odata.id` for directory objects We use the `@odata.id` field (`DirectoryObject.ODataId`) to reference objects for `@odata.bind` fields, e.g. when creating applic... — committed to hashicorp/terraform-provider-azuread by manicminer 3 years ago
Thanks to everyone who has posted debug logs. At this time we’re fairly confident of the causes of this issue and a comprehensive workaround will be included in this week’s release. I have now removed the test builds as mentioned. We no longer need further log output to diagnose this issue, thanks!
If you have used a test build and have still seen errors like
ODataId was nilorDirectoryObject encountered with nil ODataId, this is likely because the workaround in the test build was not applied to every code path - for example the test build did not fix the azuread_group_member resource, and did not fix the updating of any resource.If you are seeing the error
Invalid URL format specified in payload, this is also due to breaking changes in the API, which the released workaround will address.One last note: if you are affected by this issue, please do not post +1 or “me too” comments as it distracts from helpful discussion - instead please upvote the issue (👍 reaction on the initial comment) as this does help us to gauge the impact radius of this issue. Many thanks!
As per the above notice, please upgrade to version 2.6.0 of the AzureAD provider which implements a fix for this issue.
At first this issue seemed to affect a small number of tenants but the radius grew over the last few days to affect a large number of tenants. Thank you to all who commented, debugged and otherwise contributed to identifying and resolving this issue, and for all the discussion points - your involvement is greatly appreciated!
For anyone who effectively downgraded their provider to a v1.x release to sidestep this API issue, a gentle reminder to remember the upgrade guide when you update your configurations to work with v2.6.0.
Edit: If Terraform is not yet picking up the new version for you, please allow several minutes as the TF Registry updates
@callppatel @anarsen Good points both of you. I do think, tho, that demanding timeframes for fixes in OSS software that people are using without paying anything at all, is a bit over-the-top.
Also impacted by this in regular non-government Azure. Cannot create
azuread_applicationresources.@manicminer I just spoke with an Azure Graph API support engineer and learned two things:
There was indeed a recent change made to the
@odata.idfield to bring the format of it up to odata v4 specifications, which I would imagine is why this just “broke” theazureadTerraform providerAzure stated that Terraform should NOT use the
@odata.id,@odata.typeor@odata.contextfields because they are consideredannotationsand not user data. They can change at any time. Azure said that there is anidfield which is in the payload of all of the API’s responses which should be used instead of@odata.id. Theseannotationfields (all fields that start with@areannotations) should only ever be used by backend Azure systems internal to Microsoft.Hey!
Looking at the debug output, I’ve noticed the following header in Graph API responses:
I then tried the following curl request, using the URL that is used by the TF provider in my case:
Notice the
odata.metadata=fullinAcceptheader. As a result, I got the response with@odata.idattribute:Perhaps the Microsoft has recently changed the definition of what is
"minimial"OData, and this why the ID is no longer returned by default?With TypeScript library, including the
"Accept"header can be achieved like this:@manicminer Perhaps a similar approach could be used with Go library for Microsoft Graph? Looking at the ticket you submitted ( microsoftgraph/msgraph-metadata#94 ), it’s been quiet for 9days and I suspect there might as well be “working as intended” -kind of response incoming form MS.
Perhaps meanwhile implement the full metadata fetching by the client, since this is quite a big blocker (can’t really use the provider at all when operating in tenant affected by MS change)? I also think that explicitly specifying full metadata is more futureproof solution in general.
Edit: I notice that there may be no official MSGraph client for Go (?). Looks like “Hamilton” library is used to query MSGraph. I think it boils down to allowing customization for this line: https://github.com/manicminer/hamilton/blob/80ee8faed5254353670568f803f5828e5467a6f4/msgraph/client.go#L143
As of today, I’m also experiencing a similar error with AzureCloud (not government),
but it’s even with
azuread_applicationcreation.I suspect this is the same issue, it looks like a response that doesn’t contain what
azureadprovider is expecting.I can create a separate issue, but I suspect this is the same root cause.
Error:
Providers:
Debug output:
I’m facing the same issue
I got the following error:
Here is the provider versions
You can’t downgrade once you upgraded and store it terraform state. Terraform doesn’t like downgrade to lower version.
“that demanding timeframes for fixes in OSS software, is a bit over-the-top” I am not sure I can agree with that, OSS software is no different than any proprietary software in terms of support and being agile IMHO. Only difference is that it is supported by community not by a company or a person. And I don’t think, people collaborating on the OSS, actually “demand” anything “timeframe or fix”, they simply “request” with their fellow OSS developers. And success of OSS movement depends on such collaboration.
That’s my 2 cents.
@anarsen Are you suggesting that using the : azuread = { source = “hashicorp/azuread” version = “1.6.0” }
should solve the problem, if I am using the resource for the first time?
Dead right.
@benjbellMS https://github.com/hashicorp/terraform-provider-azuread/issues/588#issuecomment-934699783
We’re seeing this issue too – Not on Gov Cloud, standard Azure Environment. Occurs when trying to create an
azuread_groupand also anazuread_service_principal. Ignoring theownerproperty had no impact (which is expected as noted above, the field being required and defaulting to the logged in user)If it helps, here is the debug log when performing the operation:
terraform output - xxxx is my own object ID of my user account (running apply locally)
debug – tenant and object IDs replaced with xxxx
No, Azure.
Thanks for resolving this so quickly! I am back up and running.
Thanks for the quick action on resolving this folks, much appreciated!
We are also anxiously waiting for the fix. Can you please confirm at what time tomorrow we should expect it to be available. thanks in advance.
The test builds were only for investigation purposes and not a workaround, and have hence been removed. Please check tomorrow’s release for fixes to this breaking change in the API. Thanks!
@manicminer - Do we have any workaround for this? We have huge impact due to this.
To provide update, this error has nothing to do with not requesting provider within the module, the configuration works on all other resources except azuread_group_member
we have added required provider within the module as well as within the main template.
@manicminer When using
for azuread_group_member we are getting the following error:
There are my current providers and it works
MS has responded with this concerning the comment above:
@stazz @mlcooper Many thanks for digging into this.
(cc @bher2000, @helayoty, @DmytryEmery, @Bj3MaS) I’m working on an implementation to support OData-related HTTP headers and have pushed a test build to manicminer/terraform-provider-azuread. This is another class of issue that is not affecting any of our testing tenants, so if anyone affected by this issue is able to test and give feedback it would be highly appreciated!
You can give this a spin by modifying your
terraformblock:This is not a reviewed release and is cut from a development branch. Please do not use this in a production tenant. I’ll be deleting the release artifacts in time so it will only work until then.
Thanks!
I did some research and in the odata standard there are some standard request parameters that can be used, including
odata.metadata=minimalandodata.metadata=full. So I agree with @stazz, I think that line in the Hamilton library simply always needs to useodata.metadata=full. You can see the odata standard here.@manicminer, Thanks for the update and details on this issue! While its unfortunate we cannot use this module for now its honestly not super surprising that Azure Government Cloud is causing the issue 😃. I will put in a ticket with Azure support on our side about this issue and link this thread.
Thanks again for the quick response!
@Maximo1990 - No problem. Should work out, i initially hit the same error on my initial deploy with a SP.
Confirmed fixed. Big thanks to @manicminer
azuread:index:ServicePrincipal (nameofSp): error: 1 error occurred: * Could not create service principal: json.Marshal(): json: error calling MarshalJSON for type *msgraph.Owners: marshaling Owners: encountered DirectoryObject with nil ODataIdJust reporting that this is now occurring on SP creation as well (using pulumi).
@everyone screaming for a timeframe: Do you require the features in >= 2.0.0 of the provider? If not, just reference 1.6.0 and be happy until the fix is out.
Could you add a fix in the terraform provider please ? This issue is impacting a lot of projects 😦
@manicminer I have now removed the test builds as mentioned. Does it means your
is not available anymore ?
This is showstopper for us. Can we get solution around this.
Thank you @mlcooper for asking! That is a bit worrying answer, but let’s hope they manage to sort it out. I hope that whatever their final result is, they would make it uniform across whole API.
I agree - I am reaching out to MS about this now.
@manicminer in my case
azuread_applicationaffectedCan confirm using
manicminer/azureadversion12.1.0fixed the problem@manicminer Thanks to your new version
12.1.0I was able to deployazuread_applications with owners.Currently, this could be used as a workaround instead of creating resources manually and then importing them with
terraform:hashicorp/azureadprovider tomanicminer/azureadversion12.1.0azuread_applications with owners)hashicorp/azureadprovider and useterraform init -upgradeThanks @stazz - I didn’t touch the update code yet so there’s probably a bugbear in there that needs the workaround too.
We’re using the beta endpoint for Applications as there are some properties we support that haven’t made it to the 1.0 API yet.
In terms of IDs versus OData IDs, generally we use the underlying object IDs wherever possible, however when creating an object sometimes the only way to specify relational fields is via OData fields. This is a problem for
ownersin general because, aside from other considerations for complex objects like Groups, you can create objects with no owners and be left unable to update them without the necessary permissions. Others like Groups no longer permit new objects without owners.@manicminer The hacks are the backbone of everything 😉 The
12.1.0version worked! I could create an app now, with owners and also without.There was just small hiccup: when I change
ownersvalue:I can see it is passing this
directoryObject('<guid>')in POST request, which is causing havoc. I also notice that URL hasbetaprefix - is that on purpose?I have an active case opened with the Graph API team at Microsoft. If I were to send them this trace I just posted, would they be able to also quickly and clearly see there is a bug with their API?
Here’s the minimal’s test app TF code:
And here’s some debug info:
@manicminer I’ve just tested your solution, and unfortunately it doesn’t work. Earlier I stumbled upon this issue:
Now with a custom version
12.0.0there is another problem: