terraform-provider-azurerm: Authentication not working with `auth_settings_v2` because of not-omitted empty validation checks

Can confirm the issue when trying to configure App Service authentication with auth_settings_v2. For me the issue emerged when trying to provide pre-created app registration, after first testing it by toggling/auto-generating the auth and app registration directly from Portal.

I used hours to debug the issue, but only difference I found (comparing first the manual and auto-generated app registrations, then the actual /providers/Microsoft.Web/sites/xxx/config/authsettingsV2 resources as well.) was just what described by @mickare about the empty list stuff.

Examples of the differences in settings (az webapp auth microsoft show) when autogenerated vs. when created with Terraform:

Autogenerated (auth working):

{
  "enabled": true,
  "isAutoProvisioned": true,
  "login": {
    "disableWWWAuthenticate": false
  },
  "registration": {
    "clientId": "xxxxx",
    "clientSecretSettingName": "MICROSOFT_PROVIDER_AUTHENTICATION_SECRET",
    "openIdIssuer": "https://sts.windows.net/yyyyy/v2.0"
  },
  "validation": {
    "allowedAudiences": [
      "api://xxxxx"
    ],
    "defaultAuthorizationPolicy": {
      "allowedPrincipals": {}
    },
    "jwtClaimChecks": {}
  }
}

Created with Terraform (auth not working, HTTP 500 with no other details):

{
  "enabled": true,
  "login": {
    "disableWWWAuthenticate": false,
    "loginParameters": []
  },
  "registration": {
    "clientId": "xxxxx",
    "clientSecretCertificateThumbprint": "",
    "clientSecretSettingName": "MICROSOFT_PROVIDER_AUTHENTICATION_SECRET",
    "openIdIssuer": "https://sts.windows.net/yyyyy/v2.0"
  },
  "validation": {
    "allowedAudiences": [
      "api://xxxxx"
    ],
    "defaultAuthorizationPolicy": {
      "allowedApplications": [],
      "allowedPrincipals": {
        "groups": [],
        "identities": []
      }
    },
    "jwtClaimChecks": {
      "allowedClientApplications": [],
      "allowedGroups": []
    }
  }
}

auth_settings_v2 block used in Terraform

auth_settings_v2 {
    auth_enabled           = true
    require_authentication = true
    unauthenticated_action = "Return401"
    active_directory_v2 {
      client_id                  = "xxxxx"
      tenant_auth_endpoint       = "https://sts.windows.net/yyyyy/v2.0"
      client_secret_setting_name = "MICROSOFT_PROVIDER_AUTHENTICATION_SECRET"

      allowed_audiences = [
        "api://xxxxx"
      ]
    }
    login {
      token_store_enabled = true
    }
  }

This makes the auth_settings_v2 currently unusable.

Fix is already merged in #21113 and will be in next release 😃

Hi @xiaxyi, there is a HUGE difference between an “empty list” and a “missing list”.

Showing the problem with code is much faster.

package main

import (
	"encoding/json"
	"fmt"
)

type Model struct {
	X []int  `json:"x,omitempty"`
	Y *[]int `json:"y,omitempty"`
}

func main() {
	var slice []int

	a := Model{}
	b := Model{slice, &slice}
	c := Model{[]int{}, &[]int{}}
	d := Model{[]int{1, 2}, &[]int{3, 4}}
	fmt.Printf("a: %+v\n", a)
	fmt.Printf("b: %+v\n", b)
	fmt.Printf("c: %+v\n", c)
	fmt.Printf("d: %+v\n", d)

	a_json, _ := json.Marshal(a)
	b_json, _ := json.Marshal(b)
	c_json, _ := json.Marshal(c)
	d_json, _ := json.Marshal(d)
	fmt.Printf("a->Json: %s\n", a_json)
	fmt.Printf("b->Json: %s\n", b_json)
	fmt.Printf("c->Json: %s\n", c_json)
	fmt.Printf("d->Json: %s\n", d_json)
}

Output:

a: {X:[] Y:<nil>}
b: {X:[] Y:0xc000010030}
c: {X:[] Y:0xc000010048}
d: {X:[1 2] Y:0xc000010060}
a->Json: {}
b->Json: {"y":null}
c->Json: {"y":[]}
d->Json: {"x":[1,2],"y":[3,4]}

The difference between cases a,b,c show that it does matter if an empty pointer or an empty list is provided to any model that is serialized with json.

Case C is the problematic one when terraform-provider-azurerm models are transformed to azure-sdk-for-go models. E.g.: AllowedClientApplications: &aad.JWTAllowedClientApps (here)

I tried 3.49, and the problem might be fixed but I can’t tell because it crashed.

Using the following terraform code (unchanged from before really)

# Run the script to get the environment variables of interest.
# This is a data source, so it will run at plan time.
data "external" "env" {
  program = ["sh", "-c", "jq -n 'env | {ARM_TENANT_ID,ARM_SUBSCRIPTION_ID,ARM_CLIENT_ID,ARM_CLIENT_SECRET}'"]
}

# Define a Resource Group for an Azure App
resource "azurerm_resource_group" "example_rg" {
  name     = "${var.app_name}-rg"
  location = "West Europe"
}

# Define an Azure App Service Plan for Linux
# https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/app_service_plan
resource "azurerm_service_plan" "example_service_plan" {
  name                = "${var.app_name}-serviceplan"
  location            = azurerm_resource_group.example_rg.location
  resource_group_name = azurerm_resource_group.example_rg.name
  os_type             = "Linux"
  sku_name            = "B1"
}

# Define an Azure Web App
# https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/azurerm_linux_web_app
resource "azurerm_linux_web_app" "example_wa" {
  name                = "${var.app_name}"
  resource_group_name = azurerm_resource_group.example_rg.name
  location            = azurerm_service_plan.example_service_plan.location
  service_plan_id     = azurerm_service_plan.example_service_plan.id


  # Enable a System Managed Identity for the Azure Web App
  identity {
    type = "SystemAssigned"
  }

  # iisnode
  site_config {
    application_stack {
      node_version = "16-lts"
    }
  }

  # Configure the Azure Web app with your AAD Auth Provider (see web.config)
  #auth_settings {
  #  enabled                       = true
  #  issuer                        = "https://sts.windows.net/${data.external.env.result["ARM_TENANT_ID"]}"
  #  default_provider              = "AzureActiveDirectory"
  #  token_refresh_extension_hours = 24
  #  token_store_enabled           = true
  #  unauthenticated_client_action = "RedirectToLoginPage"
  #
  #  active_directory {
  #    client_id     = data.external.env.result["ARM_CLIENT_ID"]
  #    client_secret = data.external.env.result["ARM_CLIENT_SECRET"]
  #  }
  #}
  # Configure the Azure Web app with your AAD Auth Provider (see web.config)
  auth_settings_v2 {
    auth_enabled                  = true
    require_authentication        = true
    default_provider              = "AzureActiveDirectory"
    unauthenticated_action        = "RedirectToLoginPage"
    # our default_provider:
    active_directory_v2 {
      tenant_auth_endpoint        = "https://login.microsoftonline.com/${data.external.env.result["ARM_TENANT_ID"]}/v2.0"
      client_secret_setting_name  = "MICROSOFT_PROVIDER_AUTHENTICATION_SECRET"  # should be used instead of ARM_CLIENT_SECRET
      client_id                   = data.external.env.result["ARM_CLIENT_ID"]
      # client_secret               = data.external.env.result["ARM_CLIENT_SECRET"]
      allowed_groups              = var.app_allowed_groups
    }
    # use a store for tokens (az blob storage backed)
    login {
      token_store_enabled = true
    }
  }
}

# Output the Azure Web App URL
output "webapp_url" {
  value = "https://${azurerm_linux_web_app.example_wa.default_hostname}"
}

This is the output I got:

azurerm_linux_web_app.example_wa: Creating...
azurerm_linux_web_app.example_wa: Still creating... [10s elapsed]
azurerm_linux_web_app.example_wa: Still creating... [20s elapsed]
azurerm_linux_web_app.example_wa: Still creating... [30s elapsed]
azurerm_linux_web_app.example_wa: Still creating... [40s elapsed]
╷
│ Error: Plugin did not respond
│ 
│   with azurerm_linux_web_app.example_wa,
│   on main.tf line 26, in resource "azurerm_linux_web_app" "example_wa":
│   26: resource "azurerm_linux_web_app" "example_wa" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵

Stack trace from the terraform-provider-azurerm_v3.49.0_x5 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x49a993b]

goroutine 140 [running]:
github.com/hashicorp/terraform-provider-azurerm/internal/services/appservice/helpers.expandAadAuthV2Settings({0xc00117d7a0, 0x1, 0x0?})
	github.com/hashicorp/terraform-provider-azurerm/internal/services/appservice/helpers/auth_v2_schema.go:981 +0x55b
github.com/hashicorp/terraform-provider-azurerm/internal/services/appservice/helpers.ExpandAuthV2Settings({0xc00149f6c0, 0x1, 0x75b6c00?})
	github.com/hashicorp/terraform-provider-azurerm/internal/services/appservice/helpers/auth_v2_schema.go:2059 +0x12a
github.com/hashicorp/terraform-provider-azurerm/internal/services/appservice.LinuxWebAppResource.Create.func1({0x75c4168, 0xc001d41da0}, {0xc0027f0000, {0x75c4c90, 0xc000453cf8}, 0xc000227e80, 0x0, {0x75c60b0, 0xbb25668}})
	github.com/hashicorp/terraform-provider-azurerm/internal/services/appservice/linux_web_app_resource.go:385 +0x1b0d
github.com/hashicorp/terraform-provider-azurerm/internal/sdk.(*ResourceWrapper).Resource.func2({0x75c4168, 0xc001d41da0}, 0x1a3185c431c?, {0x617f780?, 0xc0027f0000?})
	github.com/hashicorp/terraform-provider-azurerm/internal/sdk/wrapper_resource.go:52 +0x163
github.com/hashicorp/terraform-provider-azurerm/internal/sdk.diagnosticsWrapper.func1({0x75c4168, 0xc001d41da0}, 0x0?, {0x617f780, 0xc0027f0000})
	github.com/hashicorp/terraform-provider-azurerm/internal/sdk/wrapper_resource.go:185 +0xbe
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0xc000176000, {0x75c41a0, 0xc001370990}, 0xd?, {0x617f780, 0xc0027f0000})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:707 +0x12e
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0xc000176000, {0x75c41a0, 0xc001370990}, 0xc000fc2b60, 0xc000227c80, {0x617f780, 0xc0027f0000})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:837 +0xa85
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0xc0013ea2b8, {0x75c41a0?, 0xc001370870?}, 0xc00136e8c0)
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/grpc_provider.go:1021 +0xe8d
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0xc0013d0500, {0x75c41a0?, 0xc001370240?}, 0xc0019647e0)
	github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/tf5server/server.go:818 +0x574
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0x68d1600?, 0xc0013d0500}, {0x75c41a0, 0xc001370240}, 0xc001964770, 0x0)
	github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:385 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000468000, {0x75d43c0, 0xc001947380}, 0xc001b80480, 0xc001a76780, 0xbae4da0, 0x0)
	google.golang.org/grpc@v1.51.0/server.go:1340 +0xd23
google.golang.org/grpc.(*Server).handleStream(0xc000468000, {0x75d43c0, 0xc001947380}, 0xc001b80480, 0x0)
	google.golang.org/grpc@v1.51.0/server.go:1713 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	google.golang.org/grpc@v1.51.0/server.go:965 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/grpc@v1.51.0/server.go:963 +0x28a

Error: The terraform-provider-azurerm_v3.49.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

##[debug]Exit code 1 received from tool '/opt/hostedtoolcache/terraform/1.4.2/x64/terraform'

Created a PR in hashicorp/go-azure-helpers to add a new pointer method FromSliceOrOmitEmpty .

This could be used to replace all the problematic pointer.To calls when creating the azure-sdk-for-go data structure

I’ve hit the same issue and resorted to using azapi as a workaround until this bug is resolved.

resource "azurerm_linux_web_app" "wa" {
  name                = var.app_name
  resource_group_name = var.resource_group_name
  location            = var.resource_location
  service_plan_id     = var.asp_id

  identity {
    type = "SystemAssigned"
  }

  site_config {
  }

  app_settings = {
    MICROSOFT_PROVIDER_AUTHENTICATION_SECRET = "${var.auth_secret}"
  }

  sticky_settings {
    app_setting_names = [
      "MICROSOFT_PROVIDER_AUTHENTICATION_SECRET"
    ]
  }

  lifecycle {
    ignore_changes = [
      auth_settings_v2
    ]
  }
}

resource "azapi_update_resource" "wa_auth_v2_config" {
  type        = "Microsoft.Web/sites/config@2022-03-01"
  resource_id = "${azurerm_linux_web_app.wa.id}/config/authsettingsV2"

  body = jsonencode({
    "properties" = {
      "globalValidation" = ...,
      "httpSettings" = ...,
      "identityProviders" = ...,
      "login" = ...
  })
  
  depends_on = [
    azurerm_linux_web_app.wa
  ]
}

It is not quite as nice as the azurerm way, but it works for now.

For everyone who have this authentication issue:

https://app-test.azurewebsites.net/.auth/login/aad/callback 500 Internal Server Error

Removing the lists jwtClaimChecks.allowedGroups, jwtClaimChecks.allowedClientApplications, allowedPrincipals.groups, allowedPrincipals.identities, allowedApplications to tempoary fix the issue.

You can use the Azure Resource Explorer to temporary fix the issue (until the next Terraform deployment): https://resources.azure.com