azure-sdk-for-net: [QUERY] Authentication exceptions attempting to read secrets with new 4.1.0 package

Query/Question Our issue is very similar to a (closed) issue from last spring. When attempting to read a secret using DefaultAzureCredentials in Visual Studio we get one exception, but using VisualStudioCredentials we get a different exception. In both cases we’re blocked. We haven’t tried to run the code in our devops pipeline, which uses an Azure managed identity and therefore (we think) would not be affected by this problem.

What are we doing wrong? Case 11900 was closed, implying it was solved. That case referenced an older, deprecated version of the KeyVault package. In fact, our code worked fine with Microsoft.Azure.KeyVault 3.0.5 and failed after upgrading and refactoring to the new Azure.Security.KeyVault.Secrets 4.1.0 package.

If we can’t get this to work in Visual Studio, we’ll have to go back to the deprecated package.

Environment:

Name and version of the Library package used: Azure.Security.KeyVault.Secrets 4.1.0
Hosting platform: .NET 5.0.0
IDE and version : Visual Studio 16.8.2
Using JetBrains ReSharper 2020.3 EAP 9 unit test runner
AZURE_TENANT_ID variable set to our tenant ID in the unit test assembly config, but may not be picked up by ReSharper

User account The Visual Studio identity is both the logged-in Windows identity and the Azure service administrator identity, created as a live.com ID back in the mid-2000s. Elsewhere in the universe there is a Microsoft Office identity created in the same era with the same email address, which I mention because other issues suggest this might be a problem.

Visual Studio is configured only to use the Windows (i.e., Azure administrator) identity, filtered to the subscription containing the vault.

The user account has an access policy with all 8 Secrets permissions on the affected KeyVault, and is in the role of Owner of the KeyVault.

Details Code

The failing code in both cases looks like this:

using var listener = AzureEventSourceListener.CreateConsoleLogger();
var options = new DefaultAzureCredentialOptions()
{
	Diagnostics =
	{
		LoggedHeaderNames = { "x-ms-request-id" },
		LoggedQueryParameters = { "api-version" },
		IsLoggingContentEnabled = true
	}
};
var credential = new DefaultAzureCredential(options); //new VisualStudioCredential(options);
var secretClient = new SecretClient(new Uri("https://contoso-dev.vault.azure.net/"), credential);
var secret = secretClient.GetSecret(key);
return secret.Value?.Value;

Using DefaultAzureCredentials Throws Azure.Identity.AuthenticationFailedException:

Azure.Identity.AuthenticationFailedException : SharedTokenCacheCredential authentication failed: A configuration issue is preventing authentication - check the error message from the server for details.You can modify the configuration in the application registration portal. See https://aka.ms/msal-net-invalid-client for details.  Original exception: AADSTS70002: The client does not exist or is not enabled for consumers. If you are the application developer, configure a new application through the App Registrations in the Azure Portal at https://go.microsoft.com/fwlink/?linkid=2083908.
Trace ID: ***
Correlation ID: ***
Timestamp: 2020-11-30 18:00:53Z
  ----> Microsoft.Identity.Client.MsalServiceException : A configuration issue is preventing authentication - check the error message from the server for details.You can modify the configuration in the application registration portal. See https://aka.ms/msal-net-invalid-client for details.  Original exception: AADSTS70002: The client does not exist or is not enabled for consumers. If you are the application developer, configure a new application through the App Registrations in the Azure Portal at https://go.microsoft.com/fwlink/?linkid=2083908.

{snip}

[Informational] Azure-Identity: DefaultAzureCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: b267ca44-fc3b-4ace-b0c1-ea7f559d430a
[Informational] Azure-Identity: EnvironmentCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: b267ca44-fc3b-4ace-b0c1-ea7f559d430a
[Informational] Azure-Identity: EnvironmentCredential.GetToken was unable to retrieve an access token. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: b267ca44-fc3b-4ace-b0c1-ea7f559d430a Exception: Azure.Identity.CredentialUnavailableException (0x80131500): EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
[Informational] Azure-Identity: ManagedIdentityCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: b267ca44-fc3b-4ace-b0c1-ea7f559d430a
[Informational] Azure-Identity: Probing IMDS endpoint for availability. Endpoint: http://169.254.169.254/metadata/identity/oauth2/token
[Informational] Azure-Identity: IMDS endpoint is did not respond. Endpoint: http://169.254.169.254/metadata/identity/oauth2/token
[Informational] Azure-Identity: ManagedIdentityCredential.GetToken was unable to retrieve an access token. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: b267ca44-fc3b-4ace-b0c1-ea7f559d430a Exception: Azure.Identity.CredentialUnavailableException (0x80131500): ManagedIdentityCredential authentication unavailable. No Managed Identity endpoint found.
[Informational] Azure-Identity: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: b267ca44-fc3b-4ace-b0c1-ea7f559d430a
[Informational] Azure-Core: Request [f5da76b0-2a50-4959-9aa0-e23e6210c187] GET https://login.microsoftonline.com/common/discovery/instance?api-version=1.1&authorization_endpoint=REDACTED

The log on the Azure Key Vault looks like this:

{ 
"time": "2020-11-30T18:00:52.3980962Z", 
"category": "AuditEvent", "operationName": 
"Authentication", "resultType": "Success", 
"correlationId": "12345678-a41b-470b-ad18-123456789ab", 
"callerIpAddress": "8.8.8.8",
"identity": {}, 
"properties": {"clientInfo":"azsdk-net-Security.KeyVault.Secrets/4.1.0 (.NET 5.0.0; Microsoft Windows 10.0.19042)","httpStatusCode":401,"requestUri":"https://contoso.vault.azure.net/secrets/unitTestSecret/?api-version=7.1"}, 
"resourceId": "/SUBSCRIPTIONS/7FE1B682-83B4-4AAC-B848-9196A7D401A9/RESOURCEGROUPS/CONTOSO-RG/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSO-DEV", 
"operationVersion": "7.1", 
"resultSignature": "Unauthorized", 
"durationMs": "39"
}

Using VisualStudioCredentials

Exception is:

Azure.RequestFailedException : Service request failed.
Status: 403 (Forbidden)

Content:
{"error":{"code":"Forbidden","message":"Access denied to first party service.\r\nCaller: name=from-infra;tid=f8cdef31-a31e-4b4a-93e4-5f571e91255a;appid=872cd9fa-d31f-45e0-9eab-6e460a02d1f1;iss=https://sts.windows.net/f8cdef31-a31e-4b4a-93e4-5f571e91255a/\r\nVault: contoso-dev;location=eastus","innererror":{"code":"AccessDenied"}}}

Headers:
Cache-Control: no-cache
Pragma: no-cache
x-ms-keyvault-region: eastus
x-ms-request-id: REDACTED
x-ms-keyvault-service-version: 1.2.80.0
x-ms-keyvault-network-info: conn_type=Ipv4;addr=8.8.8.8;act_addr_fam=InterNetwork;
X-Powered-By: REDACTED
Strict-Transport-Security: REDACTED
X-Content-Type-Options: REDACTED
Date: Mon, 30 Nov 2020 20:15:13 GMT
Content-Length: 336
Content-Type: application/json; charset=utf-8
Expires: -1

{snip}

[Informational] Azure-Identity: VisualStudioCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: 23c290b4-dd25-45d0-b2f3-c47ff8b631a0
[Informational] Azure-Identity: VisualStudioCredential.GetToken succeeded. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: 23c290b4-dd25-45d0-b2f3-c47ff8b631a0 ExpiresOn: 2020-11-30T21:15:12.0000000+00:00
[Informational] Azure-Core: Request [23c290b4-dd25-45d0-b2f3-c47ff8b631a0] GET https://contoso-dev.vault.azure.net/secrets/unitTestSecret/?api-version=7.1

Azure Key Vault log contains:

{ "time": "2020-11-30T20:15:11.5468759Z", "category": "AuditEvent", "operationName": "Authentication", "resultType": "Success", "correlationId": "6c71169d-f9bb-49ae-a3e2-3a696e4020cf", "callerIpAddress": "8.8.8.8", "identity": {}, "properties": {"clientInfo":"azsdk-net-Security.KeyVault.Secrets/4.1.0 (.NET 5.0.0; Microsoft Windows 10.0.19042)","httpStatusCode":401,"requestUri":"https://contoso-dev.vault.azure.net/secrets/unitTestSecret/?api-version=7.1"}, "resourceId": "/SUBSCRIPTIONS/7FE1B682-83B4-4AAC-B848-9196A7D401A9/RESOURCEGROUPS/CONTOSO-RG/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSO-DEV", "operationVersion": "7.1", "resultSignature": "Unauthorized", "durationMs": "34"}
{ "time": "2020-11-30T20:15:13.3719179Z", "category": "AuditEvent", "operationName": "SecretGet", "resultType": "Success", "resultDescription": "Access denied to first party service.\nCaller: name=from-infra;tid=f8cdef31-a31e-4b4a-93e4-5f571e91255a;appid=872cd9fa-d31f-45e0-9eab-6e460a02d1f1;iss=https://sts.windows.net/f8cdef31-a31e-4b4a-93e4-5f571e91255a/\nVault: contoso-dev;location=eastus", "correlationId": "ce39bd97-25a1-44b6-a12c-123456789ab", "callerIpAddress": "8.8.8.8", "identity": {"claim":{"appid":"872cd9fa-d31f-45e0-9eab-6e460a02d1f1","http://schemas.microsoft.com/identity/claims/scope":"user_impersonation","ipaddr":"8.8.8.8","http://schemas.microsoft.com/claims/authnmethodsreferences":"pwd"}}, "properties": {"id":"https://contoso-dev.vault.azure.net/secrets/unitTestSecret","clientInfo":"azsdk-net-Security.KeyVault.Secrets/4.1.0 (.NET 5.0.0; Microsoft Windows 10.0.19042)","httpStatusCode":403,"requestUri":"https://contoso-dev.vault.azure.net/secrets/unitTestSecret/?api-version=7.1","isAccessPolicyMatch":false}, "resourceId": "/SUBSCRIPTIONS/7FE1B682-83B4-4AAC-B848-9196A7D401A9/RESOURCEGROUPS/CONTOSO-RG/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSO-DEV", "operationVersion": "7.1", "resultSignature": "Forbidden", "durationMs": "98"}
{ "time": "2020-11-30T20:17:42.1404802Z", "category": "AuditEvent", "operationName": "Authentication", "resultType": "Success", "correlationId": "ec4dd886-8cc8-4c73-b72c-92ada31f3155", "callerIpAddress": "8.8.8.8", "identity": {}, "properties": {"clientInfo":"azsdk-net-Security.KeyVault.Secrets/4.1.0 (.NET 5.0.0; Microsoft Windows 10.0.19042)","httpStatusCode":401,"requestUri":"https://contoso-dev.vault.azure.net/secrets/unitTestSecret/?api-version=7.1"}, "resourceId": "/SUBSCRIPTIONS/7FE1B682-83B4-4AAC-B848-9196A7D401A9/RESOURCEGROUPS/CONTOSO-RG/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSO-DEV", "operationVersion": "7.1", "resultSignature": "Unauthorized", "durationMs": "98"}

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 2
Comments: 29 (13 by maintainers)

Commits related to this issue

Use Azure App Configuration for app settings https://docs.microsoft.com/en-us/azure/azure-app-configuration/quickstart-aspnet-core-app?tabs=core5x Add an connection to Azure Key Vault: https://do... — committed to justSteve/Eventinars by justSteve 3 years ago

Most upvoted comments

I’m digging into the code now.

The DefaultAzureCredential class (at 181) calls on the DefaultAzureCredentialFactory class (at 22) to instantiate the EnvironmentCredential class using the constructor that should instantiate a ClientSecretCredential if the three environment variables are present. The unit test class can see these variables; under what circumstances would the Identity package not see them?

FWIW, this is the code in the calling method:

Trace.WriteLine($"AZURE_CLIENT_ID = {Environment.GetEnvironmentVariable("AZURE_CLIENT_ID")}");
Trace.WriteLine($"AZURE_CLIENT_SECRET = {Environment.GetEnvironmentVariable("AZURE_CLIENT_SECRET")?.Length > 0}");
Trace.WriteLine($"AZURE_TENANT_ID = {Environment.GetEnvironmentVariable("AZURE_TENANT_ID")}");

@schaabs , to your point, it didn’t appear that the environment variables were being propagated into the running code. But I had a typo in the second line–the one looking for the client secret–and it turns out, it wasn’t being propagated after all. I had saved it in the pipeline as a “secret” variable. It was so secret, the pipeline wouldn’t reveal it to the environment. Nice.

Setting the Pipeline variable to plain text fixed it. Everything finally passes!

So: I have to have a client secret in plain text in the pipeline setup in order to use the EnvironmentCredential. Since it’s only hitting a dev/test Key Vault instance I suppose I can live with this.

PunzunLtd on Jan 19, 2021

@heaths and sorry forgot to say no I don’t have a trusted endpoint on my Key Vault.

These steps are now working reliably for me:

Close Visual Studio
Delete %LOCALAPPDATA%\.IdentityService
Re-open Visual Studio. No account should be signed in. Re-login with my MSA account,
Use the VisualStudioCredential with the TenantId set: var cred = new VisualStudioCredential(new VisualStudioCredentialOptions { TenantId = "d304cb51-c9ec-4c15-bfdd-a8d6e80018ab" });

sipsorcery on Jan 11, 2021

@PunzunLtd Thanks for filing this issue. I’m sorry for the delayed response. The original issue you hit with the DefaultAzureCredential throwing AuthenticationFailed

Azure.Identity.AuthenticationFailedException : SharedTokenCacheCredential authentication failed: A configuration issue is preventing authentication - check the error message from the server for details.You can modify the configuration in the application registration portal. See https://aka.ms/msal-net-invalid-client for details.  Original exception: AADSTS70002: The client does not exist or is not enabled for consumers. If you are the application developer, configure a new application through the App Registrations in the Azure Portal at https://go.microsoft.com/fwlink/?linkid=2083908.
Trace ID: ***
Correlation ID: ***
Timestamp: 2020-11-30 18:00:53Z
  ----> Microsoft.Identity.Client.MsalServiceException : A configuration issue is preventing authentication - check the error message from the server for details.You can modify the configuration in the application registration portal. See https://aka.ms/msal-net-invalid-client for details.  Original exception: AADSTS70002: The client does not exist or is not enabled for consumers. If you are the application developer, configure a new application through the App Registrations in the Azure Portal at https://go.microsoft.com/fwlink/?linkid=2083908.

This is caused by a known issue with MSA accounts and the SharedTokenCacheCredential described in issue https://github.com/Azure/azure-sdk-for-net/issues/16306 (although the title is a bit missleading because in this case the issue was blocking the DefaultAzureCredential from using the AzureCliCredential). The issue unfortunately isn’t fixable at this time, and so we are will actually be disabling the SharedTokenCacheCredential from the DefaultAzureCredential authentication flow in the next release of the Azure.Identity library. The end behavior of the DefaultAzureCredential will still be the same to the user as we have essentially replaced the SharedTokenCacheCredential with the VisualStudioCredential. You can work around this issue either in the way you have by using the VisualStudioCredential, or you could go back to using the DefaultAzureCredential and simply disable the SharedTokenCacheCredential.

var cred = new DefaultAzureCredential(new DefaultAzureCredentialOptions { ExcludeSharedTokenCacheCredential = true });

Unfortunately this still doesn’t explain the 403 from the Key Vault service once you were able to acquire a token, @heaths should be able to help out more here.

schaabs on Jan 8, 2021

I’m having the same problem and here’s the code I use using azure.identity:

Exception has occurred: CLR/Microsoft.Extensions.Configuration.AzureAppConfiguration.KeyVaultReferenceException An unhandled exception of type ‘Microsoft.Extensions.Configuration.AzureAppConfiguration.KeyVaultReferenceException’ occurred in Microsoft.Extensions.Configuration.AzureAppConfiguration.dll Inner exceptions found, see $exception in variables window for more details. Innermost exception Azure.RequestFailedException : Service request failed. Status: 403 (Forbidden)

Content: {“error”:{“code”:“Forbidden”,“message”:“Access denied to first party service.\r\nCaller: name=from-infra;tid=f8cdef31-a31e-4b4a-93e4-5f571e91255a;appid=872cd9fa-d31f-45e0-9eab-6e460a02d1f1;iss=https://sts.windows.net/f8cdef31-a31e-4b4a-93e4-5f571e91255a/\r\nVault: secopsreporting-kv;location=westus2”,“innererror”:{“code”:“AccessDenied”}}}

public static IHostBuilder CreateHostBuilder(string[] args) => Host.CreateDefaultBuilder(args) .ConfigureWebHostDefaults(webBuilder => { webBuilder.ConfigureAppConfiguration((hostingContext, config) => { var settings = config.Build(); var appconfigurl = settings[“AppSettings:AppConfigEndpoint”]; config.AddAzureAppConfiguration(options => { //Used for Getting App ConfigurationUrl but using Managed Identities //Note: See this bug: //https://github.com/Azure/azure-sdk-for-net/issues/11509 options.Connect(new Uri(appconfigurl), new DefaultAzureCredential(new DefaultAzureCredentialOptions{ ExcludeSharedTokenCacheCredential = true })).UseFeatureFlags()
.ConfigureKeyVault(kv => { kv.SetCredential(new DefaultAzureCredential(new DefaultAzureCredentialOptions{ ExcludeSharedTokenCacheCredential = true })); }); }); })
.UseUrls(“http://0.0.0.0:8081”) .UseStartup<Startup>(); });

Forgot to add that I do have the access policy setup in Azure

dondeetan on Jan 7, 2021