runtime: .NET Core 2.1 SocketsHttpHandler does not use Negotiate / SPNego

Overview

While testing PowerShell Core 6.1 I ran into an issue with not being able to authenticate to a Kerberized REST API running on Linux unless I disable SocketsHttpHandler. https://github.com/PowerShell/PowerShell/issues/7801

It seems like that when the server responds with both Negotiate and NTLM, the SocketsHttpHandler picks NTLM which in my case results in a 401 as the service in question is really expecting Negotiate / SPNego and is not working with NTLM.

As requested by @karelz in https://github.com/dotnet/corefx/issues/30166 I’ve reproduced it on the daily builds without PowerShell Core involved and same results, so submitting a new issue for this.

Expected result

When server sends multiple auth schemes like Negotiate and NTLM, pick the strongest one which in this case is Negotiate.

Dotnet Info

C:\dev\test\httpclient-spnego\test2>dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   2.1.403-servicing-009270
 Commit:    def6c5f48d

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.15063
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.403-servicing-009270\

Host (useful for support):
  Version: 2.1.5-servicing-26911-03
  Commit:  efdba896f7

.NET Core SDKs installed:
  2.1.201-preview-007614 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.400 [C:\Program Files\dotnet\sdk]
  2.1.402 [C:\Program Files\dotnet\sdk]
  2.1.403-servicing-009270 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.5-rtm-31008 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.5-rtm-31008 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.5-servicing-26911-03 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

Example repro

var handler = new HttpClientHandler
{
    UseDefaultCredentials = true,
    AllowAutoRedirect = true,
};
using (var client = new HttpClient(handler))
{
    var res = client.SendAsync(new HttpRequestMessage(HttpMethod.Get, uri)).GetAwaiter().GetResult();
    System.Console.WriteLine(res);
}

Result: 401

HTTP traffic from packet capture

GET / HTTP/1.1 Host: mykerberossite.lab.local

HTTP/1.1 401 Unauthorized Date: Mon, 17 Sep 2018 21:31:42 GMT Server: Apache-Coyote/1.1 WWW-Authenticate: Negotiate WWW-Authenticate: NTLM Content-Length: 0

GET / HTTP/1.1 Authorization: Negotiate **** Host: mykerberossite.lab.local

HTTP/1.1 401 Unauthorized Date: Mon, 17 Sep 2018 21:31:42 GMT Server: Apache-Coyote/1.1 WWW-Authenticate: NTLM Content-Length: 0

Workaround is to disable SocketsHttpHandler

AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);
var handler = new HttpClientHandler
{
    UseDefaultCredentials = true,
    AllowAutoRedirect = true,

};
using (var client = new HttpClient(handler))
{
    var res = client.SendAsync(new HttpRequestMessage(HttpMethod.Get, uri)).GetAwaiter().GetResult();
    System.Console.WriteLine(res);
}

result: 200

HTTP Traffic

GET / HTTP/1.1 Connection: Keep-Alive Host: mykerberosite.lab.local

HTTP/1.1 401 Unauthorized Date: Mon, 17 Sep 2018 21:30:27 GMT Server: Apache-Coyote/1.1 WWW-Authenticate: Negotiate WWW-Authenticate: NTLM Content-Length: 0 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive

GET / HTTP/1.1 Connection: Keep-Alive Host: mykerberossite.lab.local Authorization: Negotiate ***

HTTP/1.1 200 OK Date: Mon, 17 Sep 2018 21:30:27 GMT Server: Apache-Coyote/1.1 WWW-Authenticate: Negotiate *** Cache-Control: no-cache Expires: -1 Content-Type: text/plain;charset=UTF-8 Content-Length: 103 Keep-Alive: timeout=5, max=99 Connection: Keep-Alive