runtime: HttpListener sends malformed WWW-Authenticate mutual auth header

Hope this is the right place to file this, not clear to me if .NET Framework issues are in scope for this repo.

In the RFC for SPNEGO in HTTP, section 5 shows the expected exchange. In the final 200 response from the server, the server sets a WWW-Authenticate header to Negoitate base64(gssapi-data). HttpListener sets this header to only base64(gssapi-data). For other Windows .NET clients, this seems to be accepted, but things that have a stricter interpretation of the spec, such as the requests-kerberos Python library, it doesn’t fly.

I’m pretty deeply confused about this because generally speaking, my understanding is that mutual auth and Negotiate in general is more or less http.sys’s responsibility. With stuff hosted in IIS, I see the header being set with the appropriate Negotiate prefix. I threw together a very simple .NET Core app using HttpSysListener, and it also sets the header correctly. I even hacked some C sample code for the HTTP Server API to enable Negotiate and send 401s as necessary, and it also sets the header correctly. The only thing that doesn’t is HttpListener. I have not had much luck digging through the source to figure out where this might be going wrong. In my actual app, I even tried re-writing the header myself in some Owin middleware, but the actual response on the wire was still wrong.

To repro…

• Get a pair of Windows machines joined to the same domain, or with a trust in between them.
• Machine A must be reachable by FQDN, that and the machine account must have the HTTP/fqdn SPN attached in AD. Unless you explicitly assign the HTTP SPN to something, it’s implicit in the HOST SPN, and you get that by default when the machine is joined to the domain, so you probably don’t need to do anything here.
• On machine A, install .NET 4.7.1. The error is definitely there in 4.7.1, have not tested earlier frameworks.
• On machine A, make sure incoming TCP/9002 (or whatever port you want to use) is open in the firewall.
• On machine A, use psexec -i -s cmd to get a shell as SYSTEM. This is a quick and dirty way to act with the computer’s credentials, avoid the need to register the prefix with netsh.
• Build the code below and run it in that SYSTEM shell on A.
• On B, as a domain user, hit the test path on machine A form something that will do Negotiate. You must use the name, not IP and have connectivity to a DC so that Kerberos works. This PowerShell one-liner will do the trick:

(Invoke-WebRequest http://machinea.example.com:9002/test -UseDefaultCredentials -UseBasicParsing).RawContent

• Observe headers that look something like this. Note the lack of Negotiate prefix in the WWW-Authenticate header.

HTTP/1.1 200 OK
Content-Length: 11
Date: Fri, 26 Jan 2018 20:53:23 GMT
Server: Microsoft-HTTPAPI/2.0
WWW-Authenticate: oYG3MIG0oAMKAQChCwYJKoZIgvcSAQICooG <more base64 snipped>

Hello world

Sample code:

using System;
using System.Net;
using System.Text;

namespace HttpListenerTest
{
    class Program
    {
        static void Main()
        {
            try
            {
                var responseString = "Hello world";
                var buffer = Encoding.UTF8.GetBytes(responseString);
                var listener = new HttpListener {AuthenticationSchemes = AuthenticationSchemes.Negotiate};
                var prefix = "http://+:9002/test/";
                listener.Prefixes.Add(prefix);
                listener.Start();
                Console.WriteLine($"Listening on {prefix}...");

                while (true)
                {
                    var context = listener.GetContext();
                    var request = context.Request;
                    Console.WriteLine($"Hit from {request.RemoteEndPoint.Address}...");
                    var response = context.Response;
                    response.ContentLength64 = buffer.Length;
                    var output = response.OutputStream;
                    output.Write(buffer, 0, buffer.Length);
                    output.Close();
                }
            }
            catch (Exception e)
            {
                Console.WriteLine(e);
            }
        }
    }
}