runtime: Intermittent WinHttpException on Azure

I have had an issue running ASP.NET Core 2.0 app in Azure and have had a conversation with the Azure support team. They have, after some investigation, suggested I open the issue here for further help.

The scenario is as follows. I have a completely standard, out of the box, ASP.NET Core app that I published to Azure. The only modification I’ve made to this application is I’ve added a singleton HttpClient to the service container like this: services.TryAddSingleton<HttpClient>();

I’ve also modified the About.cshtml.cs file with the following additions:

private readonly HttpClient _client;

public AboutModel(HttpClient client)
{
    _client = client;
}

public string Message { get; set; }

public async Task OnGet()
{
    Message = "Your application description page.";
    var responses = new List<HttpResponseMessage>();

    responses.Add(await _client.GetAsync("https://cdn.contentful.com/spaces/qz0n5cdakyl9/entries/?access_token=secretTokenHere"));
    responses.Add(await _client.GetAsync("https://cdn.contentful.com/spaces/qz0n5cdakyl9/entries/?access_token=secretTokenHere&content_type=layout"));
    responses.Add(await _client.GetAsync("https://cdn.contentful.com/spaces/qz0n5cdakyl9/entries/?access_token=secretTokenHere&content_type=lessonCopy"));

    if(responses.Any(c => c.IsSuccessStatusCode == false))
    {
        throw new Exception("Got a !200 code");
    }
}

Basically the about page now makes 3 outgoing http requests. When I run this application under some simulated load (around 3000 req/min) every now and then one of the httprequests fail with an HttpException.

An error occurred while sending the request.The connection with the server was terminated abnormally

System.Net.Http.HttpRequestException: at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) Inner exception System.Net.Http.WinHttpException handled at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw: at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Threading.Tasks.RendezvousAwaitable`1.GetResult (System.Net.Http, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a) at System.Net.Http.WinHttpHandler+<StartRequest>d__105.MoveNext (System.Net.Http, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)

I’ve cut out part of the outer exception stack trace as I doubt it’s relevant, but let me know if you want to see it.

I’ve skimmed through the source code and from the looks of it the problem arises from somewhere around here https://github.com/dotnet/corefx/blob/master/src/System.Net.Http.WinHttpHandler/src/System/Net/Http/WinHttpHandler.cs#L791 or here: https://github.com/dotnet/corefx/blob/master/src/System.Net.Http.WinHttpHandler/src/System/Net/Http/WinHttpHandler.cs#L824

I should also add that this happens very infrequently (while I was screen sharing with Azure support I ran 6000 requests over about 5 minutes and this happened 4 times) and immediately retrying the exact same requests works fine.

Grateful if someone could try to shed some light on this.

Many thanks!

[EDIT] Add C# syntax highlighting by @karelz