runtime: HttpClient.SendAsync() throws 'Error while copying content to a stream.'

When requesting a JSON off of a Gravatar API endpoint, the call to SendAsync ends unsuccessfully with the above System.Net.Http.HttpRequestException.

Repro:

    using (HttpClient httpClient = new HttpClient())
    {
        HttpRequestMessage request = new HttpRequestMessage
        {
            RequestUri = new Uri(@"https://www.gravatar.com/caaa1c76de5cc0ceed0e32f3d4377f31.json"),
            Method = HttpMethod.Get,
            Headers = { { "User-Agent", "csharp", "Connection", "close" } }
        };

        httpClient.Timeout = new TimeSpan(0, 0, 0, 5);

        using (HttpResponseMessage response = await httpClient.SendAsync(request))
        {
            string responseContent = response.Content.ReadAsStringAsync();
        }
    }

Expected result: The responseContent points to a string with the JSON content.

Environment: System.Net.Http 4.3.0, netcoreapp1.0, runtime framework version 1.0.4

Notes: The Gravatar endpoint works in a way that it returns a 302 response to redirect to a more specific endpoint for the JSON data. When inspected with Fiddler, the above code throws without even doing the second HTTP request (after the redirect). It might seem that the root cause is that HttpClient does not follow the 302 status code and the ‘Location’ response header. But no, even with the below code it throws the same kind of exception.

    using (HttpClient httpClient = new HttpClient())
    {
        HttpRequestMessage request = new HttpRequestMessage
        {
            RequestUri = new Uri(@"https://www.gravatar.com/caaa1c76de5cc0ceed0e32f3d4377f31.json"),
            Method = HttpMethod.Get,
            Headers = { { "User-Agent", "csharp" } }
        };

        httpClient.Timeout = new TimeSpan(0, 0, 0, 5);

        string responseContent;

        using (HttpResponseMessage response = await httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead))
        {
            if (response.StatusCode == System.Net.HttpStatusCode.Found && !string.IsNullOrEmpty(response.Headers.Location.PathAndQuery))
            {
                HttpRequestMessage redirectRequest = new HttpRequestMessage
                {
                    RequestUri = response.Headers.Location,
                    Method = HttpMethod.Get,
                    Headers = { { "User-Agent", "csharp" }, { "Connection", "close" } }
                };

                using (HttpResponseMessage redirectResponse = await httpClient.SendAsync(redirectRequest, HttpCompletionOption.ResponseContentRead))
                {
                    var bytes = await response.Content.ReadAsByteArrayAsync();
                    responseContent = System.Text.Encoding.UTF8.GetString(bytes);
                }
            }
        }
    }

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 34 (23 by maintainers)

Most upvoted comments

Thanks @karelz , @stephentoub and others for pointing me to the HTTPS > HTTP issue. Sorry for mistakes in my code. I’ve just botched these examples from a different working code, without actually compiling them.

Upon experimenting with request headers, I’ve managed to avoid that Gravatar’s HTTPS > HTTP issue. Without such redirect, HttpClient did its job perfectly!

JanLenoch on May 3, 2017

For people coming through Google:

I had the same error message, I solved it by adding a HttpClientHandler to my HttpClient with the property UseProxy set to false:

new HttpClient(new HttpClientHandler() {
    UseProxy = false
}))

Smile4ever on Jul 13, 2018

@firebellys please check my last reply - it calls out there is multiple problems with same symptoms. And that we need isolated repro in separate issue to make it actionable.

karelz on Apr 24, 2019

@SaraZakizadeh it would be best to file a new issue. There is a mix of unrelated problems here already.

First, please use latest 2.1 release (2.1.5 at the moment) where we have bunch of proxy fixes already. Second, please create a minimal repro (ideally console app with hardcoded values, etc.) - find out how often it fails (100% or just sometimes with some frequency?). Try to reproduce it on another machine in the same environment. Try to reproduce it on another machine in different environment.

karelz on Oct 22, 2018

@manums if you are truly hitting HTTPS->HTTP redirection, then it is not supported in .NET Core as insecure. BTW: .NET Core 2.0 is out of support as of 10/1 (4 days ago), I would recommend to upgrade to .NET Core 2.1.

karelz on Oct 6, 2018

Discussed offline with @galvesribeiro that this might not be a dupe. Please file a new issue (with readable callstack 😉) and let’s discuss it there. If your app is not doing https -> http redirect, this may be similar symptom with different root cause.

karelz on Jun 1, 2017

I believe you’re hitting a security restriction: HttpClient blocks attempts to redirect from an https site to an http site, which the link you’re using is trying to do. That’ll prevent your first example from working.

That is correct. HTTPS -> HTTP redirection in not allowed. If you need that for a specific purpose, then you should turn off automatic redirection (HttpClientHandler.AllowAutoRedirect = false) and handle the 3xx responses yourself.

davidsh on Apr 28, 2017

@JanLenoch, a few things:

There are a few mistakes in the repro. First, it’s not compilable, e.g. the Headers assignment line doesn’t work. Second, your second example has problems with what objects are being used, e.g. the response.Content.ReadAsByteArrayAsync line should be using redirectResponse rather than response.
I believe you’re hitting a security restriction: HttpClient blocks attempts to redirect from an https site to an http site, which the link you’re using is trying to do. That’ll prevent your first example from working.
It appears that WinHTTP cancels the request when it attempts to do an invalid redirect like this. The following code runs as you’re hoping it will on both desktop and .NET Core on Linux, but fails with the error you’re seeing on .NET Core on Windows:

using System;
using System.Net.Http;
using System.Threading.Tasks;

class Test
{
    public static void Main() => MainAsync().GetAwaiter().GetResult();
    private static async Task MainAsync()
    {
        using (var httpClient = new HttpClient() { Timeout = new TimeSpan(0, 0, 0, 5) })
        {
            var request = new HttpRequestMessage
            {
                RequestUri = new Uri(@"https://www.gravatar.com/caaa1c76de5cc0ceed0e32f3d4377f31.json"),
                Method = HttpMethod.Get
            };
            request.Headers.Add("User-Agent", "csharp");

            using (HttpResponseMessage response = await httpClient.SendAsync(request))
            {
                if (response.StatusCode == System.Net.HttpStatusCode.Found)
                {
                    var redirectRequest = new HttpRequestMessage
                    {
                        RequestUri = response.Headers.Location,
                        Method = HttpMethod.Get,
                    };
                    redirectRequest.Headers.Add("User-Agent", "csharp");

                    using (HttpResponseMessage redirectResponse = await httpClient.SendAsync(redirectRequest, HttpCompletionOption.ResponseContentRead))
                    {
                        Console.WriteLine(await redirectResponse.Content.ReadAsStringAsync());
                    }
                }
                else
                {
                    Console.WriteLine(await response.Content.ReadAsStringAsync());
                }
            }
        }
    }
}

cc: @davidsh

stephentoub on Apr 28, 2017

@JanLenoch thanks for your report and small repro, we apprectiate that 😉. Could you please try it on .NET Core 2.0 as well? Here’s jow to dogfood: https://github.com/dotnet/corefx/blob/master/Documentation/project-docs/dogfooding.md … it would be very helpful to confirm it is existing problem. Thanks!

karelz on Apr 28, 2017