runtime: User-Agent header is ignored in requests

I try to set User-Agent header in .NET Core 2.2 console app:

var request = new HttpRequestMessage(HttpMethod.Get, "https://example.com");
request.Headers.Add("User-Agent", "PostmanRuntime/7.6.0");
var cl = new HttpClient();
await cl.SendAsync(request);

but when I sniff request by Fiddler: image

The same code in net461 console app produces: image

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 35 (35 by maintainers)

Most upvoted comments

I meant that I made changes in System.Net.Http, build it, and want to use this new dll in Console app to test it. But my console still use System.Net.Http.dll from SDK but not my, even when I add System.Net.Http.dll as a reference.

That isn’t a recommend way of testing changes to CoreFx libraries. You have to do various manual hacks to copy a compiled System.Net.Http.dll binary from the CoreFx build tree and overwrite some shared install of .NET Core libraries. You can try it but it is complicated and not recommended as a best-practice for a ‘dev innerlop’ pattern. And it won’t be the way you do it when you submit a PR which should include both your produce code changes as well as test code changes.

@davidsh is correct?

Yes. It isn’t useful to run this new test in WinHttpHandler nor CurlHander nor on .NET Framework or UWP either.

The simplest way to ensure that is just skip the test on everything but SocketsHttpHandler.

if (!UseSocketsHttpHandler)
{
    return;  // Skip test since the fix is only in SocketsHttpHandler.
}

@Marusyk it’s ok…test run twice because if you’re writing test inside class HttpClientHandlerTest this class is abstract and it’s inherited by some tests classes i.e.:

  1. PlatformHandler_HttpClientHandler and this use WinHTTP platform handler(windows native) that doesn’t send Agent and you cannot do anything, take a look to protected override bool UseSocketsHttpHandler => false; to disable SocketHttpHandler and use WinHTTP.
  2. SocketHttpHandler and this use your update(this handler is fully managed) I think you should add if like https://github.com/dotnet/corefx/blob/94a2d0e52360f24d08c26191e20dfa8f015f9f68/src/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.MaxConnectionsPerServer.cs#L87 to skip(not fail) test for non SocketHttpHandler and your test should work.

@davidsh is correct?

@davidsh ok I misunderstood…I thought that .NET full uses WinHTTP and you fixed that(passing in some way header to native apis) and so fix .net core in case of WinHTTP usage.

No. .NET Framework doesn’t use WinHTTP. It uses a separate managed stack.

Taking a quick look at your changes, this line should be changed:

tunnelRequest.Headers.Add(HttpKnownHeaderNames.UserAgent, _headers.UserAgent.ToString());

You need to test if the UserAgent header is actually present (i.e. non-NULL, non-empty string) in the _headers before you try to add it to the tunnelRequest.Headers. And you should use the ‘TryAddWithoutValidation’ method and not the ‘Add’ method. It’s possible the user added a malformed header to the original request. But we don’t want to validate it here when adding to tunnelRequest.headers.

Should it be only User-Agent or we want to include other headers too?

Yes, only ‘User-Agent’ should be the additional header to be fixed in this PR. In general, we don’t send all the headers to a proxy with the CONNECT verb. But ‘User-Agent’ is becoming an important header to send to proxies since they have filtering logic and lately expect to see that header.

Don’t worry!Let me know if you need further help!

In order to make sure we don’t regress the code later, we usually add tests to our functional or unit tests. Generally if we’re going to take a change, it will need to have tests in one of those locations.

If you want to do some manual testing with your own console app though, you can check out the dogfooding instructions here. As David mentioned the process is pretty complicated though.

I’m trying to build solution System.Net.Http.sln in VS2017 and got that errors for test projects.

While Visual Studio supports .NET Core generally speaking, building CoreFx test projects is not supported right now in Visual Studio. You should use the command line only as I demonstrated above.

Can you be more specific about the problem you see? Can you build the repo without your changes? Can you run System.Net.Http.Functional.Tests without your changes?

Generally speaking, the process should look like this (Windows build example):

s:\GitHub\corefx>build

// build the whole repo first

s:\GitHub\corefx>cd src\System.Net.Http\tests\FunctionalTests
s:\GitHub\corefx\src\System.Net.Http\tests\FunctionalTests>msbuild /t:rebuildandtest

or to run both Innerloop and Outerloop tests

s:\GitHub\corefx\src\System.Net.Http\tests\FunctionalTests>msbuild /t:rebuildandtest /p:Outerloop=true

Do those commands above work for you?