runtime: WinHttp buffer supplied to the function was too small error should not be exposed
07:27:26 System.Net.Http.Functional.Tests.HttpClientHandler_ServerCertificates_Test.UseCallback_BadCertificate_ExpectedPolicyErrors(url: "https://wrong.host.badssl.com/", expectedErrors: RemoteCertificateNameMismatch) [FAIL]
07:27:26 System.Net.Http.HttpRequestException : An error occurred while sending the request.
07:27:26 ---- System.Net.Http.WinHttpException : A security error occurred
07:27:26 Stack Trace:
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
07:27:26 D:\j\workspace\windows_nt_re---37265eab\src\System.Net.Http\src\System\Net\Http\HttpClient.cs(392,0): at System.Net.Http.HttpClient.<FinishSendAsync>d__58.MoveNext()
07:27:26 --- End of stack trace from previous location where exception was thrown ---
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
07:27:26 D:\j\workspace\windows_nt_re---37265eab\src\System.Net.Http\tests\FunctionalTests\HttpClientHandlerTest.ServerCertificates.cs(174,0): at System.Net.Http.Functional.Tests.HttpClientHandler_ServerCertificates_Test.<UseCallback_BadCertificate_ExpectedPolicyErrors>d__9.MoveNext()
07:27:26 --- End of stack trace from previous location where exception was thrown ---
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
07:27:26 --- End of stack trace from previous location where exception was thrown ---
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
07:27:26 --- End of stack trace from previous location where exception was thrown ---
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
07:27:26 ----- Inner Stack Trace -----
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
07:27:26 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
07:27:26 D:\j\workspace\windows_nt_re---37265eab\src\System.Net.Http.WinHttpHandler\src\System\Net\Http\WinHttpHandler.cs(843,0): at System.Net.Http.WinHttpHandler.<StartRequest>d__102.MoveNext()
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 98 (62 by maintainers)
Commits related to this issue
- Disable bad certificates tests on Windows (#21090) Due to issues with #7812 and certificates using DHE ciphers, we're disabling these tests on Windows for now. I am in the process of moving these... — committed to dotnet/corefx by davidsh 7 years ago
- Disable System.Net.Http tests due to #7812 (#24455) Porting the same logic from master branch to disable these tests on systems that don't have the fix to SCHANNEL regarding DHE cipher suites. Ad... — committed to dotnet/corefx by davidsh 7 years ago
- Test: ignore specific WinHttpException on old Windows versions The root cause issue was already fixed in Windows but old versions still hit this issue. This change makes the failing test to ignore th... — committed to pjanotti/corefx by deleted user 6 years ago
- Test: ignore specific WinHttpException on old Windows versions (#26428) The root cause issue was already fixed in Windows but old versions still hit this issue. This change makes the failing test to... — committed to dotnet/corefx by deleted user 6 years ago
- Enable ClientAndServer_OneOrBothUseDefault_Ok test Protected against error in old versions of Windows and updated assert to include the case when hash algorithm can be Sha1. Fixes #7812 (actual fix ... — committed to pjanotti/corefx by deleted user 6 years ago
- Enable ClientAndServer_OneOrBothUseDefault_Ok test (#26463) * Enable ClientAndServer_OneOrBothUseDefault_Ok test Protected against error in old versions of Windows and updated assert to include t... — committed to dotnet/corefx by deleted user 6 years ago
- appveyor: disable DHE to avoid spurious failures Our CI builds have intermittent failures in our online tests, e.g. with the message "A provided buffer was too small". This is not a programming error... — committed to pks-t/libgit2 by pks-t 6 years ago
- appveyor: disable DHE to avoid spurious failures Our CI builds have intermittent failures in our online tests, e.g. with the message "A provided buffer was too small". This is not a programming error... — committed to pks-t/libgit2 by pks-t 6 years ago
- appveyor: disable DHE to avoid spurious failures Our CI builds have intermittent failures in our online tests, e.g. with the message "A provided buffer was too small". This is not a programming error... — committed to pks-t/libgit2 by pks-t 6 years ago
- appveyor: disable DHE to avoid spurious failures Our CI builds have intermittent failures in our online tests, e.g. with the message "A provided buffer was too small". This is not a programming error... — committed to pks-t/libgit2 by pks-t 6 years ago
We have completed the root cause analysis of this problem. The problem is being caused by a Windows OS bug in SCHANNEL which is the low-level component that handles SSL/TLS handshakes… This bug has been fixed in the very latest Windows 10 and Windows Server 2016 releases.
This bug can manifest in not just .NET Core HttpClient/WinHttpHandler but also with any code that uses SSL/TLS on Windows such as SslStream, .NET Framework, Internet Explorer.
There is a bug in how TLSv1.2 handshakes are done when the client and server negotiate a DHE based cipher suite. And more specifically, only when a Public Key Length of 127 is negotiated. Key lengths can vary and lengths such as 128 are ok. This is why this problem appears “random” and hard to repro.
The bug is hard to diagnose because the errors from SCHANNEL in Windows get obscured and instead other errors are returned by components such as WinHttp.
See this article for an explanation of the root cause when this bug manifests as a connection error with Internet Explorer: https://connect.microsoft.com/IE/feedback/details/1253526/tls-serverkeyexchange-with-1024-dhe-may-encode-dh-y-as-127-bytes-breaking-internet-explorer-11
More detailed background: https://security.stackexchange.com/questions/104845/dhe-rsa-pubkey-length-in-tls-1-2
There are no simple workarounds for this except for the following:
We will be looking at ways to mitigate the random test failure noise caused by this bug in the CI system by adjusting our tests and test machine configurations.
@karelz I don’t think this error is being prioritized sufficiently. I just had a developer hit this error with a local development environment today. When hit, it’s an all-stop issue for a production environment.
I don’t know. You might want to follow up with Azure support.
I do know that many customers have been asking for Azure Web Apps to use Windows Server 2016 as the backend. It currently uses Windows Server 2012. Customers would like Server 2016 for Web Apps since that version of Windows OS has HTTP/2 support for example.
In terms of which Windows 10 / Windows Server 2016 build numbers have the fix:
This was fixed in 10.0.14393. That means it is fixed in Windows Server 2016 since the RTM release of that server SKU was build 14393. For Windows 10, it means “Windows 10 Version 1607 (Anniversary Update)” which is build 14393. See: https://en.wikipedia.org/wiki/Windows_10_version_history
Azure Web Apps (formally known as Azure WebSites), currently runs on Windows Server 2012. I have heard that they are actively working on upgrading their systems to Windows Server 2016. Many developers are asking for this. And upgrading to Windows Server 2016 also means that Azure Web Apps hosted on that will support HTTP/2.0 as well (since Windows Server 2016 is the first server SKU to support that). Please contact Azure support for more information on their timelines.
does anyone know when this will be applied to Azure Web Apps? @davidsh do you know if there is a specific Windows 10 / Windows Server 2016 build number that we could relate the presence of the fix to?
I have the issue popping up in production on a .NET Core 1.0.4 app running on Azure Web App even today.
It hasn’t been assigned yet (we have it on backlog and it will be eventually grabbed by someone). As I said above, I can 99% guarantee it won’t be punted. Mainly because of the impact on real-world scenarios (4 customer reports).
Unless it requires 6 months of work and rewriting of half of the Networking stack, we should address it in 2.0. If it is so costly that it doesn’t fit into 2.0 final release, we would definitely consider addressing it in servicing release. Again, just based on customer impact. Someone should pick it up in next couple of weeks. I hope it helps.
I will kick off a test soon and report back tomorrow. A previous mass-run of Http.Functional tests with all ActiveIssue tests disabled was not successful (or trustworthy) due to certain re-enabled tests to cause hangs, thus requiring selective re-enablement of tests going forward.
After re-reading the thread, it actually looks that 2 people hit it in the wild. That’s concerning. I think we should look into it in 2.0.
I’m encountering these same errors in an azure production environment. I run asp.net core on top of the full .net 4.6.1 so I suspect it’s not specific to .net core. It happens as the code is trying to send an email using the SendGrid nuget package, which internally seems to use HttpClient. Usually the error is ‘A security error occurred’ (it happened 5 times so far), but there is also 1 occurrence of ‘The buffers supplied to a function was too small’. Here is the top of the stack:
System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: A security error occurred at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Net.Http.WinHttpHandler.<StartRequest>d__103.MoveNext() --- End of inner exception stack trace --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Net.Http.HttpClient.<FinishSendAsync>d__58.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult() at SendGrid.Web.<DeliverAsync>d__7.MoveNext()`