runtime: SslStream throws when using OpenSSL cert

Description

[Simplified from #84505]

We’re seeing an AuthenticationException from SslStream when using a certificate from OpenSSL in a server running on Windows.

Reproduction Steps

Generate a couple of certs with openssl (they seem to have slightly different failure modes)

openssl req -new -newkey rsa:4096 -x509 -nodes -days 365 -keyout server.key -out server.crt -subj "/C=SG/ST=Singapore/L=Singapore /O=My Company Pte. Ltd./OU=My Organization/CN=localhost/emailAddress=a@b.c"
openssl pkcs12 -export -out twocerts.pfx -inkey server.key -in server.crt -certfile server.crt -passout pass:mypass
openssl pkcs12 -export -out onecert.pfx -inkey server.key -in server.crt -passout pass:mypass

Create a dotnet new console app with the following code (fix cert path for your box)

using System.Net;
using System.Net.Security;
using System.Net.Sockets;
using System.Security.Cryptography.X509Certificates;

TcpListener listener = new TcpListener(IPAddress.Any, 5000);
listener.Start();

TcpClient client = listener.AcceptTcpClient();

SslStream sslStream = new SslStream(client.GetStream(), false);
var serverCertificate = new X509Certificate2(@"c:\tmp\opensslCerts\twocerts.pfx", "mypass");
sslStream.AuthenticateAsServer(serverCertificate, clientCertificateRequired: false, checkCertificateRevocation: false);

Navigate to https://localhost:5000

Expected behavior

No exception (the server code is mangled to be minimal and wouldn’t do anything sensible if it made it past the exception).

Actual behavior

I’m seeing several different callstacks, some of which I suspect are expected.

For twocerts.pfx and curl https://localhost:5000 --insecure (or navigating to https://localhost:5000 in Edge)

Unhandled exception. System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
 ---> System.ComponentModel.Win32Exception (0x8009030D): The credentials supplied to the package were not recognized
   at System.Net.SSPIWrapper.AcquireCredentialsHandle(ISSPIInterface secModule, String package, CredentialUse intent, SCH_CREDENTIALS* scc)
   at System.Net.Security.SslStreamPal.AcquireCredentialsHandle(CredentialUse credUsage, SCH_CREDENTIALS* secureCredential)
   at System.Net.Security.SslStreamPal.AcquireCredentialsHandleSchCredentials(SslAuthenticationOptions authOptions)
   at System.Net.Security.SslStreamPal.AcquireCredentialsHandle(SslAuthenticationOptions sslAuthenticationOptions, Boolean newCredentialsRequested)
   --- End of inner exception stack trace ---
   at System.Net.Security.SslStreamPal.AcquireCredentialsHandle(SslAuthenticationOptions sslAuthenticationOptions, Boolean newCredentialsRequested)
   at System.Net.Security.SslStream.AcquireCredentialsHandle(SslAuthenticationOptions sslAuthenticationOptions, Boolean newCredentialsRequested)
   at System.Net.Security.SslStream.AcquireServerCredentials(Byte[]& thumbPrint)
   at System.Net.Security.SslStream.GenerateToken(ReadOnlySpan`1 inputBuffer, Byte[]& output)
   at System.Net.Security.SslStream.NextMessage(ReadOnlySpan`1 incomingBuffer)
   at System.Net.Security.SslStream.ProcessBlob(Int32 frameSize)
   at System.Net.Security.SslStream.ReceiveBlobAsync[TIOAdapter](CancellationToken cancellationToken)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)
   at System.Net.Security.SslStream.AuthenticateAsServer(SslServerAuthenticationOptions sslServerAuthenticationOptions)
   at System.Net.Security.SslStream.AuthenticateAsServer(X509Certificate serverCertificate, Boolean clientCertificateRequired, Boolean checkCertificateRevocation)
   at Program.<Main>$(String[] args) in c:\tmp\opensslCerts\repro\Program.cs:line 13

For onecert.pfx and navigating to https://localhost:5000 in Edge (but not curl https://localhost:5000 --insecure). (This might just be because the cert is self-signed or because the server code is incomplete?) Edit: Yes, it’s expected because there’s no SAN and the cert isn’t trusted (the unknown error is a fatal TLS alert)

Unhandled exception. System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
 ---> System.ComponentModel.Win32Exception (0x80090327): An unknown error occurred while processing the certificate.
   --- End of inner exception stack trace ---
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)
   at System.Net.Security.SslStream.AuthenticateAsServer(SslServerAuthenticationOptions sslServerAuthenticationOptions)
   at System.Net.Security.SslStream.AuthenticateAsServer(X509Certificate serverCertificate, Boolean clientCertificateRequired, Boolean checkCertificateRevocation)
   at Program.<Main>$(String[] args) in c:\tmp\opensslCerts\repro\Program.cs:line 13

Probably expected: for either cert and either client, but address http://localhost:5000 (note: http, not https) Edit: Yes, it’s expected

Unhandled exception. System.Security.Authentication.AuthenticationException: Cannot determine the frame size or a corrupted frame was received.
   at System.Net.Security.SslStream.GetFrameSize(ReadOnlySpan`1 buffer)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at System.Net.Security.SslStream.ReceiveBlobAsync[TIOAdapter](CancellationToken cancellationToken)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)
   at System.Net.Security.SslStream.AuthenticateAsServer(SslServerAuthenticationOptions sslServerAuthenticationOptions)
   at System.Net.Security.SslStream.AuthenticateAsServer(X509Certificate serverCertificate, Boolean clientCertificateRequired, Boolean checkCertificateRevocation)
   at Program.<Main>$(String[] args) in c:\tmp\opensslCerts\repro\Program.cs:line 13

Regression?

The behavior appears to be the same in dotnet 7.0.200.

Known Workarounds

No response

Configuration

OS: Microsoft Windows 11 Enterprise (10.0.22621 Build 22621) OpenSSL: 1.1.1s 1 Nov 2022 (from git version 2.39.2.windows.1) dotnet: 8.0.100-preview.4.23210.1

In the course of experimenting, I’ve seen similar behavior on win10 and with certs from openssl 3, but I haven’t rigorously built a matrix.

Other information

onecert.pfx.txt twocerts.pfx.txt

There are the files I tested with. Obviously, you’ll need to remove the .txt I added to placate GH.

About this issue

Original URL
State: open
Created a year ago
Comments: 22 (20 by maintainers)

Most upvoted comments

I still see "An unknown error occurred while processing the certificate." I installed the .pfx into both Current User and Local Computer.

khteh on Apr 12, 2023

the only actual issue is that passing the same cert as -in and -certfile doesn’t work with schannel

In general this confuses Windows’ CryptoAPI when you two identical certificates in a PKCS12 document that reference the same key. The problem can be distilled down to:

using System.Security.Cryptography.X509Certificates;

byte[] p12 = Convert.FromBase64String(@"
MIIXGQIBAzCCFt8GCSqGSIb3DQEHAaCCFtAEghbMMIIWyDCCDP8GCSqGSIb3DQEH
BqCCDPAwggzsAgEAMIIM5QYJKoZIhvcNAQcBMBwGCiqGSIb3DQEMAQYwDgQIpaCg
// snip for brevity
KVi262UlMwMQA4/lNhvT5cQzkDElMCMGCSqGSIb3DQEJFTEWBBQXIRX2Gn15+7g5
eZ3M/TDGBHWTqDAxMCEwCQYFKw4DAhoFAAQUPRj5g258sM8ZkrVEEuWxlOtiexME
CNg17IIVAVbbAgIIAA==");

X509Certificate2 c = new X509Certificate2(p12, "mypass");
Console.WriteLine(c.GetRSAPrivateKey());

That’s going to fail with CryptographicException: Keyset does not exist. So SChannel ends up believing that the provided certificate has no private key, which it surfaces as “invalid credentials”.

The reason for this is because you have two certificates in the PKCS12 document saying “That’s my private key”. But X509Certificate2 can only represent one certificate, so we end up cleaning up the other certificate. Cleaning up that certificate ends up disposing the underlying key that’s shared.

https://github.com/dotnet/runtime/blob/25be848b0ec2cceda77811edc915b86c0b83aacd/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/CertificatePal.Windows.Import.cs#L195-L198

We might be able to improve this to make sure we aren’t cleaning up the key handle if the returned certificate kept it.

Since this is in X509Certificate2 and can be reproduced without SslStream, re-homing to area-System.Security.

vcsjones on Apr 10, 2023