runtime: OpenSslCryptographicException: error:03000098:digital envelope routines::invalid digest on CentOS Stream 9

@vcsjones unless you are available to pick this up, I will take a shot at it.

I will make an attempt but time box it. If it turns out to be more work than expected I may need to defer (my time spent in this repo is purely voluntary).

@tmds @omajid one other thing I wanted to call out about the SHA1 signature disabling in RHEL9 and .NET is how this interacts with TLS.

Currently, this trivial .NET program will fail in RHEL9.

using System.Net.Http;

using HttpClient client = new HttpClient();
using HttpResponseMessage response = await client.GetAsync("https://pkgs.dev.azure.com");

The remote certificate is invalid because of errors in the certificate chain: NotSignatureValid

I suspect this is because .NET validates signatures on self-signed in trust stores, whereas OpenSSL does not normally do this. OpenSSL does support this with -check_ss_sig. You can reproduce this with the command line:

openssl s_client -check_ss_sig -strict -debug -connect pkgs.dev.azure.com:443

This will connect, but the trace will indicate the same error:

Verify return code: 7 (certificate signature failure)

I don’t know if this is a “problem” or not, but since .NET opts in to this strictness check whereas OpenSSL doesn’t normally do this, you might see additional reports about .NET Failing to connect to HTTPS endpoints where the root certificate is RSA-SHA1 signed.

@vcsjones I assume you didn’t find time for this. I’ll give it a shot next week.

No. It was added specifically because .NET on Windows was rejecting things that .NET on Linux wasn’t. This is one of those places where I feel like “.NET is .NET” is the better compat answer.

That breaks .NET on RHEL9. If the system considers the certificate secure, .NET shouldn’t reject it for Windows-compatibilty sake.

X509_STORE_CTX_set_flags(ctx, X509_V_FLAG_CHECK_SS_SIGNATURE);

@bartonjs can we drop this strict mode? It causes .NET to reject certificates on RHEL9 which are accepted by system tools like curl and wget.

What happens when you use the url with wget/curl/…? Do they also use the ‘strict’ mode?

They do not appear to, no. We explicitly enable it here:

https://github.com/dotnet/runtime/blob/424a09cb81c678fb1ba1c27211b80aba2de070ad/src/native/libs/System.Security.Cryptography.Native/pal_x509.c#L277

.NET should be made less strict, and accept it certificate too.

I think the reason why this was added is because this is what Windows does. Windows checks the signature on self-signed certificates. So this was done so that X509 chain building is more consistent across platforms, as I understand it.

@bartonjs may want to weigh in on this one when he returns.

@tmds @omajid I can get the tests to fail in CentOS 9 Stream. Here is a Docker file which does so.

This was done with the base image sha256:c498ab29be98c552440487f06e78ba22b3892f4b29d37aa208fe7ded95280f17. Note: this hard-codes some paths to assume ARM64. If you are running this docker image on x64, you will need to adjust some paths in the CMD.

I think this probably means somewhere in your build / test infrastructure, SHA1 is being enabled.

Note that with SHA1 signatures disabled, quite a number of things do not actually work. I can’t even get it to build without enabling SHA1 signatures in a few places.

FROM quay.io/centos/centos:stream9

RUN dnf -y groupinstall 'Development Tools' && \
    dnf install -y dnf-plugins-core epel-release && \
    dnf repolist --all && \
    dnf config-manager --set-enabled crb && \
    dnf install -y \
        clang \
        cmake \
        git \
        glibc-langpack-en \
        hostname \
        krb5-devel \
        libicu-devel \
        lld \
        llvm \
        make \
        openssl-devel \
        python3 \
        tar \
        util-linux \
        zlib-devel \
        lttng-ust-devel \
        libunwind-devel

RUN mkdir /projects
WORKDIR /projects
RUN git clone https://github.com/dotnet/runtime.git
WORKDIR /projects/runtime

# OPENSSL_ENABLE_SHA1_SIGNATURES needs to be set for project restore to work
# since ADO is doing RSA SHA1 signatures somewhere in TLS.
# We don't want to export it though so that unit tests don't pick this up.
RUN OPENSSL_ENABLE_SHA1_SIGNATURES=1 ./build.sh -rc release -s clr+libs

# Build tests with OPENSSL_ENABLE_SHA1_SIGNATURES so that everything
# gets restored and built.
RUN OPENSSL_ENABLE_SHA1_SIGNATURES=1 \
    ./dotnet.sh build src/libraries/System.Security.Cryptography/tests

# Actual run does NOT have OPENSSL_ENABLE_SHA1_SIGNATURES.
CMD artifacts/bin/testhost/net7.0-Linux-Debug-arm64/dotnet exec \
    --runtimeconfig artifacts/bin/System.Security.Cryptography.Tests/Debug/net7.0-unix/System.Security.Cryptography.Tests.runtimeconfig.json \
    artifacts/bin/System.Security.Cryptography.Tests/Debug/net7.0-unix/xunit.console.dll \
    artifacts/bin/System.Security.Cryptography.Tests/Debug/net7.0-unix/System.Security.Cryptography.Tests.dll \
    -notrait category=OuterLoop -notrait category=failing

I get the following test results:

=== TEST EXECUTION SUMMARY ===
   System.Security.Cryptography.Tests  Total: 6205, Errors: 0, Failed: 39, Skipped: 17, Time: 192.750s

An example failure:

Interop+Crypto+OpenSslCryptographicException : error:03000098:digital envelope routines::invalid digest
      Stack Trace:
    System.Security.Cryptography.Rsa.Tests.SignVerify_Array.InvalidKeySize_DoesNotInvalidateKey [FAIL]
        /projects/runtime/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.EvpPkey.Rsa.cs(147,0): at Interop.Crypto.RsaSignHash(SafeEvpPKeyHandle pkey, RSASignaturePaddingMode paddingMode, IntPtr digestAlgorithm, ReadOnlySpan`1 hash, Span`1 destination)
        /projects/runtime/src/libraries/Common/src/System/Security/Cryptography/RSAOpenSsl.cs(796,0): at System.Security.Cryptography.RSAOpenSsl.TrySignHash(ReadOnlySpan`1 hash, Span`1 destination, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding, Boolean allocateSignature, Int32& bytesWritten, Byte[]& signature)
        /projects/runtime/src/libraries/Common/src/System/Security/Cryptography/RSAOpenSsl.cs(726,0): at System.Security.Cryptography.RSAOpenSsl.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
<snip>

Is this worth documenting somewhere?

I’m not sure where/what we’d document.

From the RSA perspective, it only applies when on Fedora/CentOS/RHEL when using RSAOpenSsl or RSA.Create() (a custom RSA implementation, such as RSAKeyVault, wouldn’t necessarily have the problem), and at best we’d just link to the RedHat crypto docs for it. (From an API perspective, it’s covered under the blanket docs of “if something goes wrong: CryptographicException”… and this is just “the underlying provider doesn’t support the hash algorithm”. I think RSACryptoServiceProvider -might- still support RSA+MD4, but RSAOpenSsl doesn’t).

From the assembly strong name perspective, we already recommend public sign for .NET Core projects, and don’t know where in that doc stream anyone would find “if building for .NET Framework or .NET Standard (for .NET Framework) from [a RedHat-influenced distro], things get complicated”.

So, I’m not opposed to documentation, just can’t think of where it’d go and how to explain the problem or solution in an approachable manner 😄.

For what it’s worth, this breaks any .NET project that uses strong name signing, not just building .NET itself.

To reproduce, create a new C# project. Add this somewhere:

[assembly:System.Reflection.AssemblyKeyFile("testkey.snk")]

Grab a strong name key. You can use testkey.snk in this repository. Put it next to the csproj.

It will fail to build with the same error.