runtime: [macOS-arm64] CoreCLR System.Security.Cryptography.OpenSsl.Tests fail on M1 Helix queues

From https://dev.azure.com/dnceng/public/_build/results?buildId=1019012&view=results

No usable version of libssl was found

----- start Tue Mar 2 10:20:41 PST 2021 =============== To repro directly: =====================================================
pushd .
/tmp/helix/working/A43F08FC/p/dotnet exec --runtimeconfig System.Security.Cryptography.OpenSsl.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.OpenSsl.Tests.deps.json xunit.console.dll System.Security.Cryptography.OpenSsl.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing 
popd
===========================================================================================================
/private/tmp/helix/working/A43F08FC/w/C8540A7F/e /private/tmp/helix/working/A43F08FC/w/C8540A7F/e
  Discovering: System.Security.Cryptography.OpenSsl.Tests (method display = ClassAndMethod, method display options = None)
No usable version of libssl was found
./RunTests.sh: line 162: 59272 Abort trap: 6           (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Security.Cryptography.OpenSsl.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.OpenSsl.Tests.deps.json xunit.console.dll System.Security.Cryptography.OpenSsl.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing $RSP_FILE
/private/tmp/helix/working/A43F08FC/w/C8540A7F/e
----- end Tue Mar 2 10:20:43 PST 2021 ----- exit code 134 ----------------------------------------------------------

----- start Wed Mar 3 17:37:29 PST 2021 =============== To repro directly: =====================================================
pushd .
/tmp/helix/working/C24F0A5F/p/dotnet exec --runtimeconfig System.Security.Cryptography.Algorithms.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.Algorithms.Tests.deps.json xunit.console.dll System.Security.Cryptography.Algorithms.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=failing 
popd
===========================================================================================================
/private/tmp/helix/working/C24F0A5F/w/B80F0986/e /private/tmp/helix/working/C24F0A5F/w/B80F0986/e
  Discovering: System.Security.Cryptography.Algorithms.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Security.Cryptography.Algorithms.Tests (found 1595 of 1601 test cases)
  Starting:    System.Security.Cryptography.Algorithms.Tests (parallel test collections = on, max threads = 8)
    System.Security.Cryptography.Encryption.TripleDes.Tests.TripleDESContractTests.Windows7DoesNotSupportCFB64 [SKIP]
      Condition(s) not met: "IsWindows7"
    System.Security.Cryptography.Dsa.Tests.DSAImportExport.Import_2048 [SKIP]
      Condition(s) not met: "SupportsFips186_3"
    System.Security.Cryptography.Dsa.Tests.DSAImportExport.ExportAutoKey [SKIP]
      Condition(s) not met: "SupportsKeyGeneration"
    System.Security.Cryptography.Encryption.Des.Tests.DesContractTests.Windows7DoesNotSupportCFB [SKIP]
      Condition(s) not met: "IsWindows7"
    System.Security.Cryptography.Encryption.Aes.Tests.AesModeTests.Windows7DoesNotSupportCFB128 [SKIP]
      Condition(s) not met: "IsWindows7"
No usable version of libssl was found
./RunTests.sh: line 162: 37821 Abort trap: 6           (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Security.Cryptography.Algorithms.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.Algorithms.Tests.deps.json xunit.console.dll System.Security.Cryptography.Algorithms.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=failing $RSP_FILE
/private/tmp/helix/working/C24F0A5F/w/B80F0986/e
----- end Wed Mar 3 17:37:37 PST 2021 ----- exit code 134 ----------------------------------------------------------

/cc @directhex

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 25 (23 by maintainers)

Most upvoted comments

It’s certainly not a pleasing state to be in, but I’m not expecting a lot of major issues.

Our use of OpenSSL on macOS is limited to RSAOpenSsl, DSAOpenSsl, ECDsaOpenSsl, ECDiffieHellmanOpenSsl, AesGcm, and AesCcm. The first four are pretty obvious that OpenSSL is required, and no platform code will instantiate them on macOS. The latter two aren’t instantiated by platform code, and use the static IsSupported pattern to indicate that they’re not always available.

The problem at the time was that the libssl/libcrypto that the test machines get from Homebrew weren’t the same architecture as the dotnet process. Assuming that we can solve the architecture mismatch on our test rigs (presumably there are architecture-specific library load paths, and we need to a) download two copies of OpenSSL via homebrew and b) link them in with the arch paths instead of the common path) we can turn the tests back on.

If it’s the case that Homebrew doesn’t offer OpenSSL arm64 then the lightup features that depend on OpenSSL are just not available on that platform.

bartonjs on Mar 11, 2022

I can take a look. I have ARM macBook so I can easily do local test runs before we jumping to CI.

wfurt on Sep 10, 2021