runtime: [macOS 11] System.Tests.EnvironmentTests.OSVersion_ValidVersion_OSX fails on CoreCLR

See https://dev.azure.com/dnceng/public/_build/results?buildId=1021947

----- start Wed Mar 3 17:37:13 PST 2021 =============== To repro directly: =====================================================
pushd .
/tmp/helix/working/C24F0A5F/p/dotnet exec --runtimeconfig System.Runtime.Extensions.Tests.runtimeconfig.json --depsfile System.Runtime.Extensions.Tests.deps.json xunit.console.dll System.Runtime.Extensions.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=failing 
popd
===========================================================================================================
/private/tmp/helix/working/C24F0A5F/w/B1000946/e /private/tmp/helix/working/C24F0A5F/w/B1000946/e
  Discovering: System.Runtime.Extensions.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Runtime.Extensions.Tests (found 895 of 964 test cases)
  Starting:    System.Runtime.Extensions.Tests (parallel test collections = on, max threads = 8)
    System.Tests.EnvironmentTests.GetFolderPath_UWP_NotEmpty [SKIP]
      Condition(s) not met: "IsWindows10Version1709OrGreater", "IsInAppContainer"
    System.Tests.EnvironmentTests.OSVersion_ValidVersion_OSX [FAIL]
      Assert.Contains() Failure
      Not found: 11.2
      In value:  osx.11.0-arm64
      Stack Trace:
        /_/src/libraries/System.Runtime.Extensions/tests/System/EnvironmentTests.cs(191,0): at System.Tests.EnvironmentTests.OSVersion_ValidVersion_OSX()
    System.Tests.EnvironmentTests.GetFolderPath_UWP_ExistAndAccessible [SKIP]
      Condition(s) not met: "IsWindows10Version1709OrGreater", "IsInAppContainer"
    System.Tests.EnvironmentTests.UserInteractive_WindowsNano [SKIP]
      Condition(s) not met: "IsWindowsNanoServer"
  Finished:    System.Runtime.Extensions.Tests
=== TEST EXECUTION SUMMARY ===
   System.Runtime.Extensions.Tests  Total: 7509, Errors: 0, Failed: 1, Skipped: 3, Time: 2.720s
/private/tmp/helix/working/C24F0A5F/w/B1000946/e
----- end Wed Mar 3 17:37:16 PST 2021 ----- exit code 1 ----------------------------------------------------------

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 23 (23 by maintainers)

Commits related to this issue

Remove ActiveIssue on OSVersion_ValidVersion_OSX This test was fixed to work correctly on the latest macOS versions in https://github.com/dotnet/runtime/pull/64565. The test can now be re-enabled. F... — committed to eerhardt/runtime by eerhardt 2 years ago
Remove ActiveIssue on OSVersion_ValidVersion_OSX (#70367) * Remove ActiveIssue on OSVersion_ValidVersion_OSX This test was fixed to work correctly on the latest macOS versions in https://github.co... — committed to dotnet/runtime by eerhardt 2 years ago

Most upvoted comments

I believe we made changes to the RID for macOS 12+ to drop the minor version in the RID calculation. We should update the test to reflect this behavior:

https://github.com/dotnet/runtime/pull/60494

eerhardt on Jan 25, 2022

cc @wfurt as well since he implemented the RID logic here: https://github.com/dotnet/runtime/pull/41176.

I think the Environment.OSVersion.Version is doing the right thing here, it is returning the version the OS says it is (see @sdmaclea’s SOFTWARE UPDATE message above).

I think the RID is incorrect. It shouldn’t be osx.11.0 when running on an 11.2 OS. It shouldn’t contain an incorrect minor version number. We should either:

Keep including the minor version and make it correct, i.e. osx.11.2 or
Drop the minor version for 11.x, just like we do for RHEL and other distros where the minor version doesn’t dictate which assets to select. i.e. osx.11

eerhardt on May 3, 2021

Has Apple made a comment about what their next targetable release will be called? Will it be 12.0, or will it be 11.N?

No and it’s hard to predict, though there are indications that the next version will indeed be 12.x: https://www.macworld.com/article/342024/after-20-years-of-os-x-macos-11-might-be-gone-after-big-sur.html

We’ll know for sure after WWDC which is in ~3 weeks.

akoeplinger on May 21, 2021

RuntimeInformation.RuntimeIdentifier and Environment.OSVersion APIs return osx.11.0-arm64 and 11.2 on macOS 11.2 today.

@ericstj What would you like to see it change to?

To make these consistent, I think we would need to change the RID for macOS 11.X to be just osx.11. Is it feasible to go back in time like that?

jkotas on May 3, 2021

I think we need both:

Add the new macOS versions to the RID graph, to continue the established pattern and to make them targetable
Fix the host code that computes RIDs in runtime/src/native/corehost/hostmisc/pal.unix.cpp. The diagnostic output from the OSVersion_ValidVersion_OSX test above says that the RID computed on macOS 11.2 is osx.11.0-arm64.

jkotas on May 3, 2021