runtime: mono `minijit` runtime tests failing with `coreclr_initialize failed - Error: 0x80004005`
Failures 4/5-9/8 (incl. PRs and 7.0 branches):
Day | Run | OS |
---|---|---|
9/5 | Rolling run 5191 (only in Runfo) | mono Linux arm64 Release @ (Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open |
9/5 | Rolling run 4698 (only in Runfo) | mono Linux arm64 Release @ (Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open |
8/30 | Rolling run | Ubuntu.1804.Arm64.Open - Mono Interpreter |
8/30 | PR #74831 | OSX.1200.Amd64.Open - Mono Interpreter |
8/24 | PR #74459 | OSX 12 amd64 - Moni minijit |
8/8 | PR #73416 | OSX 12 amd64 - Mono Interpreter |
8/4 | PR #73230 | OSX 12 amd64 - Mono Interpreter |
8/2 | PR #73157 | OSX 12 amd64 - Moni minijit |
7/29 | Rolling run | Ubuntu.1804 arm64 - Mono llvm full aot |
7/12-7/27 | 8x unique PRs | OSX 12 amd64 and Ubuntu.1804 arm64 |
6/1-6/30 | 6x unique PRs | OSX 12 amd64 and Ubuntu.1804 arm64 |
5/1-5/31 | 2x unique PRs | OSX 12 amd64 |
4/1-4/30 | 6x unique PRs | OSX 12 amd64 and Ubuntu.1804 arm64 |
let failedTests = (testNameSubstring : string, methodName : string, includePR : bool, messageSubstr: string, includePassedOnRerun : bool) {
cluster('engsrvprod.kusto.windows.net').database('engineeringdata').AzureDevOpsTests
| where TestName contains testNameSubstring
| where includePassedOnRerun or (Outcome == 'Failed')
| extend startOfTestName = indexof_regex(TestName, @"[^.]+$")
| extend Method = substring(TestName, startOfTestName)
| extend Type = substring(TestName, 0, startOfTestName - 1)
| project-away startOfTestName
| where (methodName == '') or (Method == methodName)
| where Message contains messageSubstr
| distinct JobId, WorkItemId, Message, StackTrace, Method, Type, Arguments, Outcome
| join kind=inner (cluster('engsrvprod.kusto.windows.net').database('engineeringdata').Jobs
| where ((Branch == 'refs/heads/main') or (Branch == 'refs/heads/master') or (Branch startswith 'refs/heads/release/7.0') or (includePR and (Source startswith "pr/")))
| where Type startswith "test/functional/cli/"
and not(Properties contains "runtime-staging")
| summarize arg_max(Finished, Properties, Type, Branch, Source, Started, QueueName) by JobId
| project-rename JobType = Type) on JobId
| extend PropertiesJson = parse_json(Properties)
| extend OS = replace_regex(tostring(PropertiesJson.operatingSystem), @'\((.*)\).*|([^\(].*)', @'\1\2')
| extend Runtime = iif(PropertiesJson.runtimeFlavor == "mono", "Mono", iif(PropertiesJson.DefinitionName contains "coreclr", "CoreCLR", ""))
| extend TargetBranch = extractjson("$.['System.PullRequest.TargetBranch']", Properties)
| extend Architecture = PropertiesJson.architecture
| extend Scenario = iif(isempty(PropertiesJson.scenario), "--", PropertiesJson.scenario)
//| extend DefinitionName = PropertiesJson.DefinitionName
| project-away PropertiesJson
};
failedTests('JIT/Math/Functions/Functions_ro', '', true, 'coreclr_initialize', true);
Hit on https://github.com/dotnet/runtime/pull/72094 . Build. From one of the logs:
JIT/Math/Functions/Functions_ro/Functions_ro.sh [FAIL]
BEGIN: coreclr_initialize failed - Error: 0x80004005
END: coreclr_initialize failed - Error: 0x80004005
Return code: 1
Raw output file: /tmp/helix/working/B5FB0986/w/AC37094B/uploads/Reports/JIT.Math/Functions/Functions_ro/Functions_ro.output.txt
Raw output:
BEGIN EXECUTION
/tmp/helix/working/B5FB0986/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false Functions_ro.dll ''
Exe path: /private/tmp/helix/working/B5FB0986/p/corerun
Properties:
TRUSTED_PLATFORM_ASSEMBLIES = <see console log for the full list>
APP_PATHS = /private/tmp/helix/working/B5FB0986/w/AC37094B/e/JIT/Math/Functions/Functions_ro/
NATIVE_DLL_SEARCH_DIRECTORIES = /private/tmp/helix/working/B5FB0986/w/AC37094B/e/JIT/Math/Functions/Functions_ro/:/tmp/helix/working/B5FB0986/p/:
System.Reflection.Metadata.MetadataUpdater.IsSupported = false
Managed assembly: /private/tmp/helix/working/B5FB0986/w/AC37094B/e/JIT/Math/Functions/Functions_ro/Functions_ro.dll
Arguments (0):
Expected: 100
Actual: 255
END EXECUTION - FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=/tmp/helix/working/B5FB0986/p
> /private/tmp/helix/working/B5FB0986/w/AC37094B/e/JIT/Math/Functions/Functions_ro/Functions_ro.sh
Expected: True
Actual: False
Stack Trace:
at JIT_Math._Functions_Functions_ro_Functions_ro_._Functions_Functions_ro_Functions_ro_sh()
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
Output:
BEGIN: coreclr_initialize failed - Error: 0x80004005
END: coreclr_initialize failed - Error: 0x80004005
Return code: 1
Raw output file: /tmp/helix/working/B5FB0986/w/AC37094B/uploads/Reports/JIT.Math/Functions/Functions_ro/Functions_ro.output.txt
Raw output:
BEGIN EXECUTION
/tmp/helix/working/B5FB0986/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false Functions_ro.dll ''
Exe path: /private/tmp/helix/working/B5FB0986/p/corerun
Properties:
TRUSTED_PLATFORM_ASSEMBLIES = <see console log for the full list>
APP_PATHS = /private/tmp/helix/working/B5FB0986/w/AC37094B/e/JIT/Math/Functions/Functions_ro/
NATIVE_DLL_SEARCH_DIRECTORIES = /private/tmp/helix/working/B5FB0986/w/AC37094B/e/JIT/Math/Functions/Functions_ro/:/tmp/helix/working/B5FB0986/p/:
System.Reflection.Metadata.MetadataUpdater.IsSupported = false
Managed assembly: /private/tmp/helix/working/B5FB0986/w/AC37094B/e/JIT/Math/Functions/Functions_ro/Functions_ro.dll
Arguments (0):
Expected: 100
Actual: 255
END EXECUTION - FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=/tmp/helix/working/B5FB0986/p
> /private/tmp/helix/working/B5FB0986/w/AC37094B/e/JIT/Math/Functions/Functions_ro/Functions_ro.sh
cc @lewing
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 19 (18 by maintainers)
@karelz thanks for checking! We have enough information to move forward now.