runtime: Provide a "fail at runtime" mechanism for better LLVMFullAOT testability

We would like to add the ability to defer some AOT compilation errors until execution time.

When the AOT compiler is generating code, there are some assemblies that contain code that it cannot handle (for example platform-dependent code that is never executed on mobile that depends on assemblies that are only present on desktop Windows). In this case instead of preventing the whole assembly from compiling, we should defer the actual error until execution time.

Additionally the runtime tests in src/tests include some test cases that intentionally have malformed metadata or that are expected to not compile with AOT. In this case, we would like to allow some of the methods in the assemblies to be AOTed, without the problematic test cases blocking the testing of the whole assembly.

Tasks:

• Add an aot compiler option to allow some class of errors to be AOT-time warnings and to defer the actual error to runtime (initially as our generic “Attempting to JIT compile method… while running in aot-only mode”) #64640
• Adjust mono AOT compiler errors and warnings to emit the prefix that MSBuild uses to capture warnings/errors from Exec tasks
• Add allow-errors option to the MonoAOTCompiler task
• Enable 'allow-errors' mode in CI, re-enable disabled tests
• For cases where it makes sense, make Mono AOT produce the same warning/error codes as NativeAOT
• Add some support for specific execution-time AOT failure errors (either by emitting stub method bodies, or adding some error table to the AOT images, or something else)

Original issue, below

---

In the src/tests test tree, we test a number of corner cases of invalid code, including type layouts and UnmanagedCallersOnly usage. When running tests on Mono with LLVMFullAOT, some of these failures, including ones seen https://github.com/dotnet/runtime/pull/63320 and in some tests I’ve listed below cause AOT-time failures. As a result, we need to exclude these assemblies from running through the AOT compiler.

Today, we handle this by excluding the tests from AOT compilation via issues.targets. However, we’re looking at moving away from issues.targets and towards using [ActiveIssue] attributes throughout our whole repo. This gets into a problem where we can’t easily exclude methods from LLVMFullAOT, but we can’t easily mark an assembly as “skip AOT” in the attribute model.

I talked with @imhameed offline, and we thought of an idea that is similar to some mechanisms used within CoreCLR’s NativeAOT for some scenarios like IL stub generation: Provide a mode for the Mono AOT compiler that, on failure, emits a stub method body that throws the exception that would have been thrown if the method was generated at runtime. In this mode, AOT compilation would succeed for cases where the program will ultimately fail at runtime, whereas today it fails at AOT-compilation time.

This will allow us to re-enable some tests on llvmfullaot runs or at least exclude them in a manner more accessible with [ActiveIssue] attributes.

Tests that are disabled today because they fail at AOT-time instead of run time:

• All tests in the Loader/classloader/explicitlayout/objrefandnonobjrefoverlap subtree
• Loader/classloader/explicitlayout/NestedStructs/case03
• Loader/classloader/explicitlayout/NestedStructs/case04
• Loader/classloader/explicitlayout/NestedStructs/case05
• The RuntimeMarshallingDisabled_NativeAssemblyEnabled test suite in #63320

There may be others in that section that are disabled for the same reason but based on the issue metadata it looks like they fail at runtime.

cc: @MichalStrehovsky I don’t know if there’s other places in NativeAOT where a mechanism like this might be useful for testing scenarios. cc: @trylek as this affects our work with refactoring the test tree