runtime: JIT regression with Android+net6 P7

Description

Context: https://github.com/xamarin/xamarin-android/pull/6072 Context: https://github.com/dotnet/runtime/compare/02f70d0...0605bb3 Context: https://github.com/mono/linker/compare/a07cab7b71a1321a9e68571c0b6095144a177b4e...f574448d16af45f7ac2c4b89d71dea73dec86726

PR xamarin/xamarin-android#6072 is a .NET 6 P7 SDK bump from dotnet/installer@e8b3b6be to dotnet/installer@bd5653f3, which includes dotnet/runtime changes in https://github.com/dotnet/runtime/compare/02f70d0...0605bb3.

Changes within this commit range breaks one of our unit tests: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4952636&view=ms.vss-test-web.build-test-results-tab&runId=23305012&resultId=100052&paneView=debug

Java.Lang.RuntimeException : getNamespace() not supported
   at Java.Interop.JniEnvironment.InstanceMethods.CallObjectMethod(JniObjectReference , JniMethodInfo , JniArgumentValue* ) in Java.Interop.dll:token 0x6000326+0x6e
   at Android.Runtime.JNIEnv.CallObjectMethod(IntPtr , IntPtr , JValue* ) in Mono.Android.dll:token 0x6000a05+0xe
   at Android.Content.Res.IXmlResourceParserInvoker.GetNamespace(String ) in Mono.Android.dll:token 0x60007de+0x52
   at Android.Runtime.XmlPullParserReader..ctor(IJavaObject ) in Mono.Android.dll:token 0x6000ac8+0x3f
   at Android.Runtime.XmlResourceParserReader..ctor(IJavaObject ) in Mono.Android.dll:token 0x6000ac4+0x0
   at Android.Runtime.XmlResourceParserReader.FromNative(IntPtr , JniHandleOwnership ) in Mono.Android.dll:token 0x6000ac7+0x3b
   at Android.Runtime.XmlResourceParserReader.FromJniHandle(IntPtr , JniHandleOwnership ) in Mono.Android.dll:token 0x6000ac6+0x0
   at Android.Content.Res.Resources.GetXml(Int32 ) in Mono.Android.dll:token 0x60007fc+0x36
   at Android.RuntimeTests.XmlReaderPullParserTest.ToLocalJniHandle() in Mono.Android.NET-Tests.dll:token 0x60000f3+0xa
   at System.Reflection.RuntimeMethodInfo.Invoke(Object , BindingFlags , Binder , Object[] , CultureInfo ) in System.Private.CoreLib.dll:token 0x600274a+0x6a
  --- End of managed Java.Lang.RuntimeException stack trace ---
java.lang.RuntimeException: getNamespace() not supported
	at android.content.res.XmlBlock$Parser.getNamespace(XmlBlock.java:143)
	at crc643df67da7b13bb6b1.TestInstrumentation_1.n_onStart(Native Method)
	at crc643df67da7b13bb6b1.TestInstrumentation_1.onStart(Unknown Source:0)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2189)

This failure only happens when the JIT is used. We have a “sibling” app which runs the same tests with the interpreter enabled, and those tests pass.

Repro steps:

1. Configure an Android device.

2. Download the test app: Mono.Android.NET_Tests-Signed.zip

3. Rename Mono.Android.NET_Tests-Signed.zip to Mono.Android.NET_Tests-Signed.apk

4. Install the app on your Android device:

   adb install Mono.Android.NET_Tests-Signed.apk
   

5. In a new shell window, begin capturing adb logcat:

   adb logcat > log.txt
   

6. Run the unit tests:

   $ adb shell am instrument  -e "exclude" "DotNetIgnore" -e "loglevel Verbose" -w "Mono.Android.NET_Tests/xamarin.android.runtimetests.NUnitInstrumentation"
   INSTRUMENTATION_RESULT: failed=3
   INSTRUMENTATION_RESULT: filtered=0
   INSTRUMENTATION_RESULT: inconclusive=0
   INSTRUMENTATION_RESULT: nunit2-results-path=/storage/emulated/0/Android/data/Mono.Android.NET_Tests/files/Documents/TestResults.NUnit.xml
   INSTRUMENTATION_RESULT: passed=772
   INSTRUMENTATION_RESULT: run=777
   INSTRUMENTATION_RESULT: skipped=2
   INSTRUMENTATION_RESULT: total=0
   INSTRUMENTATION_CODE: 0
   

7. Read log.txt:

   E NUnit   :  : Java.Lang.RuntimeException : getNamespace() not supported
   E NUnit   :    at Java.Interop.JniEnvironment.InstanceMethods.CallObjectMethod(JniObjectReference , JniMethodInfo , JniArgumentValue* ) in Java.Interop.dll:token 0x6000326+0x6e
   E NUnit   :    at Android.Runtime.JNIEnv.CallObjectMethod(IntPtr , IntPtr , JValue* ) in Mono.Android.dll:token 0x6000a05+0xe
   E NUnit   :    at Android.Content.Res.IXmlResourceParserInvoker.GetNamespace(String ) in Mono.Android.dll:token 0x60007de+0x52
   E NUnit   :    at Android.Runtime.XmlPullParserReader..ctor(IJavaObject ) in Mono.Android.dll:token 0x6000ac8+0x3f
   E NUnit   :    at Android.Runtime.XmlResourceParserReader..ctor(IJavaObject ) in Mono.Android.dll:token 0x6000ac4+0x0
   E NUnit   :    at Android.Runtime.XmlResourceParserReader.FromNative(IntPtr , JniHandleOwnership ) in Mono.Android.dll:token 0x6000ac7+0x3b
   E NUnit   :    at Android.Runtime.XmlResourceParserReader.FromJniHandle(IntPtr , JniHandleOwnership ) in Mono.Android.dll:token 0x6000ac6+0x0
   E NUnit   :    at Android.Content.Res.Resources.GetXml(Int32 ) in Mono.Android.dll:token 0x60007fc+0x36
   E NUnit   :    at Android.RuntimeTests.XmlReaderPullParserTest.ToLocalJniHandle() in Mono.Android.NET-Tests.dll:token 0x60000f3+0xa
   E NUnit   :    at System.Reflection.RuntimeMethodInfo.Invoke(Object , BindingFlags , Binder , Object[] , CultureInfo ) in System.Private.CoreLib.dll:token 0x60027b0+0x6a
   E NUnit   :   --- End of managed Java.Lang.RuntimeException stack trace ---
   E NUnit   : java.lang.RuntimeException: getNamespace() not supported
   E NUnit   : 	at android.content.res.XmlBlock$Parser.getNamespace(XmlBlock.java:143)
   E NUnit   : 	at crc643df67da7b13bb6b1.TestInstrumentation_1.n_onStart(Native Method)
   E NUnit   : 	at crc643df67da7b13bb6b1.TestInstrumentation_1.onStart(Unknown Source:0)
   E NUnit   : 	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2205)
   

Of particular note – revisited below – is the call stack: unless some form of inlining or something is going on, that call stack isn’t possible: the XmlPullParserReader(IJavaObject) constructor doesn’t call .GetNamespace(). See the Other Information section for additional musings.

Configuration

• Which version of .NET is the code running on? dotnet/runtime@0605bb3

• What OS and version, and what distro if applicable? Android API-29 Emulator, as created by:

    avdmanager create avd --abi x86_64 --force --name XamarinAndroidTestRunner29-x86_64 --package "system-images;android-29;default;x86_64" --device pixel_4
  

• What is the architecture (x64, x86, ARM, ARM64)? x64

• Do you know whether it is specific to that configuration? No

Regression?

Yes. This was previously working with dotnet/runtime@02f70d0; see also xamarin/xamarin-android@d593b727930ce22d78f13966bc3aa4ddd54614db.

Other information

The original failing build may disappear, and someone will want access to the assemblies used in the test. Assemblies within the .apk are compressed. Here are the assemblies from the .apk, uncompressed: assemblies-uncompressed.zip

The unit test: https://github.com/xamarin/xamarin-android/blob/7c5fab13329ee898fb1562f83576a7ca881f2881/tests/Mono.Android-Tests/Android.Runtime/XmlReaderPullParserTest.cs#L18

Which calls:

• Android.Content.Res.Resources.GetXml(), which is generated code:

  public virtual unsafe System.Xml.XmlReader GetXml (int id)
  {
  	const string __id = "getXml.(I)Landroid/content/res/XmlResourceParser;";
  	try {
  		JniArgumentValue* __args = stackalloc JniArgumentValue [1];
  		__args [0] = new JniArgumentValue (id);
  		var __rm = _members.InstanceMethods.InvokeVirtualObjectMethod (__id, this, __args);
  		return Android.Runtime.XmlResourceParserReader.FromJniHandle (__rm.Handle, JniHandleOwnership.TransferLocalRef)!;
  	} finally {
  	}
  }
  

• XmlResourceParserReader.FromJniHandle() & XmlResourceParserReader.FromNative(): https://github.com/xamarin/xamarin-android/blob/7c5fab13329ee898fb1562f83576a7ca881f2881/src/Mono.Android/Android.Runtime/XmlPullParserReader.cs#L29-L44

• XmlResourceParserReader(IJavaObject) constructor: https://github.com/xamarin/xamarin-android/blob/7c5fab13329ee898fb1562f83576a7ca881f2881/src/Mono.Android/Android.Runtime/XmlPullParserReader.cs#L17-L21

• XmlPullParserReader(IJavaObject) constructor: https://github.com/xamarin/xamarin-android/blob/7c5fab13329ee898fb1562f83576a7ca881f2881/src/Mono.Android/Android.Runtime/XmlPullParserReader.cs#L78-L83

This is where things get weird: in particular, note the body of the XmlPullParserReader(IJavaObject) constructor:

public XmlPullParserReader (IJavaObject source)
{
	this.source = (IXmlPullParser) source;
	supports_ns = this.source.GetFeature (XmlPullParser.FeatureProcessNamespaces);
	supports_ns_report_as_attr = this.source.GetFeature (XmlPullParser.FeatureReportNamespaceAttributes);
}

Note that the XmlPullParserReader(IJavaObject) constructor invokes a .GetFeature() method.

Recall again the stack trace in the failure:

E NUnit   :    at Android.Content.Res.IXmlResourceParserInvoker.GetNamespace(String ) in Mono.Android.dll:token 0x60007de+0x52
E NUnit   :    at Android.Runtime.XmlPullParserReader..ctor(IJavaObject ) in Mono.Android.dll:token 0x6000ac8+0x3f
E NUnit   :    at Android.Runtime.XmlResourceParserReader..ctor(IJavaObject ) in Mono.Android.dll:token 0x6000ac4+0x0

Where is Android.Content.Res.IXmlResourceParserInvoker.GetNamespace(string) coming from? There is no .GetNamespace() invocation, just .GetFeature().

(Also note that this test passes when using the interpreter. Mono.Android.NET_Tests-Interpreter-Signed.zip Some of the assemblies within the interpreter-based .apk differ from those in the JIT-based .apk, but not Mono.Android.dll.)

“Fortunately”, this is a linked app, so we can “just” disassemble Mono.Android.dll and find all occurrences of ::GetNamespace:

$ ikdasm Mono.Android.dll > Mono.Android.il
$ grep 'call.*::GetNamespace\>' Mono.Android.il
    IL_0011:  callvirt   instance string Org.XmlPull.V1.IXmlPullParser::GetNamespace(string)
    IL_0022:  callvirt   instance string Org.XmlPull.V1.IXmlPullParser::GetNamespace(string

One of those call sites is from XmlPullParserReader::GetAttribute(string, string). The other is from IXmlPullParserInvoker::n_GetNamespace_Ljava_lang_String_().

n_-prefixed methods are “JNI Marshal Methods”; they’re called “by” Java, not by managed code. The only “normal” way it could appear is if Java were a calling stack frame. This is possible, but doesn’t feel particularly plausible in this situation.

Which leaves XmlPullParserReader::GetAttribute(string, string), which only has two callers: XmlPullParserReader::GetAttribute(string) and XmlReaderPullParser::GetAttributeValue(string, string).

I can’t easily reach a .GetNamespace() method from the ::GetAttribute() methods. (Maybe I gave up too soon?)

Thus, my conjecture: “something broke” in the JIT (https://github.com/dotnet/runtime/compare/02f70d0...0605bb3) which causes the post-linked IXmlPullParser::GetFeature(string) invocation to instead invoke a GetNamespace() method.

Unfortunately, nothing jumps out to me in the src/mono changes:

$ git log -p 02f70d0...0605bb3 src/mono

(Linker change could also be in play, but since the post-linked IL for XmlPullParserReader::.ctor() continues to contain a method invocation to IXmlPullParser::GetFeature(string), this seems less likely.)