quarkus: Native Image Build Failures

Describe the bug I am attempting to do a native build of an AWS Lambda function in a Maven project. The build fails with the messages shown in Actual Behavior. There appear to be unsupported features in 3 methods:

• A detected MBean server in the image heap (similar to https://github.com/oracle/graal/issues/2669, not sure if this was ever confirmed fixed. I also attempted to add com.sun.jmx.mbeanserver.JmxMBeanServer to initialize-at-runtime and the behavior changed the same was as mentioned in that issue.) I can open a separate issue with oracle/graal if the root cause is there.
• com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved method during parsing: org.joda.time.Interval.parseWithOffset(java.lang.String) (similar to https://stackoverflow.com/questions/63959478/build-failure-with-quarkus and https://github.com/quarkusio/quarkus/issues/12592)
• type is not available in this platform: org.graalvm.compiler.hotspot.management.AggregatedMemoryPoolBean (similar to https://github.com/quarkusio/quarkus/issues/11923)

Expected behavior Native build succeeds, possibly after additional/missing POM modifications.

Actual behavior The build fails with the following output:

...
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.amazonaws.util.XpathUtils (file:[redacted path]/ce-controller/target/ce-controller-release_13_0-native-image-source-jar/lib/com.amazonaws.aws-java-sdk-core-1.11.711.jar) to method com.sun.org.apache.xpath.internal.XPathContext.getDTMManager()
WARNING: Please consider reporting this to the maintainers of com.amazonaws.util.XpathUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
18:59:17,959 INFO  [org.jbo.threads] JBoss Threads version 3.1.1.Final
[ce-controller-release_13_0-runner:52983]     (clinit):     630.25 ms,  4.75 GB
[ce-controller-release_13_0-runner:52983]   (typeflow):  16,427.15 ms,  4.75 GB
[ce-controller-release_13_0-runner:52983]    (objects):  15,836.66 ms,  4.75 GB
[ce-controller-release_13_0-runner:52983]   (features):   1,016.43 ms,  4.75 GB
[ce-controller-release_13_0-runner:52983]     analysis:  35,826.82 ms,  4.75 GB
Error: Unsupported features in 3 methods
Detailed message:
Error: Detected a MBean server in the image heap. This is currently not supported, but could be changed in the future. Management beans are registered in many global caches that would need to be cleared and properly re-built at image build time. Class of disallowed object: com.sun.jmx.mbeanserver.JmxMBeanServer  Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image run time by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Trace: Object was reached by 
        indexing into array
                constant java.lang.Object[]@51bf57fd reached by 
        reading field java.util.ArrayList.elementData of
                constant java.util.ArrayList@63940761 reached by 
        scanning method javax.management.MBeanServerFactory.addMBeanServer(MBeanServerFactory.java:418)
Call path from entry point to javax.management.MBeanServerFactory.addMBeanServer(MBeanServer): 
        at javax.management.MBeanServerFactory.addMBeanServer(MBeanServerFactory.java:418)
        at javax.management.MBeanServerFactory.createMBeanServer(MBeanServerFactory.java:232)
        at javax.management.MBeanServerFactory.createMBeanServer(MBeanServerFactory.java:192)
        at com.oracle.svm.core.jdk.management.ManagementSupport.getPlatformMBeanServer(ManagementSupport.java:250)
        at com.oracle.svm.core.jdk.management.Target_java_lang_management_ManagementFactory.getPlatformMBeanServer(Target_java_lang_management_ManagementFactory.java:45)
        at org.apache.commons.pool2.impl.BaseGenericObjectPool.jmxUnregister(BaseGenericObjectPool.java:946)
        at org.apache.commons.pool2.impl.GenericObjectPool.close(GenericObjectPool.java:705)
        at redis.clients.util.Pool.closeInternalPool(Pool.java:109)
        at redis.clients.util.Pool.destroy(Pool.java:96)
        at redis.clients.util.Pool.close(Pool.java:28)
        at com.oracle.svm.core.posix.NativeSecureRandomFilesCloser$1.close(PosixSunSecuritySubstitutions.java:82)
        at com.oracle.svm.core.posix.NativeSecureRandomFilesCloser$1.run(PosixSunSecuritySubstitutions.java:66)
        at com.oracle.svm.core.jdk.RuntimeSupport.executeHooks(RuntimeSupport.java:125)
        at com.oracle.svm.core.jdk.RuntimeSupport.executeStartupHooks(RuntimeSupport.java:75)
        at com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:141)
        at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:184)
        at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved method during parsing: org.joda.time.Interval.parseWithOffset(java.lang.String). To diagnose the issue you can use the --allow-incomplete-classpath option. The missing method is then reported at run time when it is accessed the first time.
Trace: 
        at parsing com.fasterxml.jackson.datatype.joda.deser.IntervalDeserializer.deserialize(IntervalDeserializer.java:62)
Call path from entry point to com.fasterxml.jackson.datatype.joda.deser.IntervalDeserializer.deserialize(JsonParser, DeserializationContext): 
        at com.fasterxml.jackson.datatype.joda.deser.IntervalDeserializer.deserialize(IntervalDeserializer.java:37)
        at com.fasterxml.jackson.datatype.joda.deser.IntervalDeserializer.deserialize(IntervalDeserializer.java:15)
        at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2079)
        at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1453)
        at io.quarkus.amazon.lambda.runtime.AbstractLambdaPollLoop$1.run(AbstractLambdaPollLoop.java:78)
        at java.lang.Thread.run(Thread.java:834)
        at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:517)
        at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:192)
        at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)
Error: type is not available in this platform: org.graalvm.compiler.hotspot.management.AggregatedMemoryPoolBean
Trace: Object was reached by 
        reading field java.lang.ref.Reference.referent of
                constant java.util.WeakHashMap$Entry@2f8386db reached by 
        indexing into array
                constant java.util.WeakHashMap$Entry[]@3595ff42 reached by 
        reading field java.util.WeakHashMap.table of
                constant com.sun.jmx.mbeanserver.MBeanIntrospector$MBeanInfoMap@5e7699b1 reached by 
        scanning method com.sun.jmx.mbeanserver.MXBeanIntrospector.getMBeanInfoMap(MXBeanIntrospector.java:68)
Call path from entry point to com.sun.jmx.mbeanserver.MXBeanIntrospector.getMBeanInfoMap(): 
        at com.sun.jmx.mbeanserver.MXBeanIntrospector.getMBeanInfoMap(MXBeanIntrospector.java:68)
        at com.sun.jmx.mbeanserver.MBeanIntrospector.getClassMBeanInfo(MBeanIntrospector.java:416)
        at com.sun.jmx.mbeanserver.MBeanIntrospector.getMBeanInfo(MBeanIntrospector.java:392)
        at com.sun.jmx.mbeanserver.MBeanSupport.<init>(MBeanSupport.java:139)
        at com.sun.jmx.mbeanserver.MXBeanSupport.<init>(MXBeanSupport.java:66)
        at com.sun.jmx.mbeanserver.Introspector.makeDynamicMBean(Introspector.java:203)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:888)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320)
        at com.sun.jmx.mbeanserver.JmxMBeanServer$2.run(JmxMBeanServer.java:1225)
        at com.oracle.svm.core.jdk.Target_java_security_AccessController.doPrivileged(SecuritySubstitutions.java:119)
        at java.net.Socket.getOutputStream(Socket.java:969)
        at org.jboss.logmanager.handlers.TcpOutputStream.flush(TcpOutputStream.java:214)
        at java.io.PrintStream.flush(PrintStream.java:417)
        at com.oracle.svm.jni.functions.JNIFunctions.ExceptionDescribe(JNIFunctions.java:772)
        at com.oracle.svm.core.code.IsolateEnterStub.JNIFunctions_ExceptionDescribe_b5412f7570bccae90b000bc37855f00408b2ad73(generated:0)

com.oracle.svm.core.util.UserError$UserException: Unsupported features in 3 methods
Detailed message:
Error: Detected a MBean server in the image heap. This is currently not supported, but could be changed in the future. Management beans are registered in many global caches that would need to be cleared and properly re-built at image build time. Class of disallowed object: com.sun.jmx.mbeanserver.JmxMBeanServer  Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image run time by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Trace: Object was reached by 
        indexing into array
                constant java.lang.Object[]@51bf57fd reached by 
        reading field java.util.ArrayList.elementData of
                constant java.util.ArrayList@63940761 reached by 
        scanning method javax.management.MBeanServerFactory.addMBeanServer(MBeanServerFactory.java:418)
Call path from entry point to javax.management.MBeanServerFactory.addMBeanServer(MBeanServer): 
        at javax.management.MBeanServerFactory.addMBeanServer(MBeanServerFactory.java:418)
        at javax.management.MBeanServerFactory.createMBeanServer(MBeanServerFactory.java:232)
        at javax.management.MBeanServerFactory.createMBeanServer(MBeanServerFactory.java:192)
        at com.oracle.svm.core.jdk.management.ManagementSupport.getPlatformMBeanServer(ManagementSupport.java:250)
        at com.oracle.svm.core.jdk.management.Target_java_lang_management_ManagementFactory.getPlatformMBeanServer(Target_java_lang_management_ManagementFactory.java:45)
        at org.apache.commons.pool2.impl.BaseGenericObjectPool.jmxUnregister(BaseGenericObjectPool.java:946)
        at org.apache.commons.pool2.impl.GenericObjectPool.close(GenericObjectPool.java:705)
        at redis.clients.util.Pool.closeInternalPool(Pool.java:109)
        at redis.clients.util.Pool.destroy(Pool.java:96)
        at redis.clients.util.Pool.close(Pool.java:28)
        at com.oracle.svm.core.posix.NativeSecureRandomFilesCloser$1.close(PosixSunSecuritySubstitutions.java:82)
        at com.oracle.svm.core.posix.NativeSecureRandomFilesCloser$1.run(PosixSunSecuritySubstitutions.java:66)
        at com.oracle.svm.core.jdk.RuntimeSupport.executeHooks(RuntimeSupport.java:125)
        at com.oracle.svm.core.jdk.RuntimeSupport.executeStartupHooks(RuntimeSupport.java:75)
        at com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:141)
        at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:184)
        at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved method during parsing: org.joda.time.Interval.parseWithOffset(java.lang.String). To diagnose the issue you can use the --allow-incomplete-classpath option. The missing method is then reported at run time when it is accessed the first time.
Trace: 
        at parsing com.fasterxml.jackson.datatype.joda.deser.IntervalDeserializer.deserialize(IntervalDeserializer.java:62)
Call path from entry point to com.fasterxml.jackson.datatype.joda.deser.IntervalDeserializer.deserialize(JsonParser, DeserializationContext): 
        at com.fasterxml.jackson.datatype.joda.deser.IntervalDeserializer.deserialize(IntervalDeserializer.java:37)
        at com.fasterxml.jackson.datatype.joda.deser.IntervalDeserializer.deserialize(IntervalDeserializer.java:15)
        at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2079)
        at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1453)
        at io.quarkus.amazon.lambda.runtime.AbstractLambdaPollLoop$1.run(AbstractLambdaPollLoop.java:78)
        at java.lang.Thread.run(Thread.java:834)
        at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:517)
        at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:192)
        at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)
Error: type is not available in this platform: org.graalvm.compiler.hotspot.management.AggregatedMemoryPoolBean
Trace: Object was reached by 
        reading field java.lang.ref.Reference.referent of
                constant java.util.WeakHashMap$Entry@2f8386db reached by 
        indexing into array
                constant java.util.WeakHashMap$Entry[]@3595ff42 reached by 
        reading field java.util.WeakHashMap.table of
                constant com.sun.jmx.mbeanserver.MBeanIntrospector$MBeanInfoMap@5e7699b1 reached by 
        scanning method com.sun.jmx.mbeanserver.MXBeanIntrospector.getMBeanInfoMap(MXBeanIntrospector.java:68)
Call path from entry point to com.sun.jmx.mbeanserver.MXBeanIntrospector.getMBeanInfoMap(): 
        at com.sun.jmx.mbeanserver.MXBeanIntrospector.getMBeanInfoMap(MXBeanIntrospector.java:68)
        at com.sun.jmx.mbeanserver.MBeanIntrospector.getClassMBeanInfo(MBeanIntrospector.java:416)
        at com.sun.jmx.mbeanserver.MBeanIntrospector.getMBeanInfo(MBeanIntrospector.java:392)
        at com.sun.jmx.mbeanserver.MBeanSupport.<init>(MBeanSupport.java:139)
        at com.sun.jmx.mbeanserver.MXBeanSupport.<init>(MXBeanSupport.java:66)
        at com.sun.jmx.mbeanserver.Introspector.makeDynamicMBean(Introspector.java:203)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:888)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320)
        at com.sun.jmx.mbeanserver.JmxMBeanServer$2.run(JmxMBeanServer.java:1225)
        at com.oracle.svm.core.jdk.Target_java_security_AccessController.doPrivileged(SecuritySubstitutions.java:119)
        at java.net.Socket.getOutputStream(Socket.java:969)
        at org.jboss.logmanager.handlers.TcpOutputStream.flush(TcpOutputStream.java:214)
        at java.io.PrintStream.flush(PrintStream.java:417)
        at com.oracle.svm.jni.functions.JNIFunctions.ExceptionDescribe(JNIFunctions.java:772)
        at com.oracle.svm.core.code.IsolateEnterStub.JNIFunctions_ExceptionDescribe_b5412f7570bccae90b000bc37855f00408b2ad73(generated:0)

        at com.oracle.svm.core.util.UserError.abort(UserError.java:79)
        at com.oracle.svm.hosted.FallbackFeature.reportAsFallback(FallbackFeature.java:217)
        at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:765)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:555)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:468)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Unsupported features in 3 methods
...
...
Error: Image build request failed with exit status 1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:17 min
[INFO] Finished at: 2020-11-30T18:59:36-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:1.9.2.Final:native-image (default) on project ce-controller: Failed to generate native image: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR]         [error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#build threw an exception: java.lang.RuntimeException: Failed to build native image
[ERROR]         at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:307)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR]         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]         at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR]         at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:936)
[ERROR]         at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
[ERROR]         at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2046)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1578)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452)
[ERROR]         at java.base/java.lang.Thread.run(Thread.java:834)
[ERROR]         at org.jboss.threads.JBossThread.run(JBossThread.java:479)
[ERROR] Caused by: java.lang.RuntimeException: Image generation failed. Exit code: 1
[ERROR]         at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.imageGenerationFailed(NativeImageBuildStep.java:520)
[ERROR]         at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:276)
[ERROR]         ... 12 more
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

To Reproduce I can’t share the code at this time unfortunately.

Configuration Relevant POM profile section:

<profile>
            <id>native</id>
            <activation>
                <property>
                    <name>native</name>
                </property>
            </activation>
            <properties>
                <quarkus.native.additional-build-args>\
                    --initialize-at-run-time=[2 dependencies that were previously complaining of needing runtime initialization],\
                    -H:+TraceClassInitialization,\
                    -H:ResourceConfigurationFiles=resources-config.json
                </quarkus.native.additional-build-args>
            </properties>
            <build>
                <plugins>
                    <plugin>
                        <groupId>io.quarkus</groupId>
                        <artifactId>quarkus-maven-plugin</artifactId>
                        <version>${quarkus-plugin.version}</version>
                        <executions>
                            <execution>
                                <goals>
                                    <goal>native-image</goal>
                                </goals>
                                <configuration>
                                    <enableHttpUrlHandler>true</enableHttpUrlHandler>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>

Environment:

• (Partially Redacted) Output of uname -a: Darwin Kernel Version 19.6.0: Thu Oct 29 22:56:45 PDT 2020; root:xnu-6153.141.2.2~1/RELEASE_X86_64 x86_64
• Output of java -version:

openjdk version "11.0.8" 2020-07-14
OpenJDK Runtime Environment GraalVM CE 20.2.0 (build 11.0.8+10-jvmci-20.2-b03)
OpenJDK 64-Bit Server VM GraalVM CE 20.2.0 (build 11.0.8+10-jvmci-20.2-b03, mixed mode, sharing)

• GraalVM version (if different from Java): [Same]
• Quarkus version or git rev: 1.9.2.Final
• Build tool (output of mvn --version):

Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/Cellar/maven/3.6.3_1/libexec
Java version: 11.0.8, vendor: GraalVM Community, runtime: /Library/Java/JavaVirtualMachines/graalvm-ce-java11-20.2.0/Contents/Home
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.15.7", arch: "x86_64", family: "mac"