netty: native-image compilation of epoll transport fails

Expected behavior

My small repro compiles as a native image and works as expected if I use Nio* family of transports. My expectation was that switching to epoll transport would similarly work.

Actual behavior

If I switch to epoll (linux-x86_64) I get a compilation error:

Error: Classes that should be initialized at run time got initialized during image building:
 io.netty.util.AbstractReferenceCounted the class was requested to be initialized at build time (from the command line). io.netty.util.AbstractReferenceCounted has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked. Try avoiding to initialize the class that caused initialization of io.netty.util.AbstractReferenceCounted

com.oracle.svm.core.util.UserError$UserException: Classes that should be initialized at run time got initialized during image building:
 io.netty.util.AbstractReferenceCounted the class was requested to be initialized at build time (from the command line). io.netty.util.AbstractReferenceCounted has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked. Try avoiding to initialize the class that caused initialization of io.netty.util.AbstractReferenceCounted

	at com.oracle.svm.core.util.UserError.abort(UserError.java:65)
	at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.checkDelayedInitialization(ConfigurableClassInitialization.java:510)
	at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.duringAnalysis(ClassInitializationFeature.java:187)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$8(NativeImageGenerator.java:710)
	at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:63)
	at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:710)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:530)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:445)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)

Minimal yet complete reproducer code (or URL to code)

Here’s a repro: https://github.com/perezd/netty-epoll-native-repro

w/ Bazel 3.5.0 installed, run this command from the root of the project:

bazel run :main-native

Netty version

4.1.52.Final (linux-x86_64)

JVM version (e.g. `java -version`)

java -version openjdk version “11.0.8” 2020-07-14 OpenJDK Runtime Environment (build 11.0.8+10-post-Ubuntu-0ubuntu120.04) OpenJDK 64-Bit Server VM (build 11.0.8+10-post-Ubuntu-0ubuntu120.04, mixed mode, sharing)

OS version (e.g. `uname -a`)

Linux desktop 5.4.0-48-generic #52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

I’ve searched around and attempted various workarounds but nothing seems to work.

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 1
Comments: 19 (14 by maintainers)

Commits related to this issue

Initialise epoll at runtime in native images Motivation: People expect Netty to just be Netty when building native images, and so try to use the epoll transport. Modification: Add a native-image.pro... — committed to chrisvest/netty by chrisvest 3 years ago
Initialise epoll at runtime in native images Motivation: People expect Netty to just be Netty when building native images, and so try to use the epoll transport. Modification: Add a native-image.pro... — committed to chrisvest/netty by chrisvest 3 years ago
Initialise epoll at runtime in native images Motivation: People expect Netty to just be Netty when building native images, and so try to use the epoll transport. Modification: Add a native-image.pro... — committed to chrisvest/netty by chrisvest 3 years ago
Initialise epoll at runtime in native images Motivation: People expect Netty to just be Netty when building native images, and so try to use the epoll transport. Modification: Add a native-image.pro... — committed to chrisvest/netty by chrisvest 3 years ago
Add necessary native-image configuration files for epoll (#13158) Motivation: Epoll currently doesn't work out of the box on native-image without passing in additional flags and metadata. Modif... — committed to netty/netty by gradinac a year ago

Most upvoted comments

I also ran into this again with Netty 4.1.70 and GraalVM 21.3.0 but was able to get it working with:

  --initialize-at-run-time=io.netty.channel.DefaultFileRegion
  --initialize-at-run-time=io.netty.channel.epoll.Native
  --initialize-at-run-time=io.netty.channel.epoll.Epoll
  --initialize-at-run-time=io.netty.channel.epoll.EpollEventLoop
  --initialize-at-run-time=io.netty.channel.epoll.EpollEventArray
  --initialize-at-run-time=io.netty.channel.kqueue.KQueue
  --initialize-at-run-time=io.netty.channel.kqueue.KQueueEventLoop
  --initialize-at-run-time=io.netty.channel.kqueue.KQueueEventArray
  --initialize-at-run-time=io.netty.channel.kqueue.Native
  --initialize-at-run-time=io.netty.channel.unix.Limits
  --initialize-at-run-time=io.netty.channel.unix.Errors
  --initialize-at-run-time=io.netty.channel.unix.IovArray

jamesward on Dec 18, 2021

I tinkered a bit with epoll and GraalVM 23.0 - I managed to build the original sample project, but I couldn’t figure out how to tweak Bazel to use a newer GraalVM version - I’ve created the following sample repository that uses Gradle: https://github.com/gradinac/netty-epoll-native-image. This project uses the original reproducer from @perezd, but uses Gradle as the build system

I’ve hit two different categories of issues:

Class initialization problems - Netty 4.1 is by default fully initialized at image build time. However, epoll classes require native code and shouldn’t be initialized at image build time. On top of that, these classes also use other classes in their static initializer that shouldn’t be used during the image build (example: io.netty.util.AbstractReferenceCounted). Thank you @jamesward for the above list! 😃 It works without any modifications, but I’ve modified it a little bit in the repo. A sidenote: for Gradle and Maven projects, users can also add the Native Build Tools to their project and enable the metadata repository. Netty 4.1 metadata in this repository will automatically disable build-time initialization of Netty
Missing metadata - it appears we are missing some metadata for epoll, especially for JNI. Netty also appears to have many ways to fetch the native transport library, out of which one is to fetch it as a Java resource and extract it under /tmp at runtime. That approach seems to work fine, so I’ve also added a resource-config.json that includes this library if epoll is used.

If it’s okay, I’ll create a PR that adds the flags and the metadata to Netty 4.1

gradinac on Jan 26, 2023