onnxruntime: Problems with predictions on MacBook Air with M1 chip in Java project based on Maven (even on onnxruntime 1.11.0 - latest at this moment) - no osx-aarch64 dir in the native runtimes dir

Hi,

I reported a bug in #10874 regarding the fact that onnxruntime does not work on MacBook Air with M1 chip in Java project based on Maven. There is also information that this bug is fixed in PR #10981 I upgraded dependency to onnxruntime to version 1.11.0 and I still have exactly the same bug as I reported in that issue regardless of that upgrade.

Calling a function:

final OrtEnvironment environment = OrtEnvironment.getEnvironment();

causes an error:

java.lang.NoClassDefFoundError: Could not initialize class ai.onnxruntime.OrtEnvironment

I see that in project dependencies there is:

/onnxruntime/1.11.0/onnxruntime-1.11.0.jar!/ai/onnxruntime/native/osx-x64

but error occurs anyway

Moreover, when I execute a single test instead of my whole suite, then I get an error:

java.lang.UnsatisfiedLinkError: no onnxruntime in java.library.path

Probably, the source of the bug is here:

https://github.com/microsoft/onnxruntime/blob/74eaaad768e0a5ad5e65b818605383c73dece509/java/src/main/java/ai/onnxruntime/OnnxRuntime.java#L103

in the method initOsArch(). I debugged it and in my case I see the following params:

  • detectedOs = osx
  • arch = aarch64

final result of os arch: osx-aarch64

and then in init() method:

libraryDirPathProperty = System.getProperty(ONNXRUNTIME_NATIVE_PATH)

libraryDirPathProperty is null where ONNXRUNTIME_NATIVE_PATH is onnxruntime.native.path

Next, in extractFromResources(String library) method, resourcePath is:

/ai/onnxruntime/native/osx-aarch64/libonnxruntime_providers_shared.dylib

and there is no such path.

In the path ai/onnxruntime/native there are only the following dirs:

  • linux-x64
  • osx-x64
  • win-x64

but there is no osx-aarch64, which is detected as my arch on MacBook Air with M1 chip

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 34 (13 by maintainers)

Most upvoted comments

I meant a new version - not updating the current one, but ok, I’ll wait for the release.

I believe ORT is on a quarterly release schedule for feature releases.

@faxu any chance of Java macOS/arm64 binaries (#10981) landing in a 1.11 point release?

Targeting 1.12 for including this build in the Maven pkg.

@russellrc-keebo yes, it’s fixed by #12335. In the version 1.12.1 this problem no longer occurs.

I’d be careful running ONNX Runtime under Rosetta on an M1. I’m not sure if the detection in MLAS assumes all x86 CPUs have AVX instructions or not. If it fails the detection and uses an AVX instruction these are not emulated by Rosetta and can cause an illegal instruction error which will crash the JVM if one is executed under it.

Thanks for the reply @faxu. Can you tell when can I expect 1.12 release?

We just released 1.11, so it will be a few months.

I think it’ll land in the next release whenever that comes, but 1.11.0 will not be updated.