onnxruntime: [ORT JNI] OpenVINO EP cannot be added into session options

Describe the bug Hi team, our team is exploring the performance brought up by OpenVINO EP in ORT JNI solution. We have a local build pipeline that generates the ORT JNI library by the following shell:

./build.sh --config Release --build_shared_lib --use_openvino CPU_FP32 --skip_submodule_sync --parallel --build_java --build_dir /workspace/build

When running with the following config

try (OrtSession.SessionOptions opts = new OrtSession.SessionOptions()) {
          opts.setSessionLogLevel(OrtLoggingLevel.ORT_LOGGING_LEVEL_ERROR);
          opts.addOpenVINO("");
          OrtSession ortSession = _ortEnvironment.createSession(modelLocation.getPath(), opts);
          return ortSession;
        }

and run the ORT session for inference, it gives following error

/export/apps/jdk/JDK-11_0_8-msft/bin/java: symbol lookup error: /tmp/onnxruntime-java2533679640159920784/libonnxruntime4j_jni.so: undefined symbol: OrtSessionOptionsAppendExecutionProvider_OpenVINO

Seems that some Symbol is missing in the JNI C++ code.

When removing the opts.addOpenVINO(""); it works as expected.

Do you have any insights on this issue?

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): CentOS7
  • ONNX Runtime installed from (source or binary): source
  • ONNX Runtime version: 1.8.0
  • Python version: N/A
  • Visual Studio version (if applicable): N/A
  • GCC/Compiler version (if compiling from source): 7.3.1
  • CUDA/cuDNN version: N/A
  • GPU model and memory: N/A

To Reproduce Same as the description above.

Screenshots N/A

Additional context Add any other context about the problem here. If the issue is about a particular model, please share the model details as well to facilitate debugging.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 21 (11 by maintainers)

Most upvoted comments

Ok, I think that means you’ve not fully configured your OpenVINO environment, as it adds a bunch of libraries to your library path which is what ORT’s OpenVINO executor is looking for. I think you need to make sure this script is executed in your terminal (https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html#set-the-environment-variables), but I don’t have access to an OpenVINO installation to test it with.

Synced offline with Maajid, the OpenVINO EP works with default EP by splitting the model into subgraph and merge the results together. Currently we are meeting issue with the OpenVINO EP even with C++ APIs. (same issue here https://github.com/openvinotoolkit/openvino/issues/5205) . Therefore I will temporarily close this ticket and reopen it once we got a test model working with OpenVINO EP C++ APIs.

Agreed. Will try to configure OpenVINO in the test host machine. Thanks very much!