openvino: [Bug]: std::runtime_error when trying to read_model in C++

System information (version)

OpenVINO Source=> Runtime
OpenVINO Version=> 2023.0.1
Operating System / Platform => Windows 64 Bit
Compiler => Visual Studio 2019
Problem classification => Reading model
Device use: => CPU
Model name => YOLO-NAS-s

Detailed description

When I try to load the model in C++, be it the ONNX file or the XML file or the BIN file, I get a runtime_error.

Microsoft C++ exception: std::runtime_error at memory location 0x000000D6156FE068.

The model loads fine when I use Python.

I am following the official documentation for loading the model:

    ov::Core core;

    std::shared_ptr<ov::Model> model = core.read_model(model_path);  //exception here

The ONNX model file I am usung:

https://file.io/pwlkoUUVQuY2

Steps to reproduce

Issue submission checklist

I report the issue, it’s not a question
I checked the problem with documentation, FAQ, open issues, Stack Overflow, etc and have not found solution
There is reproducer code and related data files: images, videos, models, etc.

About this issue

Original URL
State: open
Created a year ago
Comments: 41 (10 by maintainers)

Most upvoted comments

@avitial You’re right. It does work. I was debugging using Visual Studio and had the exception breakpoints turned on, so it would stop whenever an exception was thrown. But if I disable and choose to ignore the exception, it still works. Tested on 2023.0.1.

These are the exceptions I get:

Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: std::runtime_error at memory location 0x000000EC5091DAF8.
'yolo-nas-openvino-cpp.exe' (Win32): Loaded 'C:\Program Files (x86)\Intel\openvino_3.0\runtime\bin\intel64\Debug\openvino_ir_frontendd.dll'. 
'yolo-nas-openvino-cpp.exe' (Win32): Loaded 'C:\Program Files (x86)\Intel\openvino_3.0\runtime\bin\intel64\Debug\openvino_auto_batch_plugind.dll'. 
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::GeneralError at memory location 0x000000EC5091C530.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::NotFound at memory location 0x000000EC50919D80.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::GeneralError at memory location 0x000000EC509174F0.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::GeneralError at memory location 0x000000EC509174F0.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::GeneralError at memory location 0x000000EC5091C340.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: std::runtime_error at memory location 0x000000EC5091ABC0.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: std::runtime_error at memory location 0x000000EC5091ABC0.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: [rethrow] at memory location 0x0000000000000000.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::GeneralError at memory location 0x000000EC50915B60.
'yolo-nas-openvino-cpp.exe' (Win32): Loaded 'C:\Program Files (x86)\Intel\openvino_3.0\runtime\bin\intel64\Debug\openvino_intel_cpu_plugind.dll'. 
'yolo-nas-openvino-cpp.exe' (Win32): Loaded 'C:\Program Files (x86)\Intel\openvino_3.0\runtime\3rdparty\tbb\bin\tbbbind_2_5_debug.dll'. 
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::GeneralError at memory location 0x000000EC5091C490.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::GeneralError at memory location 0x000000EC5091C2A0.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::NotImplemented at memory location 0x000000EC5091C910.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: [rethrow] at memory location 0x0000000000000000.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::NotImplemented at memory location 0x000000EC5091A0C0.
Exception thrown at 0x00007FF82FD32BDC in yolo-nas-openvino-cpp.exe: Microsoft C++ exception: InferenceEngine::GeneralError at memory location 0x000000EC5091D0D0.

Y-T-G on Sep 6, 2023

@avitial I get std::runtime_error exception in the core.read_model part when I try to do that in version 2022.3, 2023.0 and 2023.0.1. This only happens in C++. It’s not related to the performance. The model doesn’t load at all. It just throws that exception when I try to load it.

The same code however has no issues running in version 2022.1.

Y-T-G on Sep 2, 2023

Okay, I used the serialize function to save to xml and bin. Still throws the same error.

I converted by going from Pytorch > Torchscript > IR through convert_model using the latest Python package 2022.3.1, and tested in C++ using the latest 2022.3.1 dll files downloaded from the archive.

I couldn’t go from Pytorch > IR because the scripting function used by convert_model throws an error. I had to trace it instead.

Y-T-G on Aug 5, 2023

@Y-T-G the difference is what is the original file you feed to convert_model. If this is .onnx then result will be the same as described here. But what I actually ask is to try to put original torch.nn to convert_model and then save OV model as IR ( .bin + .xml). As input model would be PyTorch format then PyTorch FE will be used not ONNX FE and IR might be slightly different.

Just wanted to make sure if issue is specific to torch.onnx.export or something wrong with conversion to IR. Also if 2022.1 works fine it would be great to feed 2022.1 IR to 2023.0.1 runtime if possible to narrow down the faulty area

andrei-kochin on Aug 5, 2023

我同样遇到这个问题 I also encountered this problem. I tested both 2023.0.1 and 2022.3 versions for this issue！

you can use version 2022.1, it can work. Thank you, but this is not a long-term solution. Openvino developers should focus on addressing this issue

shining-love on Aug 3, 2023