opencv: Cannot load onnx (opset 12) of Resnet + MobileVit

I’m testing a semantic segmentation model made from :

https://github.com/qubvel/segmentation_models.pytorch for the Resnet backbone/decoder
https://github.com/rwightman/pytorch-image-models for mobileVit encoder

I used to perform inference with OpenCV using a different encoder (resnet) without any problem. However switching to MobileVit breaks the onnx loading. I’ve run opencv_model_diagnostics.exe (from build 4.6.0) and it returns many errors:

[ERROR:0@0.135] global \onnx_importer.cpp (1000) cv::dnn::dnn4_v20220524::ONNXImporter::handleNode DNN/ONNX: Potential problem during processing node with 1 inputs and 1 outputs: [Identity]:(onnx_node!Identity_0) from domain='ai.onnx'
OpenCV(4.6.0) \src\layer.cpp:246: error: (-215:Assertion failed) inputs.size() in function 'cv::dnn::dnn4_v20220524::Layer::getMemoryShapes'

[ERROR:0@0.142] global \onnx_importer.cpp (1000) cv::dnn::dnn4_v20220524::ONNXImporter::handleNode DNN/ONNX: Potential problem during processing node with 2 inputs and 1 outputs: [Reshape]:(onnx_node!Reshape_121) from domain='ai.onnx'
OpenCV(4.6.0) \onnx_importer.cpp:571: error: (-5:Bad argument) Blob onnx::Reshape_1114 not found in const blobs in function 'cv::dnn::dnn4_v20220524::ONNXImporter::getBlob'

[ERROR:0@0.142] global \onnx_importer.cpp (1000) cv::dnn::dnn4_v20220524::ONNXImporter::handleNode DNN/ONNX: Potential problem during processing node with 1 inputs and 1 outputs: [Transpose]:(onnx_node!Transpose_122) from domain='ai.onnx'
OpenCV(4.6.0) \src\layers\permute_layer.cpp:161: error: (-215:Assertion failed) (int)_numAxes == inputs[0].size() in function 'cv::dnn::PermuteLayerImpl::getMemoryShapes'

[ERROR:0@0.142] global \onnx_importer.cpp (1000) cv::dnn::dnn4_v20220524::ONNXImporter::handleNode DNN/ONNX: Potential problem during processing node with 1 inputs and 1 outputs: [Transpose]:(onnx_node!Transpose_127) from domain='ai.onnx'
OpenCV(4.6.0) \src\layers\permute_layer.cpp:161: error: (-215:Assertion failed) (int)_numAxes == inputs[0].size() in function 'cv::dnn::PermuteLayerImpl::getMemoryShapes'

[ERROR:0@0.142] global \onnx_importer.cpp (1000) cv::dnn::dnn4_v20220524::ONNXImporter::handleNode DNN/ONNX: Potential problem during processing node with 2 inputs and 1 outputs: [MatMul]:(onnx_node!MatMul_128) from domain='ai.onnx'
OpenCV(4.6.0) \src\layers\fully_connected_layer.cpp:129: error: (-2:Unspecified error) in function 'bool __cdecl cv::dnn::FullyConnectedLayerImpl::getMemoryShapes(const class std::vector<class std::vector<int,class std::allocator<int> >,class std::allocator<class std::vector<int,class std::allocator<int> > > > &,const int,class std::vector<class std::vector<int,class std::allocator<int> >,class std::allocator<class std::vector<int,class std::allocator<int> > > > &,class std::vector<class std::vector<int,class std::allocator<int> >,class std::allocator<class std::vector<int,class std::allocator<int> > > > &) const'
>  (expected: 'inputs[0].back() == inputs[1][dims - 2]'), where
>     'inputs[0].back()' is 288
> must be equal to
>     'inputs[1][dims - 2]' is 256

[ERROR:0@0.143] global \onnx_importer.cpp (1000) cv::dnn::dnn4_v20220524::ONNXImporter::handleNode DNN/ONNX: Potential problem during processing node with 1 inputs and 1 outputs: [Softmax]:(onnx_node!Softmax_131) from domain='ai.onnx'
OpenCV(4.6.0) \include\opencv2\dnn\shape_utils.hpp:216: error: (-2:Unspecified error) in function 'int __cdecl cv::dnn::dnn4_v20220524::normalize_axis(int,int)'
> :
>     'axis >= -dims && axis < dims'
> where
>     'axis' is 3

Is it a matter of opset incompatibility or unsupported layers?

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 20 (20 by maintainers)

Most upvoted comments

Is there any ongoing work add support of dynamic shapes to operators like resize?

We are working on the next generation OpenCV DNN, but it will only be available since OpenCV 5.

Can we workaround this by using different equivalent operator in the model definition?

It depends on the model.

fengyuentau on Sep 2, 2022

Do you mean dynamic shape is not supported for reshape (and a subset of other layers?) Because until today I was inferring simpler models with dynamic shapes without any problem

I would like to add some words to make it clear about the dynamic shape thing in OpenCV DNN. So in OpenCV DNN, each layer is initialized with memory allocated based on the input & output shape during the model importing. For an ONNX model with fixed input shape, if the actual input image has the same shape, inference should finish without issues; if the actual input image has different shape, memory will be re-allocated based on that shape in inference, and you still have a chance successfully finishing the inference if there is no operator like resize which may always generating an output of fixed shape regardless the input.

fengyuentau on Sep 2, 2022

Hi @JulienMaille, thanks for your feedback. I just test the model on my site, it works fine both in readNet and net.forward(). What is your input shape? 1x1x256x256 works fine. And 2x1x256x256 can not work as expected.

The right way to checkout the branch is the following:

Goto your opencv root directory:

git add fengyuentau https://github.com/fengyuentau/opencv.git
git fetch
git checkout fengyuentau/naive_yolov7

Then re-compile OpenCV.

Issue related models [pending]

zihaomu on Sep 1, 2022

Thanks. I’m working on it.

zihaomu on Aug 31, 2022