openvino: Problems upgrading MO 2021.4->2023.0 with a custom layer

We convert one of the models from Caffe to OpenVino using MO. The approach had been to patch mo_caffe.py/caffe_pb.py, and deploy a FrontExtractorOp-derived class and Op-derived class.

It had been a lot of trying to feel the way in the dark, but we’re making progress:

The deployment location of Python code changed from site-packages/openvino/tools/extensions/front/caffe and site-packages/openvino/tools/extensions/ops to site-packages/openvino/tools/mo/front/caffe/proto.
CustomLayersMapping.xml had to be introduced into the process
(This was a fun one to figure out) If custom layer definition in CustomLayersMapping.xml included protoParamName, the Python layer code, while loaded, was never utilized. This resulted in Caffe fallback is deprecated/ Please register python infer function for op error. Removing this attribute resolved that issue. Still not sure why.
Current problem is the process failing during serialization. A warning Please set version attribute for node detection_out with type=Yolov3DetectionOutput is logged, followed by the error Cannot create Yolov3DetectionOutput layer detection_out id:490 from unsupported opset: extension. I’ve attempted to define version attribute in the Python layer with little success. Hoping someone can save me a day or two of code archeology and get over this bump.

(Probably not critical, but all of the above is with Intel distribution of OV / toolkit installed with pip3 install openvino-dev[caffe,onnx]==2023.0.0 on Linux)

About this issue

Original URL
State: closed
Created a year ago
Comments: 24 (15 by maintainers)

Most upvoted comments

Hi @w3sip, wish some minor patches i was able to solve Python part of the problem the main left problem is

Current problem is the process failing during serialization. A warning Please set version attribute for node detection_out with type=Yolov3DetectionOutput is logged, followed by the error Cannot create Yolov3DetectionOutput layer detection_out id:490 from unsupported opset: extension.

Changes i’ve made in reproducer: in convertToVino.sh yolov3detectionoutput.py should be copied into …/ops dir: cp $EXTDIR/yolov3detectionoutput.py $PROTODIR/../../../ops/ and corrected yolov3detectionoutput op version to 'version': 'extension'.

IR is generated in 2 steps: firstly python code parses caffe model and saves a _tmp IR https://github.com/openvinotoolkit/openvino/blob/d42b53c070b31ef3eefbebbfa92b6d5ba796bdf7/tools/mo/openvino/tools/mo/convert_impl.py#L485

after that it’s loaded into IE and offline transformations which are in c++ are called upon that model object https://github.com/openvinotoolkit/openvino/blob/d42b53c070b31ef3eefbebbfa92b6d5ba796bdf7/tools/mo/openvino/tools/mo/convert_impl.py#L498 so indeed there are temporary IRs (answer to @slyalin question 4.), and the problem currently (after patching python extensions ) is solely in c++ part.

I’ve talked to colleges, and they pointed me to the place where c++ changes should be made. I will keep you updated

pavel-esir on Aug 2, 2023

@w3sip thanks for the valuable comment! Delta 2021.4 vs 2021.4.1 is much smaller than vs 2023.0 =)

Let us check and get back to you with findings

andrei-kochin on Jul 18, 2023

@rkazants / @slyalin - sorry for the delay.

Here’s the package including the model - https://1drv.ms/u/s!Am36O7LweZ6zgf0bWQGQ6zwhVa92-g?e=js2lry … The only prerequisite is available Docker installation. Running reproduce.sh will create a basic container to run in, a virtualenv with OpenVino installed (thus first run takes awhile) – and will call ./vino/convertToVino.sh to attempt the conversion.

Please let me know if this is enough for you to reproduce the problem.

w3sip on Jun 20, 2023