openvino: [Bug] Custom ops with dynamic input/output shapes
System information (version)
-
OpenVINO=> build from master branch
-
Operating System / Platform => ubuntu18.04 on WSL
-
Problem classification => custom ops
Detailed description
I have an onnx model with custom ops, which have dynamic input/output shapes. When I test the model with simple ov extension refering to the doc of custom extension，an exception are thrown as like:
terminate called after throwing an instance of 'ov::Exception' what(): Shape inference of Reference node with name o2 failed: Check 'false' failed at src/plugins/intel_cpu/src/utils/shape_inference/shape_inference.cpp:255: Can't compute static output shape on 0 port for CustomTest node with name: CustomTest_81. Input shapes = ( 0 port = [?], ). Output shapes = ( 0 port = [?,?], 1 port = [?], )
My custom op is mostly like nms, which the output shapes are determined after the operation has been performed. By reading the source code of nms op, I’ve found it uses the InternalDynShapeInfer for this problem but have no idea how to solve it with custom ops. The example code and data are as follows
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 15 (8 by maintainers)
Hi @autumn110 , we are working on the internal dynamism support within the CPU plugin
Referencenode, so custom nodes with this feature will also be supported. PR will be ready soon.I think it is ready, just needs to be implemented and tested.
hi @mbencer, thank you so much for your support.
So it looks like internal dynamism of custom ops is not supported now and I didn’t find any instruction about cpu custom ops. In this case I’ve written a new cpu node based on
Referencewhich successfully support the dynamism and temporarily solved my problem without causing other issues. However it’s only a workaround for me, I’m really looking forward to the offical support in the near future.