openvino: [Bug]: [ GENERAL_ERROR ] could not append an elementwise post-op

OpenVINO Version

2023.1.0-12185-9e6b00e51cd-releases/2023/1

Operating System

Ubuntu 20.04 (LTS)

Device used for inference

CPU

Framework

ONNX

Model used

RTMDET

Issue description

I am trying to compile and run a quantized openvino model but I get

line 543, in compile_model
super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src/inference/src/core.cpp:114:
[ GENERAL_ERROR ] could not append an elementwise post-op

my quantization script runs successfully but I get the error when trying to compile the model. Further, I can confirm I am able to successfully compile and run inference with the fp32 openVino model that my quantized model is derived from. My code for conversion quantizing and compiling my model

import nncf
import torch
import numpy as np
import openvino as ov
from torchvision import transforms
from torch.utils.data import DataLoader
from torchvision.datasets import ImageFolder
from torch.utils.data import Dataset
import os
from PIL import Image

class CustomDataset(Dataset):
def __init__(self, root_dir, transform=None):
self.root_dir = root_dir
self.transform = transform
self.image_files = [f for f in os.listdir(root_dir) if os.path.isfile(os.path.join(root_dir, f))]

def __len__(self):
return len(self.image_files)

def __getitem__(self, idx):
img_path = os.path.join(self.root_dir, self.image_files[idx])
image = Image.open(img_path).convert('RGB')

if self.transform:
image = self.transform(image)

return image


def normalize_image(img, mean, std):
"""
Normalize an OpenCV BGR image using given mean and std values.

Args:
- img (numpy.ndarray): BGR image read using OpenCV.
- mean (list): List of mean values for BGR channels.
- std (list): List of standard deviation values for BGR channels.

Returns:
- numpy.ndarray: Normalized image.
"""

# Convert image to float32 for the normalization process
img = img.astype(np.float32)

# Normalize each channel
for i in range(3): # For B, G, and R channels
img[:,:,i] = (img[:,:,i] - mean[i]) / std[i]

return img


def transform_fn(data_item):
images = data_item
normalized_images = [normalize_image(img.permute(1, 2, 0).numpy(), [103.53, 116.28, 123.675], [57.375, 57.12, 58.395]) for img in images]
return np.array(normalized_images).transpose(0, 3, 1, 2)

def main():

transform = transforms.Compose([
transforms.ToTensor(),
])

calibration_loader = DataLoader(
CustomDataset(root_dir='/home/calibration_imgs', transform=transform),
batch_size=1, shuffle=False
)

calibration_dataset = nncf.Dataset(calibration_loader, transform_fn)
model = ov.Core().read_model('../models/end2end_m_ov.xml')
quantized_model = nncf.quantize(model, calibration_dataset, target_device=nncf.TargetDevice.CPU)

ov.serialize(quantized_model, "../models/end2end_m_ov_quant.xml")

model_int8 = ov.Core().compile_model(quantized_model,device_name='CPU')

if __name__=='__main__':
main()

Step-by-step reproduction

No response

Relevant log output

No response

Issue submission checklist

• I’m reporting an issue. It’s not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.