opencv: Error loading faster rcnn using opencv

@plin24, the following experiment gives relatively similar outputs:

1. Modify to_remove method at tf_text_graph_faster_rcnn.py. And generate a .pbtxt file.

   from

        def to_remove(name, op):
            return name.startswith(scopesToIgnore) or not name.startswith(scopesToKeep)
   

   to

       def to_remove(name, op):
           if name.startswith('CropAndResize'):
               return name != 'CropAndResize/CropAndResize'
           return name.startswith(scopesToIgnore) or not name.startswith(scopesToKeep)
   

2. Open resulting .pbtxt file and find a node with name MaxPool2D/MaxPool (it should has no inputs). Add a single input for it:

   node {
     name: "MaxPool2D/MaxPool"
     op: "MaxPool"
     input: "CropAndResize/CropAndResize"
     ...
   

3. Then remove nodes with the following names:

   • SecondStageBoxPredictor/Flatten_1/flatten/Shape
   • SecondStageBoxPredictor/Flatten_1/flatten/strided_slice
   • SecondStageBoxPredictor/Flatten_1/flatten/Reshape/shape

4. Replace one of Reshape nodes to Flatten:

   from

   node {
     name: "SecondStageBoxPredictor/Flatten_1/flatten/Reshape"
     op: "Reshape"
     input: "SecondStageBoxPredictor/AvgPool_1"
     input: "SecondStageBoxPredictor/Flatten_1/flatten/Reshape/shape"
   }
   

   to

   node {
     name: "SecondStageBoxPredictor/Flatten_1/flatten/Reshape"
     op: "Flatten"
     input: "SecondStageBoxPredictor/AvgPool_1"
   }
   

Using the following script I can receive pretty similar results

import numpy as np
import tensorflow as tf
import cv2 as cv

# Run TensorFlow

# Read the graph.
with tf.gfile.FastGFile('frozen_inference_graph.pb', 'rb') as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())

with tf.Session() as sess:
    # Restore session
    sess.graph.as_default()
    tf.import_graph_def(graph_def, name='')

    # Read and preprocess an image.
    img = cv.imread('example.png')
    rows = img.shape[0]
    cols = img.shape[1]
    inp = cv.resize(img, (800, 600))
    inp = inp[:, :, [2, 1, 0]]  # BGR2RGB

    # Run the model
    out = sess.run([sess.graph.get_tensor_by_name('num_detections:0'),
                    sess.graph.get_tensor_by_name('detection_scores:0'),
                    sess.graph.get_tensor_by_name('detection_boxes:0'),
                    sess.graph.get_tensor_by_name('detection_classes:0')],
                   feed_dict={'image_tensor:0': inp.reshape(1, inp.shape[0], inp.shape[1], 3)})

    # Visualize detected bounding boxes.
    num_detections = int(out[0][0])
    for i in range(num_detections):
        classId = int(out[3][0][i])
        score = float(out[1][0][i])
        bbox = [float(v) for v in out[2][0][i]]
        if score > 0.25:
            x = bbox[1] * cols
            y = bbox[0] * rows
            right = bbox[3] * cols
            bottom = bbox[2] * rows
            cv.rectangle(img, (int(x), int(y)), (int(right), int(bottom)), (125, 255, 51), thickness=2)

cv.imshow('TensorFlow MobileNet-SSD', img)

# Run OpenCV

cvNet = cv.dnn.readNetFromTensorflow('frozen_inference_graph.pb', 'graph.pbtxt')

img = cv.imread('example.png')
rows = img.shape[0]
cols = img.shape[1]
cvNet.setInput(cv.dnn.blobFromImage(img, size=(800, 600), swapRB=True, crop=False))
cvOut = cvNet.forward()

for detection in cvOut[0,0,:,:]:
    score = float(detection[2])
    if score > 0.25:
        left = detection[3] * cols
        top = detection[4] * rows
        right = detection[5] * cols
        bottom = detection[6] * rows
        cv.rectangle(img, (int(left), int(top)), (int(right), int(bottom)), (23, 230, 210), thickness=2)

cv.imshow('img', img)
cv.waitKey()

TensorFlow:

OpenCV:

As you may see one of objects is missed. However the rest of them are predicted correctly. Please give us more time to figure out what the problem is.

@dkurt this is a test image https://github.com/priya-dwivedi/Deep-Learning/blob/master/Custom_Mask_RCNN/test_images/image1.jpg and result https://drive.google.com/drive/u/1/folders/1N6-_aIIu6CfBnzqr5Sw7sOn8_ValvUHD

@dkurt Thank you so much for solving this issue and good luck on fixing the error!