mmdetection: INVALID_ARGUMENT: getPluginCreator could not find plugin NonMaxSuppression version 1

I have successfully converted yolov3 pytorch model to onnx and then tensorrt by the tools:pytorch2onnx.py and onnx2tensorrt.py, but when i use tensorrt do inference, it gives me the errors:

[TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin NonMaxSuppression version 1 [TensorRT] ERROR: safeDeserializationUtils.cpp (322) - Serialization Error in load: 0 (Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry) [TensorRT] ERROR: INVALID_STATE: std::exception [TensorRT] ERROR: INVALID_CONFIG: Deserialize the cuda engine failed. Traceback (most recent call last): File “onnx_to_tensorrt.py”, line 106, in <module> main() File “onnx_to_tensorrt.py”, line 73, in main with get_engine(engine_file_path) as engine, engine.create_execution_context() as context: AttributeError: enter

The onnx_to_tensorrt.py file as below:

def get_engine(engine_file_path=“”):

if os.path.exists(engine_file_path):
    # If a serialized engine exists, use it instead of building an engine.
    print("Reading engine from file {}".format(engine_file_path))
    with open(engine_file_path, "rb") as f, trt.Runtime(TRT_LOGGER) as runtime:
        return runtime.deserialize_cuda_engine(f.read())
return None`

def main():

engine_file_path = "yolov3.trt"
input_image_path = 'images/00076.jpg'
image = Image.open(input_image_path)
image = np.transpose(image, [2, 0, 1])
trt_outputs = []
with get_engine(engine_file_path) as engine, engine.create_execution_context() as context:
    inputs, outputs, bindings, stream = common.allocate_buffers(engine)
    # Do inference
    print('Running inference on image {}...'.format(input_image_path))
    # Set host input to the image. The common.do_inference function will copy the input to the GPU before executing.
    inputs[0].host = image
    trt_outputs = common.do_inference_v2(context, bindings=bindings, inputs=inputs, outputs=outputs, stream=stream)
    print(trt_outputs)

I’m not sure it is the correct way to use the ‘.trt’ file. Could you give me some suggestions? Ubuntu-16.04 pytorch 1.6 cuda 10.2 tensorrt 7.2.1.6 mmcv 1.3.7 mmdet 2.13 onnx 1.7

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 18

Most upvoted comments

@BChunlei When you convert model to ONNX for TensorRT backend and hope its input shape be dynamic, you have to consider the min and max input shape. Based on the preprocessing pipeline in the config, images from coco dataset may be processed to the shape such as 192 608 or 608 192. Thus, when you create onnx using pytorch2onnx.py, the argument --shape should be min_size min_size. Then when using onnx2tensorrt.py, argument --min-shape should be min_size min_size, --max-shape should be max_size max_size.