TensorRT: [blockChooser.cpp::getRegionBlockSize::690] Error Code 2: Internal Error (Assertion memSize >= 0 failed. )

Description

Encounter the error as follows:

[blockChooser.cpp::getRegionBlockSize::690] Error Code 2: Internal Error (Assertion memSize >= 0 failed. )

Environment

TensorRT Version: 8+ NVIDIA GPU: 1660 NVIDIA Driver Version: 470 CUDA Version: 11.3 CUDNN Version: compatible with cuda 11.3 Operating System: linux x86 Python Version (if applicable): 3.8 PyTorch Version (if applicable): 1.10

Steps To Reproduce

import torch
import onnx
import tensorrt as trt

onnx_model = 'model.onnx'


class NaiveModel(torch.nn.Module):

    def __init__(self):
        super().__init__()

    def forward(self, x):
        B, C, H, W = x.shape

        pad_w = W % 7
        pad_h = H % 7
        x_t = torch.zeros((B, C, H + pad_h, W + pad_w), device=x.device)
        x_t[:, :, :H, :W] = x
        return x_t


device = torch.device('cuda:0')

# generate ONNX model
torch.onnx.export(
    NaiveModel(),
    torch.randn(1, 3, 224, 224),
    onnx_model,
    input_names=['input'],
    output_names=['output'],
    dynamic_axes=dict(
        input=dict({
            0: 'batch',
            2: 'height',
            3: 'width'
        }),
        output=dict({0: 'batch'})),
    opset_version=11)
onnx_model = onnx.load(onnx_model)

# load_tensorrt_plugin()
# create builder and network
logger = trt.Logger(trt.Logger.ERROR)
builder = trt.Builder(logger)
EXPLICIT_BATCH = 1 << (int)(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)
network = builder.create_network(EXPLICIT_BATCH)

# parse onnx
parser = trt.OnnxParser(network, logger)

if not parser.parse(onnx_model.SerializeToString()):
    error_msgs = ''
    for error in range(parser.num_errors):
        error_msgs += f'{parser.get_error(error)}\n'
    raise RuntimeError(f'Failed to parse onnx, {error_msgs}')

config = builder.create_builder_config()
config.max_workspace_size = 1 << 30
profile = builder.create_optimization_profile()

profile.set_shape('input', [1, 3, 112, 112], [1, 3, 224, 224],
                  [1, 3, 512, 512])
config.add_optimization_profile(profile)
# create engine
with torch.cuda.device(device):
    engine = builder.build_engine(network, config)

with open('model.engine', mode='wb') as f:
    f.write(bytearray(engine.serialize()))
    print("generating file done!")

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 15

Most upvoted comments

A workaround is to replace the reshape in prepare_onnx_paddings with the Concat of two Slice with step=2 (begins and ends) of paddings.


    # paddings = sym_help._reshape_helper(
    #     g, paddings, g.op("Constant", value_t=torch.tensor([-1, 2])))
    # paddings = g.op(
    #     "Transpose",
    #     torch.onnx.symbolic_opset10.flip(g, paddings, [0]),
    #     perm_i=[1, 0])
    # paddings = sym_help._reshape_helper(
    #     g, paddings, g.op("Constant", value_t=torch.tensor([-1])))


    paddings = torch.onnx.symbolic_opset10.flip(g, paddings, [0])
    begins = sym_help._slice_helper(
        g, paddings, axes=[0], starts=[1], ends=[0xffff], steps=[2])
    ends = sym_help._slice_helper(
        g, paddings, axes=[0], starts=[0], ends=[0xffff], steps=[2])
    paddings = g.op('Concat', begins, ends, axis_i=0)

grimoire on Jun 11, 2022

Yes, for model

class NaiveModel(torch.nn.Module):

    def __init__(self):
        super().__init__()

    def forward(self, x):
        B, C, H, W = x.shape

        pad_h = H - (H // 7) * 7
        pad_w = W - (W // 7) * 7
        pad = torch.nn.ZeroPad2d((0, pad_w, 0, pad_h))
        x = pad(x)
        x = x.reshape(B, C, (H + pad_h) * (W + pad_w))
        return x

I can see the error

[06/09/2022-15:34:34] [E] [TRT] ModelImporter.cpp:775: input: "input"
input: "onnx::Pad_59"
input: "onnx::Pad_60"
output: "onnx::Reshape_61"
name: "Pad_51"
op_type: "Pad"
attribute {
  name: "mode"
  s: "constant"
  type: STRING
}

[06/09/2022-15:34:34] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[06/09/2022-15:34:34] [E] [TRT] ModelImporter.cpp:779: ERROR: ModelImporter.cpp:180 In function parseGraph:
[6] Invalid Node - Pad_51
[shuffleNode.cpp::symbolicExecute::392] Error Code 4: Internal Error (Reshape_40: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2])
[06/09/2022-15:34:34] [E] Failed to parse onnx file
[06/09/2022-15:34:34] [I] Finish parsing network model
[06/09/2022-15:34:34] [E] Parsing model failed
[06/09/2022-15:34:34] [E] Failed to create engine from model or file.
[06/09/2022-15:34:34] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8401] # /usr/src/tensorrt/bin/trtexec --onnx=model.onnx --optShapes=input:1x3x224x224 --shapes=input:1x3x224x224

@jackwish @nvpohanh Do we support ND shape tensor in 8.4?

zerollzeng on Jun 9, 2022