TensorRT: [TensorRT] ERROR: Network must have at least one output
Hi, I try to convert my .onnx model to .trt model using tensorrt, my script is:
import tensorrt as trt
def ONNX_build_engine(onnx_file_path, engine_file_path):
G_LOGGER = trt.Logger(trt.Logger.WARNING)
with trt.Builder(G_LOGGER) as builder, builder.create_network() as network, trt.OnnxParser(network, G_LOGGER) as parser:
builder.max_batch_size = 16
builder.max_workspace_size = 1 << 20
print('Loading ONNX file from path {}...'.format(onnx_file_path))
with open(onnx_file_path, 'rb') as model:
print('Beginning ONNX file parsing')
parser.parse(model.read())
print('Completed parsing of ONNX file')
print('Building an engine from file {}; this may take a while...'.format(onnx_file_path))
engine = builder.build_cuda_engine(network)
print("Completed creating Engine")
with open(engine_file_path, "wb") as f:
f.write(engine.serialize())
return engine
ONNX_build_engine('./BiSeNet.onnx', './BiSeNet.trt')
I have tested in my two desktops, one configue:
cuda 10.0 pytorch1.3 torch 5.0.6.6
the other configure:
cuda 10.1 pytorch1.3 torch6.0.1.5
Both have the same error. there is an error:
Loading ONNX file from path ./BiSeNet_simplifier.onnx… Beginning ONNX file parsing Completed parsing of ONNX file Building an engine from file ./BiSeNet_simplifier.onnx; this may take a while… [TensorRT] ERROR: Network must have at least one output Completed creating Engine Traceback (most recent call last): File “onnx2trt.py”, line 31, in <module> ONNX_build_engine(‘./BiSeNet_simplifier.onnx’, ‘./BiSeNet.trt’) File “onnx2trt.py”, line 28, in ONNX_build_engine f.write(engine.serialize()) AttributeError: ‘NoneType’ object has no attribute ‘serialize’
The .onnx file is converted by torch.onnx.export fcn, it seems right.
import torch
from model.build_BiSeNet import BiSeNet
model = BiSeNet(13, 'resnet18').eval() # .cuda().half().eval()
model.load_state_dict(torch.load('./checkpoints_18_sgd/best_dice_loss_13_class.pth'))
x = torch.ones((1, 3, 720, 960)) # .cuda().half()
output_names = ["output"]
onnx_model = torch.onnx._export(model, x, "BiSeNet.onnx", verbose=False, output_names=output_names, export_params=True) #, keep_initializers_as_inputs=True)
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 21
Hi @RizhaoCai
I just had the same issue with TensorRT 7.0. The solution, as mentioned in the documentaiton, was creating the network using the EXPLICIT_BATCH flag:
just add
it will work. It’s a necessary procedure to convert successfully, I’m curious why most tutorials don’t mention it.
Hi, @rmccorm4
Thanks for your solution. It works for many cases. However, when my network has two outputs, this seems not to work.
For examples
In this case, using last_layer.get_output(0) is OK. But if I put last_layer.get_output(1), and OutOfIndex error will occur. And I find that the using
second_last_layer = network.get_layer(network.num_layers - 2)
second_last_layer.get_output(0)
do not give me what I want.