TensorRT: ❓ [Question] How do you fix "broadcast dimensions must be conformable"?

❓ Question

I’m converting my (EfficientDet) model using torch_tensorrt.compile like so, after having successfully torchscript’ed it and verified that the output is the same as the non-torchscripted version:

trt_module = torch_tensorrt.compile(scripted_model, inputs=[torch.ones(1, 3, 512, 512)], enabled_precisions={torch.half})

When running the above, I’m getting this error:

ERROR: [Torch-TensorRT TorchScript Conversion Context] - 4: [graphShapeAnalyzer.cpp::analyzeShapes::1285] Error Code 4: Miscellaneous (IElementWiseLayer %46481 : Tensor = aten::add(%46424, %46480, %self.model.blocks.1) # /home/user/model.py:189:41: broadcast dimensions must be conformable)

The offending line is:

p5_up = self.conv5_up(self.swish(weight[0] * p5_in + weight[1] * self.p5_upsample(p6_up)))

where weight is a two-element tensor.

What you have already tried

When broken up, the error happens more specifically with the add:

weight_p5_in = weight[0] * p5_in
weight_p5_upsample = weight[1] * self.p5_upsample(p6_up)
weight_swish_add = weight_p5_in + weight_p5_upsample # error happens here
weight_swish = self.swish(weight_swish_add)
p5_up = self.conv5_up(weight_swish)

The shapes of both weight_p5_in and weight_p5_upsample are [1, 64, 16, 16], so it can’t be the shapes that are the problem. Furthermore, the line before it is very similar and has no error:

p6_up = self.conv6_up(self.swish(weight[0] * p6_in + weight[1] * self.p6_upsample(p7_in)))

I took a look at the values and didn’t see anything fishy. I’m pretty much at a loss of what else to look at.

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

• PyTorch Version (e.g., 1.0): 1.11.0a0+17540c5
• CPU Architecture: x86_64
• OS (e.g., Linux): Linux
• How you installed PyTorch (conda, pip, libtorch, source): NGC PyTorch container, nvcr.io/nvidia/pytorch:22.02-py3
• Build command you used (if compiling from source):
• Are you using local sources or building from archives:
• Python version: 3.8.12
• CUDA version: 11.6
• GPU models and configuration: Quadro RTX 5000
• Any other relevant information: Torch-TensorRT version 1.1.0a0

Additional context

I was under the impression that any model that is torch.jit.script()able is able to be converted using Torch-TensorRT. That has not been the case for me, this is actually the second problem I’m facing when converting my torch.jit.script()ed model after having spent a while fixing the first issue (dealing with math.ceil() not being supported). Could I perhaps be using the wrong API?