mmdeploy: When I ran the deploy.py, I got a ERROR: mmdeploy.apis.utils.utils.to_backend with Call id: 1 failed. exit.

Checklist

• I have searched related issues but cannot get the expected help.
• 2. I have read the FAQ documentation but cannot get the expected help.
• 3. The bug has not been fixed in the latest version.

Describe the bug

When I ran the deploy.py, I got a ERROR: mmdeploy.apis.utils.utils.to_backend with Call id: 1 failed. exit.

This error just occurred when using TensorRT. It was successful when using onnxruntime with the same env.

Anyone can help solve this? Thanks so much.

Reproduction

THE command: python .\tools\deploy.py .\configs\mmseg\segmentation_tensorrt_static-512x512.py …\mmsegmentation-dev-1.x\configs\segformer\segformer_mit-b0_8xb2-160k_feces-512x512.py …\mmsegmentation-dev-1.x\work_dirs\segformer_mit-b0_8xb2-160k_feces-512x512\iter_80000.pth .\demo\resources\feces.jpg –work-dir .\mmdepoly_models\mmseg\segformer_tensorrt –device cuda –dump-info

I did not change any config file.

segmentation_tensorrt_static-512x512.py base = [‘./segmentation_static.py’, ‘…/base/backends/tensorrt.py’]

onnx_config = dict(input_shape=[512, 512]) backend_config = dict( common_config=dict(max_workspace_size=1 << 30), model_inputs=[ dict( input_shapes=dict( input=dict( min_shape=[1, 3, 512, 512], opt_shape=[1, 3, 512, 512], max_shape=[1, 3, 512, 512]))) ])

Environment

THE env:
09/08 10:30:21 - mmengine - INFO - TorchVision: 0.14.1
09/08 10:30:21 - mmengine - INFO - OpenCV: 4.8.0
09/08 10:30:21 - mmengine - INFO - MMEngine: 0.8.4
09/08 10:30:21 - mmengine - INFO - MMCV: 2.0.0
09/08 10:30:21 - mmengine - INFO - MMCV Compiler: MSVC 192829924
09/08 10:30:21 - mmengine - INFO - MMCV CUDA Compiler: 11.7
09/08 10:30:21 - mmengine - INFO - MMDeploy: 1.2.0+unknown
09/08 10:30:21 - mmengine - INFO -

09/08 10:30:21 - mmengine - INFO - Backend information
09/08 10:30:21 - mmengine - INFO - tensorrt: 8.6.1
09/08 10:30:21 - mmengine - INFO - tensorrt custom ops: Available
09/08 10:30:21 - mmengine - INFO - ONNXRuntime: 1.8.1
09/08 10:30:21 - mmengine - INFO - ONNXRuntime-gpu: None
09/08 10:30:21 - mmengine - INFO - ONNXRuntime custom ops: Available
09/08 10:30:21 - mmengine - INFO - pplnn: None
09/08 10:30:21 - mmengine - INFO - snpe: None
09/08 10:30:21 - mmengine - INFO - openvino: None
09/08 10:30:21 - mmengine - INFO - torchscript: 1.13.1
09/08 10:30:21 - mmengine - INFO - torchscript custom ops: NotAvailable
09/08 10:30:21 - mmengine - INFO - rknn-toolkit: None
09/08 10:30:21 - mmengine - INFO - rknn-toolkit2: None
09/08 10:30:21 - mmengine - INFO - ascend: None
09/08 10:30:21 - mmengine - INFO - coreml: None
09/08 10:30:21 - mmengine - INFO - tvm: None
09/08 10:30:21 - mmengine - INFO - vacc: None
09/08 10:30:21 - mmengine - INFO -

09/08 10:30:21 - mmengine - INFO - Codebase information
09/08 10:30:21 - mmengine - INFO - mmdet: None
09/08 10:30:21 - mmengine - INFO - mmseg: 1.1.0
09/08 10:30:21 - mmengine - INFO - mmpretrain: None
09/08 10:30:21 - mmengine - INFO - mmocr: None
09/08 10:30:21 - mmengine - INFO - mmagic: None
09/08 10:30:21 - mmengine - INFO - mmdet3d: None
09/08 10:30:21 - mmengine - INFO - mmpose: None
09/08 10:30:21 - mmengine - INFO - mmrotate: None
09/08 10:30:21 - mmengine - INFO - mmaction: None
09/08 10:30:21 - mmengine - INFO - mmrazor: None
09/08 10:30:21 - mmengine - INFO - mmyolo: None

Error traceback

THE output:
09/08 10:51:43 - mmengine - WARNING - Failed to search registry with scope "mmseg" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the bui
lt modules. Please check whether "mmseg" is a correct scope, or whether the registry is initialized.

09/08 10:51:43 - mmengine - WARNING - Failed to search registry with scope "mmseg" in the "mmseg_tasks" registry tree. As a workaround, the current "mmseg_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmseg" is a correct scope, or whether the registry is initialized.

09/08 10:51:44 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
09/08 10:51:45 - mmengine - WARNING - Failed to search registry with scope "mmseg" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmseg" is a correct scope, or whether the registry is initialized.

09/08 10:51:45 - mmengine - WARNING - Failed to search registry with scope "mmseg" in the "mmseg_tasks" registry tree. As a workaround, the current "mmseg_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmseg" is a correct scope, or whether the registry is initialized. d:\allcode\mmlab\mmsegmentation-dev-1.x\mmseg\models\builder.py:36: UserWarning: build_loss would be deprecated soon, please use mmseg.registry.MODELS.build()

warnings.warn('build_loss would be deprecated soon, please use '
d:\allcode\mmlab\mmsegmentation-dev-1.x\mmseg\models\losses\cross_entropy_loss.py:235: UserWarning: Default avg_non_ignore is False, if you would like to ignore the certain label and average loss over non-ignore labels, which is the same with PyTorch officia
l cross_entropy, set avg_non_ignore=True.

warnings.warn(Loads checkpoint by local backend from path: ..\mmsegmentation-dev-1.x\work_dirs\segformer_mit-b0_8xb2-160k_feces-512x512\iter_80000.pth
09/08 10:51:46 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
09/08 10:51:46 - mmengine - INFO - Export PyTorch model to ONNX: .\mmdepoly_models\mmseg\segformer_tensorrt\end2end.onnx.
G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\mmdeploy\core\optimizers\function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ys_shape = tuple(int(s) for s in ys.shape)

G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\mmdeploy\codebase\mmseg\models\segmentors\base.py:47: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
img_shape = [int(val) for val in img_shape]

d:\allcode\mmlab\mmsegmentation-dev-1.x\mmseg\models\utils\shape_convert.py:15: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a const
ant in the future. This means that the trace might not generalize to other inputs!
assert L == H * W, 'The seq_len doesn't match H, W'

G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\mmcv\cnn\bricks\wrappers.py:44: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated a
s a constant in the future. This means that the trace might not generalize to other inputs!
if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 4)):

G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\mmdeploy\codebase\mmseg\models\segmentors\base.py:61: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this v
alue will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if seg_logit.shape[1] == 1:

09/08 10:51:50 - mmengine - INFO - Execute onnx optimize passes.
09/08 10:51:50 - mmengine - WARNING - Can not optimize model, please build torchscipt extension.
More details: https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/experimental/onnx_optimizer.md
09/08 10:51:50 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
09/08 10:51:52 - mmengine - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in subprocess

Process Process-3:
Traceback (most recent call last):
File "G:\softwares\path\Anaconda3\envs\mmseg\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "G:\softwares\path\Anaconda3\envs\mmseg\lib\multiprocessing\process.py", line 108, in run
self.target(*self.args, **self.kwargs)
File "G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py", line 107, in call
ret = func(*args, **kwargs)
File "G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\mmdeploy\apis\utils\utils.py", line 98, in to_backend
return backend_mgr.to_backend(
File "G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\mmdeploy\backend\tensorrt\backend_manager.py", line 113, in to_backend
from . import is_available
File "G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\mmdeploy\backend\tensorrt_init.py", line 13, in
from .utils import from_onnx, load, save
File "G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\mmdeploy\backend\tensorrt\utils.py", line 9, in
import tensorrt as trt
File "G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\tensorrt_init.py", line 127, in
ctypes.CDLL(find_lib(lib))
File "G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\tensorrt_init.py", line 81, in find_lib
raise FileNotFoundError(
FileNotFoundError: Could not find: nvinfer.dll. Is it on your PATH?
Note: Paths searched were:
['G:\softwares\path\Anaconda3\envs\mmseg\Lib\site-packages\cv2\../../x64/vc14/bin', 'G:\softwares\path\Anaconda3\envs\mmseg\Lib\site-packages\cv2\../../x64/vc14/bin', 'G:\softwares\path\Anaconda3\envs\mmseg', 'G:\softwares\path\Anaconda
3\envs\mmseg\Library\mingw-w64\bin', 'G:\softwares\path\Anaconda3\envs\mmseg\Library\usr\bin', 'G:\softwares\path\Anaconda3\envs\mmseg\Library\bin', 'G:\softwares\path\Anaconda3\envs\mmseg\Scripts', 'G:\softwares\path\Anaconda3\envs
\mmseg\bin', 'G:\softwares\path\Anaconda3\condabin', 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\extras\CUPTI\lib64', 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\cudnn\bin', 'C:\Program Files\NVIDIA GPU Computing
Toolkit\CUDA\v11.7\bin', 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\libnvvp', 'C:\Program Files (x86)\Common Files\MVS\Runtime\Win32_i86', 'C:\Program Files (x86)\Common Files\MVS\Runtime\Win64_x64', 'C:\Windows', 'C:\Windows\
System32', 'C:\Windows\System32\Wbem', 'C:\Windows\System32\WindowsPowerShell\v1.0', 'C:\Windows\System32\OpenSSH', 'C:\Program Files\dotnet', 'C:\Program Files (x86)\HP\IdrsOCR_15.2.10.1114', 'C:\Program Files\Microsoft SQL Server\130\Tools\
Binn', 'C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn', 'G:\softwares\path\Anaconda3', 'G:\softwares\path\Anaconda3\Scripts', 'G:\softwares\path\Anaconda3\Library\bin', 'G:\softwares\path\Anaconda3\Library\mingw-w64
', 'G:\softwares\path\Anaconda3\Library\plugins\platforms', 'G:\softwares\path\Python39', 'G:\softwares\path\Python39\Scripts', 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64', 'C:
\Program Files (x86)\dotnet', 'C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR', 'G:\softwares\path\Java\bin', 'G:\softwares\path\pycharm-professional\PyCharm 2021.2.3\bin', 'G:\softwares\path\Matlab2021b\bin', 'C:\Program Files\NVIDIA Cor
poration\Nsight Compute 2022.2.0', 'C:\Program Files\IVI Foundation\VISA\Win64\Bin', 'C:\Program Files (x86)\IVI Foundation\VISA\WinNT\Bin', 'C:\Program Files (x86)\IVI Foundation\VISA\WinNT\Bin', 'C:\Users\我的电脑\AppData\Local\Microsoft\
WindowsApps', 'C:\Users\我的电脑\.dotnet\tools', 'C:\Program Files\Bandizip', 'C:\Users\我的电脑\AppData\Local\GitHubDesktop\bin', 'C:\Users\我的电脑\AppData\Local\gitkraken\bin', 'G:\softwares\path\pycharm-professional\PyCharm 2023.1.2\bi
n', '.']
09/08 10:51:52 - mmengine - ERROR - G:\softwares\path\Anaconda3\envs\mmseg\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py - pop_mp_output - 80 - mmdeploy.apis.utils.utils.to_backend with Call id: 1 failed. exit.