frigate: [Support]: TensorRT detector fails on 0.13.0 Beta 2 nvidia-driver v535 Quadro K620
Describe the problem you are having
TensorRT detector fails to start returning a KeyError
This appeared more than once in the issues page as seen here https://github.com/blakeblackshear/frigate/issues/7149 or here https://github.com/blakeblackshear/frigate/issues/8240 turning out it was an outdated nvidia-driver problem, being the requirement >=530
I have tried to fresh build the models (yolov7-320 and yolov7-tiny-416) with driver 535.113.01 and 545.23.06 with no success and the same KeyError. I have tried multiple times to be sure it is not a problem building the models.
I’m thinking about that perhaps there is another nvidia or cuda library version requirement apart from the nvidia driver, but I can’t find anything related.
Version
0.13.0-0858859
Frigate config file
mqtt:
enabled: false
host: 192.168.20.201
port: 1883
user: user
password: password
detectors:
tensorrt:
type: tensorrt
device: 0
model:
path: /config/model_cache/tensorrt/yolov7-320.trt
input_tensor: nchw
input_pixel_format: rgb
width: 320
height: 320
birdseye:
enabled: false
width: 1920
height: 1080
quality: 8
mode: continuous
snapshots:
enabled: false
timestamp: True
retain:
default: 7
record:
sync_on_startup: false
enabled: True
events:
retain:
default: 7
mode: active_objects
cameras:
yi_pasillo: # <------ Name the camera
ffmpeg:
hwaccel_args: preset-nvidia-h264
inputs:
- path: rtsp://172.16.1.201:8554/yi_pasillo?mp4 # <----- The stream you want to use for detection
roles:
- detect
- record
detect:
enabled: true # <---- disable detection until you have a working camera feed
width: 1920 # <---- update for your camera's resolution
height: 1080 # <---- update for your camera's resolution
objects:
track:
- cat
- person
# Optional: Telemetry configuration
telemetry:
# Optional: Enable the latest version outbound check (default: shown below)
# NOTE: If you use the HomeAssistant integration, disabling this will prevent it from reporting new versions
version_check: True
logger:
# Optional: Default log verbosity (default: shown below)
default: debug
# Optional: Component specific logger overrides
logs:
frigate.event: debug
Relevant log output
Generating the following TRT Models: yolov7-320
Downloading yolo weights
2023-10-22 23:48:54.693998801 [INFO] Preparing go2rtc config...
2023-10-22 23:48:55.279999791 [INFO] Starting go2rtc...
2023-10-22 23:48:55.358361354 23:48:55.358 INF go2rtc version 1.7.1 linux/amd64
2023-10-22 23:48:55.358584913 23:48:55.358 INF [api] listen addr=0.0.0.0:1984
2023-10-22 23:48:55.358747912 23:48:55.358 INF [rtsp] listen addr=0.0.0.0:8554
2023-10-22 23:48:55.358892872 23:48:55.358 INF [webrtc] listen addr=0.0.0.0:8555/tcp
Creating yolov7-320.cfg and yolov7-320.weights
Done.
2023-10-22 23:49:04.576457981 [INFO] Starting go2rtc healthcheck service...
Generating yolov7-320.trt. This may take a few minutes.
/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:147: DeprecationWarning: Use network created with NetworkDefinitionCreationFlag::EXPLICIT_BATCH flag instead.
builder.max_batch_size = MAX_BATCH_SIZE
/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:149: DeprecationWarning: Use set_memory_pool_limit instead.
config.max_workspace_size = 1 << 30
/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:172: DeprecationWarning: Use build_serialized_network instead.
engine = builder.build_engine(network, config)
2023-10-22 23:51:10.011689421 [2023-10-22 23:51:10] detector.tensorrt INFO : Starting detection process: 806
2023-10-22 23:51:10.270448331 [2023-10-22 23:51:10] frigate.app INFO : Output process started: 808
2023-10-22 23:51:10.270586861 [2023-10-22 23:51:10] frigate.app INFO : Camera processor started for yi_pasillo: 813
2023-10-22 23:51:10.270682860 [2023-10-22 23:51:10] frigate.app INFO : Capture process started for yi_pasillo: 815
2023-10-22 23:51:10.310125605 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Num Available Devices: 1
2023-10-22 23:51:10.313907082 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
2023-10-22 23:51:10.314015131 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::BatchedNMS_TRT version 1
2023-10-22 23:51:10.314120181 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::BatchTilePlugin_TRT version 1
2023-10-22 23:51:10.314205411 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::Clip_TRT version 1
2023-10-22 23:51:10.314289530 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::CoordConvAC version 1
2023-10-22 23:51:10.314372600 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::CropAndResizeDynamic version 1
2023-10-22 23:51:10.314454470 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::CropAndResize version 1
2023-10-22 23:51:10.314541389 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::DecodeBbox3DPlugin version 1
2023-10-22 23:51:10.314623549 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::DetectionLayer_TRT version 1
2023-10-22 23:51:10.314708839 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
2023-10-22 23:51:10.314789049 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
2023-10-22 23:51:10.314869118 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
2023-10-22 23:51:10.314951268 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::EfficientNMS_TRT version 1
2023-10-22 23:51:10.315035468 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::FlattenConcat_TRT version 1
2023-10-22 23:51:10.315127527 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::fMHA_V2 version 1
2023-10-22 23:51:10.315209387 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::fMHCA version 1
2023-10-22 23:51:10.315288777 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::GenerateDetection_TRT version 1
2023-10-22 23:51:10.315371597 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::GridAnchor_TRT version 1
2023-10-22 23:51:10.315455446 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::GridAnchorRect_TRT version 1
2023-10-22 23:51:10.315536416 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::GroupNorm version 1
2023-10-22 23:51:10.315622836 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::InstanceNormalization_TRT version 1
2023-10-22 23:51:10.315702825 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::InstanceNormalization_TRT version 2
2023-10-22 23:51:10.315786415 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::LayerNorm version 1
2023-10-22 23:51:10.315872565 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::LReLU_TRT version 1
2023-10-22 23:51:10.315955005 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
2023-10-22 23:51:10.316035184 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::MultilevelProposeROI_TRT version 1
2023-10-22 23:51:10.316122694 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
2023-10-22 23:51:10.316210084 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::NMSDynamic_TRT version 1
2023-10-22 23:51:10.316292433 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::NMS_TRT version 1
2023-10-22 23:51:10.316375663 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::Normalize_TRT version 1
2023-10-22 23:51:10.316454713 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::PillarScatterPlugin version 1
2023-10-22 23:51:10.316533333 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::PriorBox_TRT version 1
2023-10-22 23:51:10.316614902 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::ProposalDynamic version 1
2023-10-22 23:51:10.316698302 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::ProposalLayer_TRT version 1
2023-10-22 23:51:10.316788992 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::Proposal version 1
2023-10-22 23:51:10.316878601 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::PyramidROIAlign_TRT version 1
2023-10-22 23:51:10.316955081 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::Region_TRT version 1
2023-10-22 23:51:10.317032391 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::Reorg_TRT version 1
2023-10-22 23:51:10.317124101 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::ResizeNearest_TRT version 1
2023-10-22 23:51:10.317215630 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::ROIAlign_TRT version 1
2023-10-22 23:51:10.317293980 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::RPROI_TRT version 1
2023-10-22 23:51:10.317375530 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::ScatterND version 1
2023-10-22 23:51:10.317455299 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::SeqLen2Spatial version 1
2023-10-22 23:51:10.317534089 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::SpecialSlice_TRT version 1
2023-10-22 23:51:10.317613139 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::SplitGeLU version 1
2023-10-22 23:51:10.317692879 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::Split version 1
2023-10-22 23:51:10.317774358 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Registered plugin creator - ::VoxelGeneratorPlugin version 1
2023-10-22 23:51:10.477917168 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt INFO : Loaded engine size: 147 MiB
2023-10-22 23:51:10.608822897 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Trying to load shared library libcublas.so.11
2023-10-22 23:51:10.608894947 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Loaded shared library libcublas.so.11
2023-10-22 23:51:10.625204291 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Using cublas as plugin tactic source
2023-10-22 23:51:10.625287371 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Using cublas as core library tactic source
2023-10-22 23:51:10.625414400 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt INFO : [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +6, GPU +8, now: CPU 268, GPU 181 (MiB)
2023-10-22 23:51:10.625571520 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Trying to load shared library libcudnn.so.8
2023-10-22 23:51:10.625744969 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Loaded shared library libcudnn.so.8
2023-10-22 23:51:10.625828209 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Using cuDNN as plugin tactic source
2023-10-22 23:51:10.631090911 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Using cuDNN as core library tactic source
2023-10-22 23:51:10.631202561 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt INFO : [MemUsageChange] Init cuDNN: CPU +2, GPU +10, now: CPU 270, GPU 191 (MiB)
2023-10-22 23:51:10.640966717 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Deserialization required 63731 microseconds.
2023-10-22 23:51:10.641061507 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt INFO : [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +148, now: CPU 0, GPU 148 (MiB)
2023-10-22 23:51:10.660826399 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Trying to load shared library libcublas.so.11
2023-10-22 23:51:10.660896619 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Loaded shared library libcublas.so.11
2023-10-22 23:51:10.660985948 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Using cublas as plugin tactic source
2023-10-22 23:51:10.661068018 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Using cublas as core library tactic source
2023-10-22 23:51:10.661165898 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt INFO : [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 122, GPU 184 (MiB)
2023-10-22 23:51:10.661252707 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Trying to load shared library libcudnn.so.8
2023-10-22 23:51:10.661374237 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Loaded shared library libcudnn.so.8
2023-10-22 23:51:10.661454747 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Using cuDNN as plugin tactic source
2023-10-22 23:51:10.661539506 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Using cuDNN as core library tactic source
2023-10-22 23:51:10.661626246 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt INFO : [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 122, GPU 192 (MiB)
2023-10-22 23:51:10.661708566 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Total per-runner device persistent memory is 1064448
2023-10-22 23:51:10.661792345 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Total per-runner host persistent memory is 199056
2023-10-22 23:51:10.661872475 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Allocated activation device memory of size 29286400
2023-10-22 23:51:10.661959755 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt INFO : [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +29, now: CPU 0, GPU 177 (MiB)
2023-10-22 23:51:10.662045545 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : CUDA lazy loading is enabled.
2023-10-22 23:51:10.662467553 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Allocated Tensor Binding input Memory 1228800 Bytes (307200 * DataType.FLOAT)
2023-10-22 23:51:10.662552103 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Input has Shape (1, 3, 320, 320)
2023-10-22 23:51:10.662633233 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Allocated Tensor Binding detections Memory 176400 Bytes (44100 * DataType.FLOAT)
2023-10-22 23:51:10.662710112 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : Output has Shape (1, 44100, 1, 1)
2023-10-22 23:51:10.662793262 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : TensorRT loaded. Input shape is ((320, 320), <class 'numpy.float32'>)
2023-10-22 23:51:10.662880662 [2023-10-22 23:51:10] frigate.detectors.plugins.tensorrt DEBUG : TensorRT version is 8
2023-10-22 23:51:14.987266359 [2023-10-22 23:51:14] asyncio DEBUG : Using selector: EpollSelector
2023-10-22 23:51:16.579766952 Process camera_processor:yi_pasillo:
2023-10-22 23:51:16.580948828 Traceback (most recent call last):
2023-10-22 23:51:16.580993938 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-10-22 23:51:16.580996188 self.run()
2023-10-22 23:51:16.580997848 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-10-22 23:51:16.580999438 self._target(*self._args, **self._kwargs)
2023-10-22 23:51:16.581009397 File "/opt/frigate/frigate/video.py", line 501, in track_camera
2023-10-22 23:51:16.581010787 process_frames(
2023-10-22 23:51:16.581012267 File "/opt/frigate/frigate/video.py", line 876, in process_frames
2023-10-22 23:51:16.581013437 detect(
2023-10-22 23:51:16.581014917 File "/opt/frigate/frigate/video.py", line 575, in detect
2023-10-22 23:51:16.581016347 region_detections = object_detector.detect(tensor_input)
2023-10-22 23:51:16.581018427 File "/opt/frigate/frigate/object_detection.py", line 225, in detect
2023-10-22 23:51:16.581019577 (self.labels[int(d[0])], float(d[1]), (d[2], d[3], d[4], d[5]))
2023-10-22 23:51:16.581037467 KeyError: -1
2023-10-22 23:51:19.987372569 [2023-10-22 23:51:19] asyncio DEBUG : Using selector: EpollSelector
2023-10-22 23:51:24.987429541 [2023-10-22 23:51:24] asyncio DEBUG : Using selector: EpollSelector
2023-10-22 23:51:25.051001568 [2023-10-22 23:51:25] peewee DEBUG : ('SELECT "t1"."start_time", "t1"."end_time" FROM "event" AS "t1" WHERE ((("t1"."camera" = ?) AND (("t1"."end_time" IS ?) OR ("t1"."end_time" >= ?))) AND "t1"."has_clip") ORDER BY "t1"."start_time"', ['yi_pasillo', None, 1698011471.0])
2023-10-22 23:51:25.053278069 [2023-10-22 23:51:25] frigate.record.maintainer ERROR : Error occurred when attempting to maintain recording cache
2023-10-22 23:51:25.053354819 [2023-10-22 23:51:25] frigate.record.maintainer ERROR : list index out of range
FFprobe output from your camera
root@411cd292336e:/opt/frigate# ffprobe rtsp://172.16.1.201:8554/yi_pasillo?mp4
ffprobe version n5.1-2-g915ef932a3-20220731 Copyright (c) 2007-2022 the FFmpeg developers
built with gcc 12.1.0 (crosstool-NG 1.25.0.55_3defb7b)
configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20220731
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
libpostproc 56. 6.100 / 56. 6.100
[rtsp @ 0x5630a4865140] method SETUP failed: 461 Unsupported transport
Input #0, rtsp, from 'rtsp://172.16.1.201:8554/yi_pasillo?mp4':
Metadata:
title : go2rtc/1.8.1
Duration: N/A, start: 0.046478, bitrate: N/A
Stream #0:0: Video: h264 (Main), yuvj420p(pc, progressive), 1920x1088, 20 fps, 19.92 tbr, 90k tbn
Frigate stats
No response
Operating system
UNRAID
Install method
Docker CLI
Coral version
CPU (no coral)
Network connection
Wired
Camera make and model
yi dome
Any other information that may be helpful
Thu Oct 26 09:15:59 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.113.01 Driver Version: 535.113.01 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Quadro K620 Off | 00000000:08:00.0 Off | N/A |
| 46% 58C P0 3W / 30W | 199MiB / 2048MiB | 2% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 13832 C frigate.detector.tensorrt 89MiB |
| 0 N/A N/A 13854 C ffmpeg 106MiB |
+---------------------------------------------------------------------------------------+
Sun Oct 22 23:53:32 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.23.06 Driver Version: 545.23.06 CUDA Version: 12.3 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Quadro K620 Off | 00000000:08:00.0 Off | N/A |
| 53% 60C P0 3W / 30W | 328MiB / 2048MiB | 2% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 6448 C frigate.detector.tensorrt 219MiB |
| 0 N/A N/A 6466 C ffmpeg 106MiB |
+---------------------------------------------------------------------------------------+
About this issue
- Original URL
- State: closed
- Created 8 months ago
- Comments: 36 (2 by maintainers)
Yes, it’s the same, only difference the model, but tried both and the same KeyError. Files are being created, I am deleting the folder each time I try.
Also I tried to create them using the old 0.12 way but changing the tensorrt container to 23.02. However the same error.
I also tried reverting 0.13 beta 2 to tensorrt 8.4.1 but I kept getting the same error.
As I said, I’m thinking about some python dependency is breaking the detection in legacy cards i.e maxwell, but reviewing the code and the commits I can’t get to a conclusion
I’m not certain where the negative numbers are coming from, there are a couple layers to the process that spits out the detections.
The TensorRT detector uses YOLO models which have a very different output than the SSD model frigate was originally designed with. Part of the project where we get our YOLO models from includes a port-processing library (
libyolo_layer.so) that will do the math required to take the detections as YOLO spits them out, do the sigmoid calculations, and return a single detection per object, closer to what SSD would.Once the Frigate TensorRT detector gets the results from the model, it further has to sort and reshape the results into an array format the the Object Detection expects.
Somewhere between the model, post-processing, and parsing the class is mis-calculated. We can try adding more logging straight out of the TensorRT, but I’m not an expert when it comes to how the class numbers are being generated by the YOLO model. We could test if the absolute value is sufficient? Try discarding negative values? But if these values are wrong, will we get any good detections out of your setup?
I don’t think I saw, were you able to get TensorRT working with 0.12 on your machine?