frigate: [Support]: NVIDIA hardware decoding errors with h264

Describe the problem you are having

I have an NVIDIA Geforce 970 graphics card. It’s listed here as supporting decoding of h264. I’ve done my best to follow the directions, but something is still not right. I’m getting errors loading the h264_cuvid library. I’ve also looked through issue #1847. Thanks in advance.

Version

0.9.2

Frigate config file

# https://blakeblackshear.github.io/frigate/configuration/index#mqtt
mqtt:
  host: 192.168.1.226
  port: 1883
# https://blakeblackshear.github.io/frigate/configuration/index#cameras
cameras:
  inside:
    ffmpeg:
      input_args:
        - -c:v
        - h264_cuvid
      inputs:
        # sub-stream, lower quality and FPS
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.1.111:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        # full 4k stream
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.1.111:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
    # detect on the sub-stream for improved performance
    detect:
      width: 640
      height: 480
      fps: 5
    objects:
      track:
        - person
    record:
      # Optional: Enable recording
      enabled: True
      retain_days: 0
      events:
        retain:
          default: 10
    rtmp:
      # Required: Enable the live stream
      enabled: False
        # Optional: Configuration for the jpg snapshots written to the clips directory for each event
    snapshots:
      # Optional: Enable writing jpg snapshot to /media/frigate/clips
      # This value can be set via MQTT and will be updated in startup based on retained value
      enabled: True
      # Optional: print a timestamp on the snapshots
      timestamp: False
      # Optional: draw bounding box on the snapshots
      bounding_box: False
      # Optional: crop the snapshot
      crop: False
      # Optional: Camera override for retention settings
      retain:
        # Required: Default retention days
        default: 10
        # Optional: Per object retention days
        objects:
          person: 15
    # Optional: Configuration for the jpg snapshots published via MQTT
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: True
      # Optional: print a timestamp on the snapshots
      timestamp: False
      # Optional: draw bounding box on the snapshots
      bounding_box: False
      # Optional: crop the snapshot
      crop: False
  frontdoor:
    ffmpeg:
      input_args:
        - -c:v
        - h264_cuvid
      inputs:
        # sub-stream, lower quality and FPS
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.1.110:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
        # full 4k stream
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.1.110:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
    # detect on the sub-stream for improved performance
    detect:
      width: 704
      height: 480
      fps: 5
    objects:
      track:
        - person
    record:
      # Optional: Enable recording
      enabled: True
      retain_days: 0
      events:
        retain:
          default: 10
    rtmp:
      # Required: Enable the live stream
      enabled: False
        # Optional: Configuration for the jpg snapshots written to the clips directory for each event
    snapshots:
      # Optional: Enable writing jpg snapshot to /media/frigate/clips
      # This value can be set via MQTT and will be updated in startup based on retained value
      enabled: True
      # Optional: print a timestamp on the snapshots
      timestamp: False
      # Optional: draw bounding box on the snapshots
      bounding_box: False
      # Optional: crop the snapshot
      crop: False
      # Optional: Camera override for retention settings
      retain:
        # Required: Default retention days
        default: 10
        # Optional: Per object retention days
        objects:
          person: 15
    # Optional: Configuration for the jpg snapshots published via MQTT
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: True
      # Optional: print a timestamp on the snapshots
      timestamp: False
      # Optional: draw bounding box on the snapshots
      bounding_box: False
      # Optional: crop the snapshot
      crop: False
  backdoor:
    ffmpeg:
      input_args:
        - -c:v
        - h264_cuvid
      inputs:
        # sub-stream, lower quality and FPS
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.1.112:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
        # full 4k stream
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.1.112:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
    # detect on the sub-stream for improved performance
    detect:
      width: 704
      height: 480
      fps: 5
    objects:
      track:
        - person
    record:
      # Optional: Enable recording
      enabled: True
      retain_days: 0
      events:
        retain:
          default: 10
    rtmp:
      # Required: Enable the live stream
      enabled: False
        # Optional: Configuration for the jpg snapshots written to the clips directory for each event
    snapshots:
      # Optional: Enable writing jpg snapshot to /media/frigate/clips
      # This value can be set via MQTT and will be updated in startup based on retained value
      enabled: True
      # Optional: print a timestamp on the snapshots
      timestamp: False
      # Optional: draw bounding box on the snapshots
      bounding_box: False
      # Optional: crop the snapshot
      crop: False
      # Optional: Camera override for retention settings
      retain:
        # Required: Default retention days
        default: 10
        # Optional: Per object retention days
        objects:
          person: 15
    # Optional: Configuration for the jpg snapshots published via MQTT
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
      # All other messages will still be published.
      enabled: True
      # Optional: print a timestamp on the snapshots
      timestamp: False
      # Optional: draw bounding box on the snapshots
      bounding_box: False
      # Optional: crop the snapshot
      crop: False

detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu
  cpu3:
    type: cpu
  cpu4:
    type: cpu

Relevant log output

[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2021-10-21 21:00:45] frigate.app                    INFO    : Starting Frigate (0.9.2-25bb515)
Starting migrations
[2021-10-21 21:00:45] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2021-10-21 21:00:45] peewee_migrate                 INFO    : There is nothing to migrate
[2021-10-21 21:00:45] frigate.mqtt                   INFO    : MQTT connected
[2021-10-21 21:00:45] detector.cpu1                  INFO    : Starting detection process: 225
[2021-10-21 21:00:45] detector.cpu2                  INFO    : Starting detection process: 227
[2021-10-21 21:00:45] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2021-10-21 21:00:45] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2021-10-21 21:00:45] detector.cpu3                  INFO    : Starting detection process: 229
[2021-10-21 21:00:45] detector.cpu4                  INFO    : Starting detection process: 231
[2021-10-21 21:00:45] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2021-10-21 21:00:45] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2021-10-21 21:00:45] frigate.app                    INFO    : Output process started: 233
[2021-10-21 21:00:45] ws4py                          INFO    : Using epoll
[2021-10-21 21:00:45] frigate.app                    INFO    : Camera processor started for inside: 237
[2021-10-21 21:00:45] frigate.app                    INFO    : Camera processor started for frontdoor: 239
[2021-10-21 21:00:45] frigate.app                    INFO    : Camera processor started for backdoor: 240
[2021-10-21 21:00:45] frigate.app                    INFO    : Capture process started for inside: 241
[2021-10-21 21:00:45] frigate.app                    INFO    : Capture process started for frontdoor: 248
[2021-10-21 21:00:45] frigate.app                    INFO    : Capture process started for backdoor: 250
[2021-10-21 21:00:45] ws4py                          INFO    : Using epoll
[2021-10-21 21:00:51] frigate.video                  INFO    : backdoor: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
[2021-10-21 21:00:51] frigate.video                  INFO    : backdoor: ffmpeg process is not running. exiting capture thread...
[2021-10-21 21:00:51] frigate.video                  INFO    : inside: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
[2021-10-21 21:00:51] frigate.video                  INFO    : inside: ffmpeg process is not running. exiting capture thread...
[2021-10-21 21:00:51] frigate.video                  INFO    : frontdoor: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
[2021-10-21 21:00:51] frigate.video                  INFO    : frontdoor: ffmpeg process is not running. exiting capture thread...
[2021-10-21 21:01:05] watchdog.inside                ERROR   : FFMPEG process crashed unexpectedly for inside.
[2021-10-21 21:01:05] watchdog.inside                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2021-10-21 21:01:05] watchdog.inside                ERROR   : You may have invalid args defined for this camera.
[2021-10-21 21:01:05] ffmpeg.inside.detect           ERROR   : [h264_cuvid @ 0x55572ac06900] Cannot load libnvcuvid.so.1
[2021-10-21 21:01:05] ffmpeg.inside.detect           ERROR   : [h264_cuvid @ 0x55572ac06900] Failed loading nvcuvid.
[2021-10-21 21:01:05] ffmpeg.inside.detect           ERROR   : Error while opening decoder for input stream #0:0 : Operation not permitted
[2021-10-21 21:01:05] watchdog.frontdoor             ERROR   : FFMPEG process crashed unexpectedly for frontdoor.
[2021-10-21 21:01:05] watchdog.frontdoor             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2021-10-21 21:01:05] watchdog.frontdoor             ERROR   : You may have invalid args defined for this camera.
[2021-10-21 21:01:05] ffmpeg.frontdoor.detect        ERROR   : [h264_cuvid @ 0x5642eb867980] Cannot load libnvcuvid.so.1
[2021-10-21 21:01:05] ffmpeg.frontdoor.detect        ERROR   : [h264_cuvid @ 0x5642eb867980] Failed loading nvcuvid.
[2021-10-21 21:01:05] ffmpeg.frontdoor.detect        ERROR   : Error while opening decoder for input stream #0:0 : Operation not permitted
[2021-10-21 21:01:05] watchdog.backdoor              ERROR   : FFMPEG process crashed unexpectedly for backdoor.
[2021-10-21 21:01:05] watchdog.backdoor              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2021-10-21 21:01:05] watchdog.backdoor              ERROR   : You may have invalid args defined for this camera.
[2021-10-21 21:01:05] ffmpeg.backdoor.detect         ERROR   : [h264_cuvid @ 0x559fa825a980] Cannot load libnvcuvid.so.1

FFprobe output from your camera

Can't get this to run presumably because of special characters in the password but I have gotten the camera to work before with frigate on just the CPU.

Frigate stats

No response

Operating system

Other Linux

Install method

Docker Compose

Coral version

CPU (no coral)

Network connection

Wireless

Camera make and model

Amcrest IP4M-1051W - 4MP

Any other information that may be helpful

My docker-compose.yml

version: '3.9'
services:
  frigate:
    container_name: frigate
    runtime: nvidia
    restart: unless-stopped
    privileged: true
    image: blakeblackshear/frigate:0.9.2-amd64nvidia
    devices:
      - /dev/bus/usb:/dev/bus/usb
      - /dev/dri/renderD128:/dev/dri/renderD128
      - /dev/vpu_service:/dev/vpu_service
      - /dev/dri/card0:/dev/dri/card0
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config:/config:ro
      - ./recordings/:/media/frigate/:rw
    ports:
      - 5000:5000/tcp # web-interface
      - 1935:1935/tcp # RTMP feeds
    environment:
      - FRIGATE_RTSP_PASSWORD=${FRIGATE_RTSP_PASSWORD}
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility,video

Proof of camera working and h264 setting via VLC:

The decoder is listed within the container:

root@81f7c8c4a1b6:/opt/frigate# ffmpeg -decoders | grep h264
ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-libsrt --enable-libaribb24 --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags='-I/opt/ffmpeg/include -I/opt/ffmpeg/include/ffnvcodec -I/usr/local/cuda/include/' --extra-ldflags='-L/opt/ffmpeg/lib -L/usr/local/cuda/lib64 -L/usr/local/cuda/lib32/'
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
 VFS..D h264
 V..... h264_v4l2m2m          (codec h264)
 V..... h264_cuvid            (codec h264)

Nvidia CUDA support is listed within the container:

root@81f7c8c4a1b6:/opt/frigate# ffmpeg -hwaccels
ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-libsrt --enable-libaribb24 --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags='-I/opt/ffmpeg/include -I/opt/ffmpeg/include/ffnvcodec -I/usr/local/cuda/include/' --extra-ldflags='-L/opt/ffmpeg/lib -L/usr/local/cuda/lib64 -L/usr/local/cuda/lib32/'
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Hardware acceleration methods:
cuda

Libraries within the container:

root@81f7c8c4a1b6:/opt/frigate# ldconfig -p | grep libnv
        libnvrtc.so.11.1 (libc6,x86-64) => /usr/local/cuda-11.1/targets/x86_64-linux/lib/libnvrtc.so.11.1
        libnvrtc-builtins.so.11.1 (libc6,x86-64) => /usr/local/cuda-11.1/targets/x86_64-linux/lib/libnvrtc-builtins.so.11.1
        libnvrtc-builtins.so (libc6,x86-64) => /usr/local/cuda-11.1/targets/x86_64-linux/lib/libnvrtc-builtins.so
        libnvjpeg.so.11 (libc6,x86-64) => /usr/local/cuda-11.1/targets/x86_64-linux/lib/libnvjpeg.so.11
        libnvblas.so.11 (libc6,x86-64) => /usr/local/cuda-11.1/targets/x86_64-linux/lib/libnvblas.so.11
        libnvToolsExt.so.1 (libc6,x86-64) => /usr/local/cuda-11.1/targets/x86_64-linux/lib/libnvToolsExt.so.1
        libnvToolsExt.so (libc6,x86-64) => /usr/local/cuda-11.1/targets/x86_64-linux/lib/libnvToolsExt.so

Docker and Nvidia passthrough working:

name@host:/dev/dri$ docker run --rm --gpus 'all,"capabilities=compute,utility"' nvidia/cuda:11.0-base nvidia-smi
Fri Oct 22 03:13:19 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.119.03   Driver Version: 450.119.03   CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 970     Off  | 00000000:01:00.0 Off |                  N/A |
|  0%   30C    P8    14W / 201W |     40MiB /  4034MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+