frigate: [Config Support]: Can't get Nvidia hardware acceleration to work

Describe the problem you are having

Adding hwaccel_args: -c:v h264_cuvid to a camera’s config breaks FFmpeg for that camera. Without it, everything works fine, but all CPU cores are at 100% and the load average on the system is very high.

Version

v0.11.0-rc1

Frigate config file

mqtt:
  host: redacted
  user: mosquitto
  password: redacted

detectors:
  coral0:
    type: edgetpu
    device: "pci:0"
  coral1:
    type: edgetpu
    device: "pci:1"
  coral2:
    type: edgetpu
    device: "pci:2"
  coral3:
    type: edgetpu
    device: "pci:3"
  coral4:
    type: edgetpu
    device: "pci:4"
  coral5:
    type: edgetpu
    device: "pci:5"
  coral6:
    type: edgetpu
    device: "pci:6"
  coral7:
    type: edgetpu
    device: "pci:7"

birdseye:
  mode: continuous

objects:
  track:
    - person
    - bicycle
    - car
    - motorcycle
    - bus
    - boat
    - bird
    - cat
    - dog
    - frisbee
    - sports ball
    - kite
    - skateboard
    - bottle
    - cell phone

cameras:
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
      hwaccel_args: -c:v h264_cuvid
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2592
      height: 1944
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@redacted:554/profile2/media.smp
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 1920
      height: 1080
      fps: 25
  redacted:
    ffmpeg:
      inputs:
        - path: rtsp://redacted:redacted@redacted:554/stream1
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 1920
      height: 1080
      fps: 15

record:
  enabled: True
  retain_days: 14
  events:
    retain:
      default: 365

snapshots:
  enabled: True
  timestamp: True
  bounding_box: True

timestamp_style:
  format: "%Y-%m-%d %H:%M:%S"

Relevant log output

[2022-08-09 12:11:12] frigate.video                  ERROR   : redacted: Unable to read frames from ffmpeg process.
[2022-08-09 12:11:12] frigate.video                  ERROR   : redacted: ffmpeg process is not running. exiting capture thread...
[2022-08-09 12:11:29] watchdog.redacted       ERROR   : Ffmpeg process crashed unexpectedly for redacted.
[2022-08-09 12:11:29] watchdog.redacted       ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-08-09 12:11:29] ffmpeg.redacted.detect  ERROR   : [h264_cuvid @ 0x55dbb6278300] Cannot load libnvcuvid.so.1
[2022-08-09 12:11:29] ffmpeg.redacted.detect  ERROR   : [h264_cuvid @ 0x55dbb6278300] Failed loading nvcuvid.
[2022-08-09 12:11:31] frigate.video                  ERROR   : redacted: Unable to read frames from ffmpeg process.
[2022-08-09 12:11:31] frigate.video                  ERROR   : redacted: ffmpeg process is not running. exiting capture thread...

Frigate stats

No response

Operating system

Other Linux

Install method

Docker Compose

Coral version

PCIe

Any other information that may be helpful

docker-compose.yml:

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: blakeblackshear/frigate:0.11.0-rc1
    shm_size: "1536mb"
    devices:
      - /dev/apex_0:/dev/apex_0
      - /dev/apex_1:/dev/apex_1
      - /dev/apex_2:/dev/apex_2
      - /dev/apex_3:/dev/apex_3
      - /dev/apex_4:/dev/apex_4
      - /dev/apex_5:/dev/apex_5
      - /dev/apex_6:/dev/apex_6
      - /dev/apex_7:/dev/apex_7
      - /dev/dri/card0
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/ehn/frigate/config.yml:/config/config.yml:ro
      - /media/datastore/frigate:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935" # RTMP feeds
    environment:
      FRIGATE_RTSP_PASSWORD: "redacted"
      PLUS_API_KEY: "redacted"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Output from nvidia-smi (in the container):

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA RTX A2000    Off  | 00000000:65:00.0 Off |                  Off |
| 30%   41C    P0    24W /  70W |      0MiB /  6138MiB |      2%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 29

Most upvoted comments

@ehn I’m curious, what “Inference Speed” do you get with your A2000 GPU? I get around 10ms on my Coral USB, wondering if getting an A2000 would be better as I could also get H264 encode/decode.

@dragozet I’m not using the GPU for object detection, only video decoding. I have 8 TPUs in this machine.

ehn on Jan 22, 2023

Figured it out. The library was simply missing from the host. Installing the libnvidia-decode-515-server package resolved the issue. Thanks, @NickM-27, for linking to https://github.com/jellyfin/jellyfin/issues/2395! That helped me find the underlying problem.

This is on Ubuntu 22.04 btw, in case someone else has a similar problem.

Based on the output from nvidia-smi, it looks like 13 video streams are being successfully decoded on the GPU. All 20 CPU cores are still at close to 100%, and the system load is 40–60, however.

ehn on Aug 9, 2022