frigate: [Support]: Beta2 FFmpeg QSV HW Acceleration and audio not working on Reolink Cams

HTTP streams with hwaccel and audio are working with this Frigate 0.11-beta2 config :

detect:
  width: 896
  height: 672
  fps: 5

ffmpeg:
  hwaccel_args: -hwaccel qsv -hwaccel_output_format nv12 -c:v h264_qsv
  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -strict experimental -analyzeduration 1000M -probesize 1000M -rw_timeout 5000000
  output_args:
    record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy

cameras:
  Balcony:
    ffmpeg:
      inputs:
        - path: http://192.168.1.76/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.76/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record

EDIT2: Updated ffmpeg block as below, for a complete working config.

One more update for smoother clips; similar/same as they were in 0.10, vs the above configs that produced clips full of stutter.

ffmpeg:
  hwaccel_args: -hwaccel qsv -hwaccel_output_format nv12 -c:v h264_qsv
  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -strict experimental -analyzeduration 1000M -probesize 1000M -rw_timeout 5000000
  output_args:
    record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy

Three differences from my 0.10 config, but multiple hours of trying arguments to work that out. 🤦‍♂️

Or, as per @minglarn’s post above, use this for hardware accel using VAAPI: hwaccel_args: -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format yuv420p

I found something interesting related to iframes aka keyframes and http vs rtsp.

I have a reolink duo gen1 and an 810A and this thread was invaluable to getting things working great.

The newer firmware versions for these cameras have the option to set the “Interframe Space” value from 1x to 4x. I understand keyframes and fps and all that, but I wasn’t exactly sure what “Interframe Space” meant in reolink’s setting.

I used ffprobe to see how changing that value would affect the keyframe interval using: ffprobe -loglevel error -select_streams v:0 -show_entries packet=pts_time,flags -of csv=print_section=0 "http://192.168.1.150/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=[redacted]&password=[redacted]" But, I found that changing “Interframe Space” made no difference to the keyframe interval. With my fps set to 15 the keyframes were occuring once every other second, and then I noticed that it was actually outputting at 20 fps, not 15. After trying a few other settings the fps and keyframe interval didn’t change.

I tried the same ffprobe command but pointed it to my rtsp url. ffprobe -loglevel error -select_streams v:0 -show_entries packet=pts_time,flags -of csv=print_section=0 "rtsp://[redacted]:[redacted]@192.168.1.150:554/h264Preview_01_sub" Using rtsp, the keyframes are once per second and the fps was what I actually set it to. So, I set my conf to use the http url for recording and the rtsp stream for detect. We’ll see how it goes but it seems fine so far.

In the end, I think their “Interframe Space” setting really means number of seconds per keyframe. So, 1x would be 1 keyframe/sec, 4x would be 1 keyframe/4sec. Also, the http url doesn’t seem to adhere to the fps/keyframe settings, at least for the substream. This issue does only seem to be on the substream as the main stream was recorded via http and my videos have their keyframes 1/sec and recording at 15fps.

Can someone provide a full working config of a Reolink POE doorbell?

https://docs.frigate.video/configuration/camera_specific#reolink-cameras

Yep, I have the drivers installed. Looks like I may have just had too many args in the hwaccel_args line.

Fingers crossed, this is what appears to be working for now:

  GARAGE:
    ffmpeg:
      hwaccel_args: -c:v h264_cuvid
      input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -strict experimental -analyzeduration 1000M -probesize 1000M -rw_timeout 5000000
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
      inputs:
        - path: rtmp://192.168.1.105/bcs/channel0_sub.bcs?channel=0&stream=1&user=USER&password=PASS
          roles:
            - rtmp
        - path: rtmp://192.168.1.105/bcs/channel0_main.bcs?channel=0&stream=1&user=USER&password=PASS
          roles:
            - detect
            - record

Originally I had quite a few things and now it is just stripped back to one for the hwaccel_args

hwaccel_args: -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format yuv420p

Th

One more update for smoother clips; similar/same as they were in 0.10, vs the above configs that produced clips full of stutter.

ffmpeg:
  hwaccel_args: -hwaccel qsv -hwaccel_output_format nv12 -c:v h264_qsv
  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -strict experimental -analyzeduration 1000M -probesize 1000M -rw_timeout 5000000
  output_args:
    record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy

Three differences from my 0.10 config, but multiple hours of trying arguments to work that out. 🤦‍♂️

Or, as per @Minglarn’s post above, use this for hardware accel using VAAPI: hwaccel_args: -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format yuv420p

Thanks @jasonpstokes This does improve the recording from Reolink 410 playback even without hardware accel which doesn’t work on my system with the beta version and I don’t notice much cpu utilization difference.

Unfortunately it seems this only applies to their newer cameras. A lot of people I’ve seen on here have the RLC-410 or something from that era, which Reolink has now end of lifed and has said they will not fix the issues with it (even though the issues were apparent while they were still selling them a few months ago).

I just bit the bullet on 4 new Loryta 5442s and will be putting all of my reolinks on ebay if anyone is interested. I’m done with that company.

Just as another note, just saw this video which shows new reolink rtsp version / settings that may give another option and hopefully fix some issues with reolink cams in general

For what it’s worth I’m running 5 Reolink cameras using RTSP h.265 without a problem.

I have 3 x 811’s at 3380x2160 h.265, 1 x Duo Floodlight at 4096x1536 h.265 and 1 x WifiDoorbell at 2560x1920 h.264. Hardware is a $200 Mini PC with at Celeron N5095A and a Coral USB module. Cameras are all set to 15fps with 1x keyframe (except the duo which won’t do 1x). Detect is set to 5fps.

This is how I have it setup, I’m converting the h.265 to 264 in go2rtc which avoids the playback issues with h.265 - Intel Quicksync is carrying most of the load here.

go2rtc:
  streams:
    username: "viewer"
    password: "xxxxxxx"
    frontyard: 
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.201.102:554/h265Preview_01_main
      - "ffmpeg:frontyard#video=h264#hardware"
    garage:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.201.107:554/h265Preview_01_main
      - "ffmpeg:garage#video=h264#hardware"
    sideyard:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.201.104:554/h265Preview_01_main
      - "ffmpeg:sideyard#video=h264#hardware"
    doorbell:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.201.108:554/h264Preview_01_main
    viewcam:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.201.105:554/h265Preview_01_main
      - "ffmpeg:viewcam#video=h264#hardware"
detectors:
  coral:
    type: edgetpu
    device: usb

ffmpeg:
  hwaccel_args: preset-vaapi

birdseye:
  enabled: True
  mode: continuous
  restream: False
  width: 1920
  height: 1080
  quality: 4
   
rtmp:
  enabled: false

detect:
  fps: 5
  enabled: true

Then for each camera I’m using the full res stream, example:

  garage:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage
          roles:
            - detect   
            - record
    detect:
      width: 4096
      height: 1536

This is what the system load looks like. I’m fine on coral and graphics but running at the limit for CPU so if I add any more cameras I might have to upgrade to a $400 mini pc. 😃

This works fine for my Reolink 811A 4K camera. Thanks for sharing. Have you been able to get audio working as well? I’ve added to the ffmpeg transcoding of the go2rtc.streams.camera section, but to no avail:

go2rtc:
  rtsp:
    #The go2rtc restream can be secured with RTSP based username / password authentication. Ex:
    #NOTE This does not apply to localhost requests, there is no need to provide credentials when using the restream as a source for frigate cameras.
    username: "rtsp_frigate_restream"
    password: "bladiebla"
  streams:
    # dit is RTSP (zie port 554)
    camera_terras:
    - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.70.12:554/h265Preview_01_main
      #dit is een kopie van de stream. which transcodes video to 264
      #07/12/23 audio toegevoegd. geen idee of het werkt.
    - ffmpeg:camera_terras#video=h264#hardware#audio=opus

Update: got audio working. Had to enable audio recording in camera settings.

these args should not be used anymore, in general preset-vaapi should be used https://docs.frigate.video/configuration/hardware_acceleration

Do we just add a -audio role to the record stream?

you could but that would be wasteful as it would be asking go2rtc to send another copy of the record stream including the video which won’t be used anyway. It would be better to do:

go2rtc:
  streams:
    your_reolink_camera:
      - "ffmpeg:http://reolink_ip/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=username&password=password#video=copy#audio=copy#audio=opus"
    your_reolink_camera_sub:
      - "ffmpeg:http://reolink_ip/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=username&password=password"

cameras:
  your_reolink_camera:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/your_reolink_camera
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/your_reolink_camera_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/your_reolink_camera_sub?audio
          input_args: preset-rtsp-restream
          roles:
            - audio

That is correct

For what it’s worth I’m running 5 Reolink cameras using RTSP h.265 without a problem.

I have 3 x 811’s at 3380x2160 h.265, 1 x Duo Floodlight at 4096x1536 h.265 and 1 x WifiDoorbell at 2560x1920 h.264. Hardware is a $200 Mini PC with at Celeron N5095A and a Coral USB module. Cameras are all set to 15fps with 1x keyframe (except the duo which won’t do 1x). Detect is set to 5fps.

This is how I have it setup, I’m converting the h.265 to 264 in go2rtc which avoids the playback issues with h.265 - Intel Quicksync is carrying most of the load here.

go2rtc:
  streams:
    username: "viewer"
    password: "xxxxxxx"
    frontyard: 
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.201.102:554/h265Preview_01_main
      - "ffmpeg:frontyard#video=h264#hardware"
    garage:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.201.107:554/h265Preview_01_main
      - "ffmpeg:garage#video=h264#hardware"
    sideyard:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.201.104:554/h265Preview_01_main
      - "ffmpeg:sideyard#video=h264#hardware"
    doorbell:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.201.108:554/h264Preview_01_main
    viewcam:
      - rtsp://viewer:{FRIGATE_RTSP_PASSWORD}@192.168.201.105:554/h265Preview_01_main
      - "ffmpeg:viewcam#video=h264#hardware"
detectors:
  coral:
    type: edgetpu
    device: usb

ffmpeg:
  hwaccel_args: preset-vaapi

birdseye:
  enabled: True
  mode: continuous
  restream: False
  width: 1920
  height: 1080
  quality: 4
   
rtmp:
  enabled: false

detect:
  fps: 5
  enabled: true

Then for each camera I’m using the full res stream, example:

  garage:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage
          roles:
            - detect   
            - record
    detect:
      width: 4096
      height: 1536

This is what the system load looks like. I’m fine on coral and graphics but running at the limit for CPU so if I add any more cameras I might have to upgrade to a $400 mini pc. 😃

Please create a new issue. Everyone who has contributed to this thread will be getting notifications 😃

@jasonpstokes I bugged reolink support until they sent me a beta firmware for my RLC-810A.

I sent it over to this repo, which keeps track of a lot of reolink firmware versions https://github.com/AT0myks/reolink-fw-archive

What is the best argument to use for Intel NUC : Intel® Core™ i5-7300U CPU (with Intel® HD Graphics 620 ) Kaby Lake U

ffmpeg:

hwaccel_args: -c:v h264_qsv

or

ffmpeg:

hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p

I tried both in turn, but did not understand which is better. Same CPU load (avg +/-10%)

Thanks a lot!

They’re more or less the same, use qsv and if it has problems use vaapi

Unfortunately in higher resolutions, http has quite a big delay (8-12s) compared to rtsp (2-4s). Also, the high resolution delay is closer to the low resolution delay in rtsp recordings compared to http, so if you’re using the low resolution for detection and the high resolution for recording, you are going to have a bad time in http where objects just left the recording. These observations were made with a couple of Reolink E1 Zoom PTZ ONVIF cameras.

How are you measuring that delay?

That’s an older camera with old firmware. Their newer cameras have more options like setting iframe interval and CBR.

I have been using reolinks new doorbell as well as the 511wa and on both of them rtsp or http has virtually no delay. I have their http streams setup in go2rtc and can watch via WebRTC while looking out my window and things are the same as real time.

Are these delays and smears common properties of video feeds, or is this typical of Reolink?

I’ve never heard of delay being an issue. The smearing on rtsp is typical reolink due to their bad rtsp engine that they use, though cameras with newer firmware have improved on this front too.

Does anyone have any new insights as to what is the best approach?

Like I said above I’ve had great results with http on these two newer reolinks.

@DrSpaldo see my (now updated) config above for my running config using http streams, which (appears to) give me the same recordings as 0.10. Let me know how it works for you?

@Minglarn what are your recordings like using RTMP? frps=fps=5 ? And my guess is that fps=fps=5 is a copy and paste error in ffmpeg’s docs, as every other example only has fps=x??? But as Frigate’s command contains -r x to force the frame rate based on the detect config block, this may be redundant anyway?

frps is a typo… It should be: fps=fps=5 Edit: found the video where Linus is showing his config for Frigate. https://www.youtube.com/watch?v=B635wcdr6-w It’s about 7:20 into the video.

Thanks @Minglarn & @jasonpstokes for posting the configs. I’ve got the RLC-410 & RLC-511. So I’ll have a look today and see what difference using your config/s goes. Once that’s up & running then I’ll also try to get my nvidia acceleration working as well 😃

Thanks @NickM-27 will do - will learn how! - after running it for a few days to confirm, plus to see what feedback others have.

Can confirm that THIS works with ReoLink rtmp streams. Nice! Thanks for finding out the right settings… Edit: Works with both 510A, 520A and 410 models… 👍

Turns out hardware acceleration AND audio both work for RTSP and RTMP streams, but not for HTTP streams. 🤷‍♂️ (see updated config here ) At least with my Reolink RLC-520A cameras - anyone else seeing this?

e.g. working RTSP stream config:

detect:
  width: 640
  height: 480
  fps: 5

ffmpeg:
  hwaccel_args: -hwaccel_output_format qsv -c:v h264_qsv
  output_args: # record audio
    record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy # -an

cameras:
  Balcony:
    ffmpeg:
      inputs:
        - path: rtsp://{FRIGATE_USER}:{FRIGATE_PASSWORD}@192.168.1.76:554//h264Preview_01_sub
          roles:
            - detect
        - path: rtsp://{FRIGATE_USER}:{FRIGATE_PASSWORD}@192.168.1.76:554
          roles:
            - record

e.g. working RTMP stream config (RTMP has the advantage of a ‘Balanced’ stream option for better detect/live view resolution and, for me at least, less delay when viewing a live stream, vs. RTSP)

detect:
  width: 896
  height: 672
  fps: 5

ffmpeg:
  hwaccel_args: -hwaccel_output_format qsv -c:v h264_qsv
  input_args: -avoid_negative_ts make_zero -flags low_delay -fflags discardcorrupt -strict experimental -rw_timeout 5000000 -use_wallclock_as_timestamps 1 -f live_flv
  output_args:
    record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy # -an

cameras:
  Balcony:
    ffmpeg:
      inputs:
        - path: rtmp://192.168.1.76/bcs/channel0_ext.bcs?channel=0&stream=2&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: rtmp://192.168.1.76/bcs/channel0_main.bcs?channel=0&stream=0&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record

GPU working again:

EDIT: updated input_args to closer match the recommended ones in the docs.

There’s likely two different issues here: hardware acceleration for Intel QSV with Jellyfin FFmpeg using VAAPI, and then finding the new correct arguments to get a stable stream with audio for the (troublesome) Reolink cameras.

The latest camera firmware lists updates to RTSP and stream fixes, so I’ll try it and different protocols/arguments over the weekend. (Hopefully it’s raining…!)

Really like the direction of 11 and especially the UI improvements.