amazon-kinesis-video-streams-webrtc-sdk-c: [QUESTION] Kinesis Video Streams with WebRTC browser stops displaying new frames when using TURN with large frame sizes

I am working on a project that requires sending very large video (about 400KB per frame) at 1fps in as close to real time as possible from a server to one or more web clients. I was able to modify the C SDK sample master to send the video with an acceptable latency to a browser viewer, but am running into a strange issue specifically when using TURN;

After 181 frames the video feed stops displaying new frames.

  • According to nettop and chrome://webrtc-internals/ the viewer is still receiving the new frames, as the bytes-in continues to rise at the same rate until either the viewer or server is stopped, but the web browser stops displaying them. This was tested with both Google Chrome and Firefox, on both the JS SDK sample and the AWS console video viewer with the same result.
  • The number of frames it displays does not seem to be affected by the frame rate, length of the video, or elapsed time. However it does seem to be affected by the size of the frames. If I use a more compressed version of the video (approx. 150KB per frame) the number changes from 181 frames to 505 frames. It does not stop at a consistent total size though, the 181 frame test totals 74.4MB and the 505 frame test totals 83MB.
  • The issue seems to stop occurring if the frame size is reduced to at or below 100KB on average (in the case of my test video that happens around CRF 28), at which point the quality stops being ideal for our use case
  • This shouldn’t be an issue of going over the 5Mbps TURN server quota, as I am only hitting at most 3Mbps.
  • I also can’t find any information about the TURN server having a frame size limitation
  • Disabling NAT traversal and just running the test localhost to localhost causes the issue to disappear, and I am unable to find a consistent setup to test this with STUN.
  • I get no error messages from either end in these tests
  • The videos are h.264 encoded using ffmpeg (CRF 15 ultrafast, yuv 4:2:0) with no audio

Here is some webrtc-internals data from a different run that got 300 frames in: Pasted image 20230524141420 In this graph frames received / decoded per second is the red and bytes received per second is the blue line, you can see that the channel is still receiving data, it just stops decoding it.

and here: Pasted image 20230524141444 In this graph frames received / decoded per second is the red, packets received per second is the yellow line and total packets lost is the purple line. The total number of lost packets steadily increases after the video stops, but it does not come close to the number of packets still coming in per second.

This can be repro’d with only a couple simple changes to the master (just changes necessary to point it to a new video):

  • NUMBER_OF_H264_FRAME_FILES in Samples.h changed to 67, the length of my test video
  • DEFAULT_FPS_VALUE in Samples.h changed to 1
  • SAMPLE_STATS_DURATION in Samples.h changed to ((NUMBER_OF_H264_FRAME_FILES / DEFAULT_FPS_VALUE) * HUNDREDS_OF_NANOS_IN_A_SECOND)
  • encoderStats.width on line 262 of kvsWebRTCClientMaster.c set to 1024
  • encoderStats.height on line 263 of kvsWebRTCClientMaster.c set to 1024
  • encoderStats.targetBitrate on line 264 of kvsWebRTCClientMaster.c set to 4000000 or 4 Mbps
  • A sample video with frame sizes around 400KB each

The test video is created from a set of images with the following two ffmpeg commands from a small set of images

ffmpeg -y -loglevel error -hide_banner -f image2 -i "input/frame-%04d.png" -profile:v baseline -c:v libx264 -crf 15 -preset ultrafast -pix_fmt yuv420p "output-full.h264"
ffmpeg -y -loglevel error -hide_banner -i "output-full.h264" -f image2 -vcodec copy -r 1 "output/frame-%04d.h264"

I am at a bit of a loss as to how to even debug this issue further. Can anyone provide either a suggestion as to why this might be happening, or other steps I can take to further diagnose this issue?

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 22 (12 by maintainers)

Most upvoted comments

To run the provided .tar file for the docker container

  1. docker load -i kvs-turn-frame-bug-test.tar
  2. docker run --detach -e AWS_ACCESS_KEY_ID=<access_key> -e AWS_SECRET_ACCESS_KEY=<secret_key> -e AWS_DEFAULT_REGION=us-east-2 kvs-turn-frame-bug-test

it will create a channel called test_channel on the account provided, and will start streaming when a viewer connects.