xpra-html5: native video decoding errors

Sometime since e0e5de35b63eab7c5a6a6ff60827c7af0e77c2f5 (2016!)

I have added an example to enable fastdecode: ca3d1db4d9a242600bc5d86019be027ab9645617 And the xpra server will now honour it, both with h264+mp4 and mpeg4+mp4: https://github.com/Xpra-org/xpra/commit/b10da3f3291d492839843e12a08359c40cd40f8a

Note: to change which actual codec the server will choose for the html5 client in “native” video mode, using the “score-delta” for the codec chosen: 8cf87c2035ef5fca158e1126fdf362bd642f1564 (h264+mp4 is now ahead of mpeg4+mp4).


This command generates an h264 stream that browsers can play:

ffmpeg -f x11grab -i :99 -c:v libx264 -pix_fmt yuv420p \
    -profile:v baseline -tune fastdecode -preset ultrafast \
    -f mp4 -movflags frag_keyframe+default_base_moof -frag_duration 1 \
    pipe:1

Link: Fragmented MP4 - problem playing in browser

Comparing it with our enc_ffmpeg encoder:


There are a lot more fastdecode tweaks in the plain enc_x264 encoder: https://github.com/Xpra-org/xpra/blob/b10da3f3291d492839843e12a08359c40cd40f8a/xpra/codecs/enc_x264/encoder.pyx#L594-L597 https://github.com/Xpra-org/xpra/blob/b10da3f3291d492839843e12a08359c40cd40f8a/xpra/codecs/enc_x264/encoder.pyx#L634-L638 removed B frames, sliced threads, b_weighted_bipred, i_weighted_pred. Perhaps these tweaks would help with the enc_ffmpeg encoder?


Here is the cryptic browser error:

Error: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source.
    at XpraWindow._push_video_buffers (http://localhost:12000/js/Window.js:1223:7)
    at XpraWindow.paint [as do_paint] (http://localhost:12000/js/Window.js:1592:10)
    at XpraWindow.paint (http://localhost:12000/js/Window.js:1321:33)
    at XpraClient.do_process_draw (http://localhost:12000/js/Client.js:2990:7)
    at Worker.<anonymous> (http://localhost:12000/js/Client.js:433:7)
error painting mpeg4+mp4 InvalidStateError: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 19

Commits related to this issue

Most upvoted comments

We did some internal testing and the speed of the native Video- and AudioDecoders is very good. Combined with offscreen painting ther performance is excellent. Needs some work; I tuned ACK_DELAY and ACK_JITTER with env. variables. This needs to be reported reliable from the HTML5 client. Maybe using SetInterval and check the queue for old packages may be the way to go.

It seems that our best bet might be the MediaStreamTrackProcessor (https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrackProcessor) This interface can be used to capture video playback frame-by-frame. PofA:

  • Feed the MSE video chunks.
  • Set the MSE as source of the video element
  • Get the stream: this.video.captureStream().getVideoTracks()[0]
  • Use the MediaStreamTrackProcessor to process the stream.
  • Use TrackProcessor.readable.getReader() and TrackReader.read()
  • Get a bitmap of the processed frame with the right widht & heigt set
  • Get x & y coordinate from the package
  • Paint
  • Process next package.

Got the first 6 steps working, will commit the code after the last three steps. This will not work on older browser, but they can fall back to broadway or non-video.