xpra-html5: native video decoding errors
Sometime since e0e5de35b63eab7c5a6a6ff60827c7af0e77c2f5 (2016!)
I have added an example to enable fastdecode
: ca3d1db4d9a242600bc5d86019be027ab9645617
And the xpra server will now honour it, both with h264+mp4
and mpeg4+mp4
:
https://github.com/Xpra-org/xpra/commit/b10da3f3291d492839843e12a08359c40cd40f8a
Note: to change which actual codec the server will choose for the html5 client in “native” video mode, using the “score-delta” for the codec chosen: 8cf87c2035ef5fca158e1126fdf362bd642f1564 (h264+mp4
is now ahead of mpeg4+mp4
).
This command generates an h264 stream that browsers can play:
ffmpeg -f x11grab -i :99 -c:v libx264 -pix_fmt yuv420p \
-profile:v baseline -tune fastdecode -preset ultrafast \
-f mp4 -movflags frag_keyframe+default_base_moof -frag_duration 1 \
pipe:1
Link: Fragmented MP4 - problem playing in browser
Comparing it with our enc_ffmpeg
encoder:
- we use
YUV420P
(orNV12
withlibva
- which translates to the same subsampling) baseline
profile since 8cf87c2035ef5fca158e1126fdf362bd642f1564fastdecode
since b10da3f3291d492839843e12a08359c40cd40f8aultrafast
preset- movflags:
empty_moov+frag_keyframe+default_base_moof+faststart
- IIRC, faststart is important too
There are a lot more fastdecode
tweaks in the plain enc_x264
encoder:
https://github.com/Xpra-org/xpra/blob/b10da3f3291d492839843e12a08359c40cd40f8a/xpra/codecs/enc_x264/encoder.pyx#L594-L597
https://github.com/Xpra-org/xpra/blob/b10da3f3291d492839843e12a08359c40cd40f8a/xpra/codecs/enc_x264/encoder.pyx#L634-L638
removed B frames, sliced threads, b_weighted_bipred, i_weighted_pred.
Perhaps these tweaks would help with the enc_ffmpeg
encoder?
Here is the cryptic browser error:
Error: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source.
at XpraWindow._push_video_buffers (http://localhost:12000/js/Window.js:1223:7)
at XpraWindow.paint [as do_paint] (http://localhost:12000/js/Window.js:1592:10)
at XpraWindow.paint (http://localhost:12000/js/Window.js:1321:33)
at XpraClient.do_process_draw (http://localhost:12000/js/Client.js:2990:7)
at Worker.<anonymous> (http://localhost:12000/js/Client.js:433:7)
error painting mpeg4+mp4 InvalidStateError: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 19
Commits related to this issue
- #114 remove mediasource video support — committed to Xpra-org/xpra-html5 by totaam 2 years ago
We did some internal testing and the speed of the native Video- and AudioDecoders is very good. Combined with offscreen painting ther performance is excellent. Needs some work; I tuned ACK_DELAY and ACK_JITTER with env. variables. This needs to be reported reliable from the HTML5 client. Maybe using SetInterval and check the queue for old packages may be the way to go.
It seems that our best bet might be the MediaStreamTrackProcessor (https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrackProcessor) This interface can be used to capture video playback frame-by-frame. PofA:
Got the first 6 steps working, will commit the code after the last three steps. This will not work on older browser, but they can fall back to broadway or non-video.