tfjs: TFJS WebGL on WebWorker still blocks GUI
After investigating a report by a user who was attempting to use web workers + tfjs to reduce lagging of GUI when model performs inference I have noticed that when using backend “webgl” web worker makes no effect on executing code in a separate context as this does not apply to GPU.
I have confirmed that by setting backend to “cpu” there is no performance issue, as the TFJS execution correctly is executed on a new thread, such that the browser DOM updates are not interfered with. I have also confirmed that browser relies on GPU to update DOM for anything visual - not CPU.
Thus the request here is how to limit “webgl” execution to leave enough processing for DOM updates for other user tasks to prevent this “jankiness” from occuring.
Demo of issue:
https://codepen.io/jasonmayes/project/editor/DBYaRj
Simply change first line of tfWorker.js to import one of:
importScripts('dist/cpu.js');
or for WebGL backend change to:
importScripts('dist/main.js');
Confirmed this issue across devices including Windows 10, Desktop, Chrome and also Xiaomi 8 (Android phone with Snapdragon 845 processor)
Example output from Chrome Dev tools shows GPU block when using WebGL backend and also that all execution comes from WebWorker.js.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 19 (7 by maintainers)
So the WebGL part that was blocking should get for free with latest build of TFJS. See our announcement at Google IO last year 2022: https://youtu.be/GbgjafMdAIs?t=762 Thanks @lina128 for the fix.
So the team have been researching into how to reduce GPU load when loading models / inference to give time back to browser render to reduce freezing of GUI etc. It seems to be more about how the browser schedules tasks to GPU. From what I understand WebWorker is aimed at CPU multithreading, and does not account for GPU tasks that are spawned from it which are still shared with the main browser level GPU render thread it seems. However focusing on GPU sharing instead we have had some promising results but still working on refining publishable solution to prod. This is actively under investigation though.
Hello, @jasonmayes, @lina128 ! Thank you very much for your work and for the wonderful library you build. Do you have any updates regarding this topic? Any approximate estimates or early builds?
I shall let @lina128 reply as she is currently working on the final solution to this which looks very promising in terms of that initial load blocking issue you mentioned by allowing time back to the browser render thread to do what it needs to do for updating GUI, and maybe she has some thoughts on q2 too.
Essentially though:
In the WebGL API, some methods are blocking, while others are not. These are different from the sync/async concept in JS. The blocking methods in WebGL mean that, certain WebGL entry points cause synchronous stalls on the calling thread (possibly the same calling thread Chrome uses for GUI rendering I believe as everything is rendered via GPU now on Chrome if I remember correctly). Even basic requests can take as long as 1ms, but they can take even longer if they need to wait for all graphics work to be completed.
In the new version Na is working on, we first compile and link everything without waiting because they are async, and then check and wait everything at the end, instead of individually, which gives time back to the browser to do other things it needs to perform in that time frame too.
So this new fix should be out shortly - please tune into our TensorFlow.js updates in May, that is all we can say for now but we look forward to hearing your feedback once you get a chance to try it.
I made a slightly simplified version of the code to the demo the original reporter created here: https://codepen.io/jasonmayes/project/editor/DBYaRj
Executing this - click the big button at the bottom - and you will notice the webcam feed freezes as do the red numbers rendered to the left of the button.
If I change the TFJS backend to CPU it does not have this “lag”.
I am unsure of the exact reason that is causing this, if you have suggestions of what it could be please do feel free to suggest, but the only difference here is the backend being used indicating something is not sharing the GPU as intended when using WebGL backend.
I will follow up with the team as to how WebGL execution works in TFJS when called from a WebWorker context in case something needs to be changed there.