brave-browser: WebTorrent should not store torrent data in memory
Description
WebTorrent seems to be keeping the entire torrent in memory as the file is downloaded. I can see the RAM usage of the “Brave Browser Helper” process increase along with the amount of the torrent that is downloaded, which should not happen. It appears that it is being initialized with the in-memory storage mode instead of the filesystem mode. This is because webpack is used to build the JS for the WebTorrent extension and that causes WebTorrent to substitute certain packages using the package.json “browser” field.
May be related to: https://github.com/brave/brave-browser/issues/5372
Steps to Reproduce
- Visit http://mirror.math.princeton.edu/pub/mageia/iso/7.1/torrents/Mageia-7.1-x86_64.torrent
- Start the torrent download
- Open
Activity Monitor.appon Mac and observe memory usage climbing in proportion to the amount of torrent that is downloaded.
Actual result:
Torrent is stored in memory instead of on the disk.
Expected result:
Torrent should not consume RAM in proportion to the torrent size, since this means that users with e.g. 8GB RAM are going to have a sad time when they download an 8GB+ torrent.
Reproduces how often:
Easily reproduced
Brave version (brave://version info)
All Chromium versions.
About this issue
- Original URL
- State: open
- Created 5 years ago
- Reactions: 20
- Comments: 30 (3 by maintainers)
Hopefully, in the next few weeks I might be able to take a look, no promise though. This is on my todo list 😄
This should probably be prioritized and fixed soon. Torrent feature becomes useless if the browser starts to freeze up and download is lost.
cc: @rebron @bsclifton @feross
It has been almost 4 years now, and absolutely nothing has been done about this since then. I am forced to use Transmission whenever I download anything over 10GB. Why is this feature still here if it’s neglected and flawed.
I disagree. I think that as someone who uses torrents frequently just like a lot of people in this community, this feature is valuable to us.
Hi everyone,
Thanks for all the feedback. I have a plan for fixing this issue and reducing RAM usage.
We’ll use a chunk store that stores data to disk instead of to memory. Many of the existing chunk store options available for use in the browser have disappointing performance, which is why I haven’t wanted to make the change yet. The IndexedDB chunk stores in particular have historically had atrocious performance.
I’ve done a thorough investigation of the existing stores and discovered many low-hanging performance improvements in the chunk stores, but also in the core WebTorrent library itself. It’s now looking like we can reasonably use one of these and still get good performance.
Finally, I’ve been developing a new chunk store based on the new Filesystem Access API which has shipped in Chromium-based browsers. It performs almost as good as the
memory-chunk-store(the current solution) but, of course, uses no RAM.The only remaining issue to figure out is how to handle clearing the data. We’ll need to make some decisions about when is the right time to do that – when the user closes the tab? when they stop the torrent? when they explicitly click “delete”?
Webtorrent is one of the worst integrations of anything I have ever seen.
Since it’s open source, does that mean one of us is eventually supposed to fix it?
Nobody wants it perhaps due to the memory issue discussed in this thread? Fixing that by supporting disk downloads instead would change that.
My interest is from the opposite side of a user, a community website with large downloads (game mods) where we can offload bandwidth expenses through torrents. Some users may not be comfortable with third-party torrent software and it’s easier to support them via webtorrents and advising a client like Brave making it seamless in the users browsing experience.
For other users comfortable with a separate desktop app, they can use that along with magnet links. I know it’s not a big ask for that, but some users do consider that as enough friction vs their comfort of using a web browser only to perform a download.
We are not too big, but reducing the 20TB we average monthly would be good, especially when a download gets very popular (last month one download link alone used 8TB). Torrents can really help there, and the easier it is for transitioning our users to the better.
Not sure why Brave doesn’t just do away with webtorrent as a default add-on at this point. Nobody actually prefers using it to using a standalone torrent client and it doesn’t generate any revenue. Years in, most users still consider it an annoyance – why waste developers’ time “fixing” a feature nobody wants when so many valuable features beg for attention?
Any update on this? Very disappointing.
I would prefer it should store files in Downloads folder to be able open with other players (for media).
It makes sense (to me) to treat content downloaded via WebTorrent the same as when you download a file from a website
Given that, I don’t know if it would ever go out of scope unless you are in a private / Tor window and close all the tabs. Even then, I think Chromium/Brave will keep downloads on disk, it just won’t be listed in the download manager interface
Can confirm this is still an ongoing issue. I was also trying to find out if using webtorrent inside Brave protects you from sneaky ISPs, the same way that a VPN does…? Sorry, side point, but wasn’t really able to find an answer and it led me here.