webpack: Webpack 5 memory leak in watch mode
Bug report
What is the current behavior? When I make a change in my web app and Webpack compiles, memory usage seems to increase. If I keep making changes and Webpack keeps re-compiling, memory usage will continue to climb. In a large project I work in, it takes about 10 compilations until Webpack crashes with a “Allocation failed - JavaScript heap out of memory” exception.
If the current behavior is a bug, please provide the steps to reproduce.
Please see the minimal repo I created to reproduce the issue: https://github.com/Geksanit/wp5-memory-leak.
1 yarn start
2 save any ts/tsx files without changes
What is the expected behavior? I would expect some kind of garbage collection to happen in between compilations.
Other relevant information: webpack version: 5.24.1 Node.js version: 12.19.0 Operating System: Win10 Additional tools: yarn 1.22.10
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 15
- Comments: 20 (12 by maintainers)
Interesting thing I figured out:
Using the devtools sometimes show fake memory leaks. Seems like the something in devtools is holding some references which prevents that some WeakMap references to be collected. I think it’s either the Source panel or the memory dumps themselves. Creating heapdumps via API fixes that problem…
Anyway I found a memory leak that happens when modules are no longer referenced by the compilation, but stay in cache. Here they keep some references to the old compilation and keep it alive. That should not happen and will be fixed by #12894.
Ok I looked into that a bit deeper and there is a pretty serious memory leak that happens when removing modules from the compilation. e. g. removing an
import. But it’s not that easy to fix.One problem are backward-compat back references to objects one should not have access too. That’s easy to fix by removing the backward-compat references when the next Compilation starts. In the next major they will be removed anyway.
Another problem is that modules in cache keep referencing the Compilation they were created in (indirectly e. g. through the parser), which then references a lot other objects. We need to move such references (like parser, generator, etc) from the Module to the ModuleGraph and give the Module access to the ModuleGraph at places where it want to access it’s references. And to make it backward-compatible we need to add a lot getters for the old references.
yeah, trying to create minimal repo with minimal configuration that we can work on.
devtool: false
Trying to pin this but nothing concrete, till then kinda suspicious looks Compilation 🤔
Compering snapshot 2 vs 3
I think it’s connected with new asset/resource and mini-css-extract-plugin
running webpack v5.24.2, node v12.14.1