rspack: [Bug]: Memory Leak in Dev Server Causing System to Run Out of Application Memory
Edited by @h-a-n-a:
For further issue reporting, please provide the system info. For example, Apple M2 Max, macOS 15.0.
Original Issue:
Version
System:
OS: macOS 14.6.1
CPU: (11) arm64 Apple M3 Pro
Memory: 119.66 MB / 18.00 GB
Shell: 5.9 - /bin/zsh
Browsers:
Safari: 17.6
npmPackages:
@rsbuild/core@1.11.13
@rsbuild/plugin-react@1.1.0
Details
While using rsbuild dev, I encountered a significant memory leak that caused the system to run out of application memory. This issue makes the system unstable and forces the user to quit applications to free up resources.
CodeSandbox: https://codesandbox.io/p/github/Fausto95/react-native-web-rsbuild/main?import=true
Reproduce link
https://github.com/Fausto95/react-native-web-rsbuild
Reproduce Steps
pnpm installpnpm start:web- Keep on applying changes to
/apps/rnw/src/App.js - Use
Activity Monitorand observe the continuous increase in memory usage for thersbuild-nodeprocess
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 21
- Comments: 53 (19 by maintainers)
I’ve merged a fix to lift memory consumption issue of memory cache. Rspack will now clean the invalid memory cache. By default, memory cache unused in the next compilation will be removed (defaults
maxGenerationsis set to 1)https://github.com/web-infra-dev/rspack/pull/9517. I’ve tested it against a fewhmr + rss memorytests, showingthreejs_10xreducing almost 20% of memory usage. https://github.com/web-infra-dev/rspack/commit/25661fe9e41496966699e3427f19f97865746544#commitcomment-153133945We’re also seeing leaks in
@rspack/core@1.3.2. This is our config:Though our use case is pretty atypical as we’re initializing new compilers per-request to serve highly concurrent bundling (all uncacheable due to the nature of the use-case). As expected we have hundreds of thousands of compiler instances - unexpectedly though calling close and / or
compiler.inputFileSystem.purgeorcompiler.purgeFileSystemdoes nothing. This deviates from the normal webpack implementation, which cleans up all references on close and allows the GC to sweep the unused compiler instances.☝ hundreds of thousands of cached input filesystems left orphaned and un-collectable.
EDIT: I did a little more digging and notice that these file system book-keeping data structures are being held onto by what I’m assuming is a native binding to the rust implementation, but I am unable to properly build a local version of the bindings to introspect what’s going on (most likely just user error on my end, but I also see M1 mac unsupported warnings everywhere when attempting to build). Manually munging the references seems to slightly improve the impact of memory pressure but I am assuming there is rust-implementation-side cleanup as well, as the leak does not completely disappear.
Sample munging:
Would appreciate some help debugging given I have zero experience with rust and JS-native bindings 🙏
EDIT 2: My rust build locally being bricked locally was indeed just user error; I got things running and after sanity checking stuff I realized that
#instance.closeseems to not be implemented at all? Anyways cleaning up the references does reduce the memory footprint a bit but I am still seeing what I’m assuming is a native leak. Anyways this is the extent of what I can reasonably debug and I will leave it up to the experts 😉Update on the current status of this issue.
I’ve figured out there’s some huge memory retention issue if cache was enabled during the build. This is not plaform specific. Here’s some data on my ubuntu-22.04 when cache was enabled as a complement to my previous result on macOS using
Instruments.app.On top of that, we do have some platform specific issue related to macOS according to my investigation and this might be related to mimalloc, which is the memory allocator we used under the hood to speedup our compilation. I also filed an issue over there: https://github.com/microsoft/mimalloc/issues/1025. At the same time, I’m continuously trying to add a minimal reproducible demo to help tackle down the issue as soon as possible.
The canary version fixed the issue for me, @h-a-n-a!
Thank you
Took a test on mimalloc v3. Although memory is still growing, the issue has been greatly lifted. I’ve applied a fix to main and released a new canary if you would like to give it a shot right now,
1.2.8-canary-3d971bbf-20250304061156. Could use install-rspack to install canary. Refer to this PR to see changes in this fix https://github.com/web-infra-dev/rspack/pull/9533.I’ll hold off on closing this until it’s fully fixed. It would be better if you could provide as much feedback as possible for this fix (better come up with the platform and OS info).
This might be related to memory cache.
I wrote an example. In this example, rspack was invoked rebuild for 4 times.
As you can see,
when memory cache was ON:
when memory cache was OFF:
This indicates that with memory cache turned on, the more rspack rebuilds, the more memory accumulates. Without memory cache, rspack rebuilds more slowly but also with lower memory retention. You can see every time rebuild was invoked, memory will drop down almost the same as it was when compiler was started.
Issue was created: https://github.com/web-infra-dev/rspack/issues/9405
I am updating from the rspack 1.2.8 to the latest 1.3.9 and found the dev server is dying with OOM (goes beyond 20GB, the project is ~5kk LoC). The 1.2.8 runs the dev server just fine with
rspack-nodeprocess taking ~10Gb ram)I was able to narrow down the issue to this flag
It is enabled by default since 1.3.0. and hence I saw this behavior on every version beyond that. I also tried to enable this flag ie
parallelCodeSplitting: true, for the 1.2.8 and actually it also leaks memory (the dev server throws the OOM) so it is been around for a while already.The project relies heavily on dynamic imports for sure. We also use magic comments such as
webpackChunkName.I am looking for advice on how to narrow the root cause further, but without the logs or something like this, I am not sure where to look
PS: Ugh probably the thread is not related? I can create a separate issue, let me know
PSS: Posted this as a separate issue https://github.com/web-infra-dev/rspack/issues/10320
FYI, v1.2.8 was released, including these two major fixes:
The canary version solves the mem leak (almost) completely for me 🥳. I’ve just used the canary version to test RAM consumption, I haven’t used it for active development, so I can’t comment on the stability
@n0th1ng-else let’s do it
I can verify @n0th1ng-else 's findings,
setting
...experiments.parallelCodeSplitting = falseresolves the crashes on our end as well.Commit a273443 @JSerFeng FYI
@uladzimirdev agree, and our tracing log is not good enough to track deadlock issue but you can still give it a try by following https://rspack.dev/contribute/development/profiling#terminal
The canary version solves the issue with consecutive HMR
But it’s inconsistent, don’t know yet on which case it happens
Thanks @h-a-n-a for working on this. I will try the canary and let you know how it goes.
I got the reproduction running on Linux. The result showed the memory consumption is normal without cache enabled. If you happen to have a Linux, feel free to help me test it out your own project. That would be very helpful.
Do you mean this config?
In fact, I just tried this configuration, but the memory growth was even faster
My team has seen it grow from 2GB up to 20GB+. It’s present on at least 1.1 and 1.2; we have tested on both. It does appear to be HMR-related.
Yeah, it seems to be general edits. Not necessarily changes to imports.
The steps are as described:
rsbuild-nodeprocessAre there any logs or dumps I can collect to help? Happy to jump on a call or share configs etc
Hello @h-a-n-a
I used the reproduce link from @Fausto95, and I have the same issue (even with @rsbuild/core: “^1.1.13”)
Memory used on start-up
Memory used after some builds (at each save it increases a bit even without any change on the code)
After quitting and restarting the build
Memory used after some builds (Just by spamming
cmd+swithout any changes on the code)Let me know if you need more context, or you want me to test things.
Same here. Our application starts out with under 2GB of RAM usage for the rspack-node process, and easily accumulates around 20GB during usage. We are on
1.2.1and i am using a Apple M3 Pro.Are there any plans to investigate / fix this? Thank you for consideration.
@h-a-n-a I used it to install canary, let me try one more time.
yeah, no difference. It looks like it happens mostly when I switch branches without restarting dev server, memory grows with every HMR in action
@uladzimirdev Do you mind adding a reproducible demo? Which version of rspack do you use that doesn’t have this issue? You may also try @rspack-canary/core@1.2.0-canary-c47c2431-20250117114227
I can report the same issue with rspack,
rspack-nodeprocess can take ~30Gb of RAM.I use
@rspack/core@1.1.6, rspack config can be found here