rspack: [Bug]: Memory Leak in Dev Server Causing System to Run Out of Application Memory

Edited by @h-a-n-a:

For further issue reporting, please provide the system info. For example, Apple M2 Max, macOS 15.0.


Original Issue:

Version

System:
    OS: macOS 14.6.1
    CPU: (11) arm64 Apple M3 Pro
    Memory: 119.66 MB / 18.00 GB
    Shell: 5.9 - /bin/zsh
  Browsers:
    Safari: 17.6
  npmPackages:
    @rsbuild/core@1.11.13
    @rsbuild/plugin-react@1.1.0

Details

While using rsbuild dev, I encountered a significant memory leak that caused the system to run out of application memory. This issue makes the system unstable and forces the user to quit applications to free up resources.

Image

CodeSandbox: https://codesandbox.io/p/github/Fausto95/react-native-web-rsbuild/main?import=true

Reproduce link

https://github.com/Fausto95/react-native-web-rsbuild

Reproduce Steps

  1. pnpm install
  2. pnpm start:web
  3. Keep on applying changes to /apps/rnw/src/App.js
  4. Use Activity Monitor and observe the continuous increase in memory usage for the rsbuild-node process

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 21
  • Comments: 53 (19 by maintainers)

Most upvoted comments

I’ve merged a fix to lift memory consumption issue of memory cache. Rspack will now clean the invalid memory cache. By default, memory cache unused in the next compilation will be removed (defaults maxGenerations is set to 1)https://github.com/web-infra-dev/rspack/pull/9517. I’ve tested it against a few hmr + rss memory tests, showing threejs_10x reducing almost 20% of memory usage. https://github.com/web-infra-dev/rspack/commit/25661fe9e41496966699e3427f19f97865746544#commitcomment-153133945

We’re also seeing leaks in @rspack/core@1.3.2. This is our config:

{
    entry: tempLocalFile,
    plugins: [
      new WebpackObfuscatorPlugin({
        rotateStringArray: true,
      }),
    ],
    output: {
      path: tempPersistedPath,
      clean: true,
      module: true,
    },
    experiments: {
      outputModule: true,
    },
    mode: "production",
    cache: false,
    stats: false,
  }

Though our use case is pretty atypical as we’re initializing new compilers per-request to serve highly concurrent bundling (all uncacheable due to the nature of the use-case). As expected we have hundreds of thousands of compiler instances - unexpectedly though calling close and / or compiler.inputFileSystem.purge or compiler.purgeFileSystem does nothing. This deviates from the normal webpack implementation, which cleans up all references on close and allows the GC to sweep the unused compiler instances.

Image

☝ hundreds of thousands of cached input filesystems left orphaned and un-collectable.

EDIT: I did a little more digging and notice that these file system book-keeping data structures are being held onto by what I’m assuming is a native binding to the rust implementation, but I am unable to properly build a local version of the bindings to introspect what’s going on (most likely just user error on my end, but I also see M1 mac unsupported warnings everywhere when attempting to build). Manually munging the references seems to slightly improve the impact of memory pressure but I am assuming there is rust-implementation-side cleanup as well, as the leak does not completely disappear.

Sample munging:

					this.#compilation = undefined;
					this.#instance = undefined; <-- #instance.close supposed to be defined? locally I cannot get this to work.
					this.purgeInputFileSystem();
					this.inputFileSystem = null;

Would appreciate some help debugging given I have zero experience with rust and JS-native bindings 🙏

EDIT 2: My rust build locally being bricked locally was indeed just user error; I got things running and after sanity checking stuff I realized that #instance.close seems to not be implemented at all? Anyways cleaning up the references does reduce the memory footprint a bit but I am still seeing what I’m assuming is a native leak. Anyways this is the extent of what I can reasonably debug and I will leave it up to the experts 😉

Update on the current status of this issue.

I’ve figured out there’s some huge memory retention issue if cache was enabled during the build. This is not plaform specific. Here’s some data on my ubuntu-22.04 when cache was enabled as a complement to my previous result on macOS using Instruments.app.

Image Image

On top of that, we do have some platform specific issue related to macOS according to my investigation and this might be related to mimalloc, which is the memory allocator we used under the hood to speedup our compilation. I also filed an issue over there: https://github.com/microsoft/mimalloc/issues/1025. At the same time, I’m continuously trying to add a minimal reproducible demo to help tackle down the issue as soon as possible.

The canary version fixed the issue for me, @h-a-n-a!

Thank you

Took a test on mimalloc v3. Although memory is still growing, the issue has been greatly lifted. I’ve applied a fix to main and released a new canary if you would like to give it a shot right now, 1.2.8-canary-3d971bbf-20250304061156. Could use install-rspack to install canary. Refer to this PR to see changes in this fix https://github.com/web-infra-dev/rspack/pull/9533.

I’ll hold off on closing this until it’s fully fixed. It would be better if you could provide as much feedback as possible for this fix (better come up with the platform and OS info).

This might be related to memory cache.

I wrote an example. In this example, rspack was invoked rebuild for 4 times.

As you can see,

when memory cache was ON:

Image

when memory cache was OFF:

Image

This indicates that with memory cache turned on, the more rspack rebuilds, the more memory accumulates. Without memory cache, rspack rebuilds more slowly but also with lower memory retention. You can see every time rebuild was invoked, memory will drop down almost the same as it was when compiler was started.

Issue was created: https://github.com/web-infra-dev/rspack/issues/9405

I am updating from the rspack 1.2.8 to the latest 1.3.9 and found the dev server is dying with OOM (goes beyond 20GB, the project is ~5kk LoC). The 1.2.8 runs the dev server just fine with rspack-node process taking ~10Gb ram)

I was able to narrow down the issue to this flag

experiments: {
			parallelCodeSplitting: false,
}

It is enabled by default since 1.3.0. and hence I saw this behavior on every version beyond that. I also tried to enable this flag ie parallelCodeSplitting: true, for the 1.2.8 and actually it also leaks memory (the dev server throws the OOM) so it is been around for a while already.

The project relies heavily on dynamic imports for sure. We also use magic comments such as webpackChunkName.

I am looking for advice on how to narrow the root cause further, but without the logs or something like this, I am not sure where to look

PS: Ugh probably the thread is not related? I can create a separate issue, let me know

PSS: Posted this as a separate issue https://github.com/web-infra-dev/rspack/issues/10320

The canary version solves the mem leak (almost) completely for me 🥳. I’ve just used the canary version to test RAM consumption, I haven’t used it for active development, so I can’t comment on the stability

version initial compile after 10 HMR
1.2.2 4.57 GB 5.48 GB
1.2.7 3.93 GB 4.18 GB
1.2.8-canary-3d971bbf-20250304061156 3.66 GB 3.72 GB

I can create a separate issue, let me know

@n0th1ng-else let’s do it

I can verify @n0th1ng-else 's findings,

setting ...experiments.parallelCodeSplitting = false resolves the crashes on our end as well.

Commit a273443 @JSerFeng FYI

@hardfist if there was a way to print logs in case of deadlocks at CI, it would be easier to repro probably

@uladzimirdev agree, and our tracing log is not good enough to track deadlock issue but you can still give it a try by following https://rspack.dev/contribute/development/profiling#terminal

The canary version solves the issue with consecutive HMR

But it’s inconsistent, don’t know yet on which case it happens

Image

Thanks @h-a-n-a for working on this. I will try the canary and let you know how it goes.

I got the reproduction running on Linux. The result showed the memory consumption is normal without cache enabled. If you happen to have a Linux, feel free to help me test it out your own project. That would be very helpful.

@memory1668 Try turn off memory cache to see if memory is stable across builds.

Do you mean this config?

Image

In fact, I just tried this configuration, but the memory growth was even faster

My team has seen it grow from 2GB up to 20GB+. It’s present on at least 1.1 and 1.2; we have tested on both. It does appear to be HMR-related.

just general file edits

Yeah, it seems to be general edits. Not necessarily changes to imports.

The steps are as described:

  • Keep on applying changes to (some file referenced in the build)
  • Use Activity Monitor and observe the continuous increase in memory usage for the rsbuild-node process

Are there any logs or dumps I can collect to help? Happy to jump on a call or share configs etc

Hello @h-a-n-a

I used the reproduce link from @Fausto95, and I have the same issue (even with @rsbuild/core: “^1.1.13”)

Memory used on start-up

Image

Memory used after some builds (at each save it increases a bit even without any change on the code)

Image

After quitting and restarting the build

Image

Memory used after some builds (Just by spamming cmd+s without any changes on the code)

Image

Let me know if you need more context, or you want me to test things.

Same here. Our application starts out with under 2GB of RAM usage for the rspack-node process, and easily accumulates around 20GB during usage. We are on 1.2.1 and i am using a Apple M3 Pro.

Are there any plans to investigate / fix this? Thank you for consideration.

@h-a-n-a I used it to install canary, let me try one more time.

yeah, no difference. It looks like it happens mostly when I switch branches without restarting dev server, memory grows with every HMR in action

@h-a-n-a I’m ready to test possible fixes as just running dev server for a while increases memory consumption for me, it wasn’t a problem in v1

@uladzimirdev Do you mind adding a reproducible demo? Which version of rspack do you use that doesn’t have this issue? You may also try @rspack-canary/core@1.2.0-canary-c47c2431-20250117114227

I can report the same issue with rspack, rspack-node process can take ~30Gb of RAM.

I use @rspack/core@1.1.6, rspack config can be found here