webpack: Process out of memory - Webpack

Did someone resolved this problem？

Another data point here; I have filename hashes disabled and my build isn’t terribly large (it uses Angular 2, but few other dependencies), but after about an hour of dev work the watch dies with Abort trap: 6.

I’ve temporarily increased the memory limit, which gives me a lot more time, but I’d rather not be consuming gigabytes just to run a watch command.

FYI if anyone is curious about upping the memory limit, I did it like so in my package.json (obviously the additional flags are dependent on your needs):

{
  "scripts": {
    "start": "node --max_old_space_size=4096 node_modules/webpack-dev-server/bin/webpack-dev-server.js --inline --progress --port 3000 --open"
  }
}

Don’t use hashes when watching. i. e. [chunkhash] causes all assets to be stored.

@buildbreakdo A better option is to do this:

"scripts": {
    "build": "cross-env NODE_OPTIONS=--max_old_space_size=8192 webpack"
}

(It uses the cross-env package.)

The nice thing about using NODE_OPTIONS is that it will affect everything, including any sub-processes that Webpack might spawn.

(It’s useful for non-Webpack things as well: we’re using it for TypeDoc, which consumes a lot of RAM.)

this is happening on webpack 4 as well

I had this problem with the version 10… of node but, when I upgraded to the v12.4.0 the error gone.

I got this in production mode, but not development 🤷‍♂️

I’m getting the FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory basically after every 5-10 rebuilds in dev mode using --watch. Is there any way to get around this in webpack 1.14.0?

jup.

You can navigate to http://localhost:8080/webpack-dev-server to see a list of all files in memory.

Cause is usually processing of a large file that causes V8 to run out of RAM. By default V8 is only allocated 1400 MB of RAM. If you are seeing this error, it is likely that you need to increase this number.

V8 flags are set in a package.json script by declaring the flag between the call to node and ./yourScriptName.

Do not do this in your package.json scripts section, this passes the flag to webpack which is the wrong thing to do:

...
  "scripts": {
    "build": "webpack --max_old_space_size=8192" <-- Do not do this
  },
...

Do this:

...
  "scripts": {
    "build": "node --max_old_space_size=8192 node_modules/.bin/webpack "
  },
...

This is equivalent to:

#!/usr/local/bin/node --max_old_space_size=8192
..

or

const v8 = require('v8');
v8.setFlagsFromString('--max_old_space_size=8192');
..

Derived this answer from these two issues, credits to the angular/node folks: https://github.com/nodejs/node/issues/7937 https://github.com/angular/angular-cli/issues/1652

I can never find the answer to my question when looking at github threads like this. I see merges, closes, tags, etc but no body really says what causes the issue and how to fix it. sigh. Lets try my luck in stack overflow again.

Not sure if there is anyone else here using webpacker, but the problem for me was that in config/webpack/shared.js they changed this:

const extensionGlob = `*{${paths.extensions.join(',')}}*`

to this:

const extensionGlob = `**/*{${settings.extensions.join(',')}}*`

Which explains why it may be traversing down directories and loading files that are never actually used by the dependency graph. Dropping the **/ fixed the out of memory issue for me.

Same issue for a very small project

export NODE_OPTIONS=--max_old_space_size=4096

Works for me.

Three hours? I think the problem is just that you need more patience.

+1…

remove max_old_space_size

just add to ts-loader

           loader: 'ts-loader',
           options: {
             transpileOnly: true // this works for me
           }

@buildbreakdo A better option is to do this:

"scripts": {
    "build": "cross-env NODE_OPTIONS=--max_old_space_size=8192 webpack"
}

(It uses the cross-env package.)

The nice thing about using NODE_OPTIONS is that it will affect everything, including any sub-processes that Webpack might spawn.

(It’s useful for non-Webpack things as well: we’re using it for TypeDoc, which consumes a lot of RAM.)

This solution did resolve the problem in my project.

Same problem. Our production builds have recently started failing too. Typescript + React. We’ve increased memory for our node processes to no avail. Any advice or what version to roll back to?

Having the same problem running webpack via webpack-stream in gulp. I use gulp.watch to rerun on code changes, which then just runs

    return gulp.src(tsStartFile)
              .pipe(webpackStream(webPackConfig))
              .on('error', logError) 
              .pipe(gulp.dest(outputDir))

I’d have thought this would run webpack from scratch each time, but after 5 or so builds I get the out of memory error and have to rerun my gulp file.

Still happening on webpack 4.17.1

+1. Is someone able to give a definitive solution outside of setting max_old_space_size?

any solution to this ?

Having the same issue as @thunderkid here. Any update?

We have the exact same memory leak problem after using webpack -w on MacOS X. Can you please suggest what is a the best way to fix / workaround this issue. The ngOfficeUIFabric seems to deal with a different problem.

Thanks, Deepak

Memory leak found https://github.com/suguru03/neo-async/pull/83

I had this issue a year ago (see above in July 2017).

Some things I’ve done to our project since then - this issue has gone away entirely, confirmed on Mac, Linux, and Windows 10 dev workstations:

1. Upgraded to Node 8+
2. Upgraded to webpack v4+ (version 3 was causing some issues for me)
3. Upgraded to webpack-dev-middleware v3+
   • Ensure config for this is watchOptions: { poll: false }, stats: 'minimal' }
   • poll: false is important if you are using Docker 🐳 (I’m also using an alpine image for those who care node:8.11.4-alpine), it solves some performance hiccups
4. If you are using eslint plugin, make sure it is v4+, especially if its part of webpack config, ensure any non-source files are ignored
   • node_modules
   • coverage
   • build
   • static

Also still using react-transform-hmr v1.0.4, no issues with this version.

Another final note: If you use WebStorm, I would disable “safe write” as it usually causes performance issues, most often when your hard drive is symlinked into a Docker container. Similar settings may exist in other IDEs.

For what it’s worth, here is a snippet of the webpack config:

{
... { // other config stuff here}
module: {
		rules: [
			{
				test: /\.jsx?$/,
				exclude: /node_modules/,
				enforce: 'pre',
				use: [
					{
						loader: 'eslint-loader'
					}
				]
			},
			{
				test: /\.jsx?$/,
				exclude: /node_modules/,
				use: [
					{
						loader: 'babel-loader',
						options: {
							cacheDirectory: !isProduction // set elsewhere in file
						}
					}
				]
			}
		]
	},
	resolve: {
		extensions: ['.json', '.js', '.jsx']
	},
	plugins: [
		new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/), // only if you use moment timezone
		new webpack.HotModuleReplacementPlugin() // only for dev mode
	]
}

Got an issue resolved on one of the machine using below:

Try this out: In "ProjectDirectory\node_modules\.bin" modify webpack.cmd to:

@IF EXIST “%~dp0\node.exe” ( "%~dp0\node.exe --max_old_space_size=4096 " “%~dp0..\webpack\bin\webpack.js” %* ) ELSE ( @SETLOCAL @SET PATHEXT=%PATHEXT:;.JS;=;% node --max_old_space_size=4096 “%~dp0..\webpack\bin\webpack.js” %* )

and try to build. Basically, its going to use –max_old_space_size=4096 with node webpack build.

See if that helps.

Same issue here. Running webpack dev server takes up ~3-3.5gb of ram on my machine

Not sure if this is going to help anyone but we had a problem after updating to webpack 4 with the out of memory error. The build itself would often get stuck or become very slow. What very unexpectedly helped us was removing the BundleAnalyzerPlugin.

I fixed it disabling ModuleConcatenationPlugin!

Seems related to Webpack 3 upgrade for me as well

On Node v8.1.2 with upgrade from webpack@2.4.1 => webpack@3.0.0 caused much slowdown and the CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory error.

Upon doing: rm -rf node_modules and then npm install with webpack@2.4.1, the problem goes away, and compiling/build times are dramatically reduced.

We faced a similar problem with webpack-dev-server. Using a more aggressive policy for CommonsChunk plugin reduced memory usage from 1.8GB to 1.1GB, minChunks was set to 3 to include most of the shared code:

    new webpack.optimize.CommonsChunkPlugin({
      name: 'common',
      filename: 'common.js',
      minChunks: 3,
    }),

Is there a way to disable the usage of memory of FS and to use regular file system?

Having had this problem today, I updated this line:

  devtool: undefined

To:

  devtool: false

And all was suddenly OK…

Our solution was to shorten project folder’s name from about 25 down to 4 symbols. It works. So far… webpack 3.4.1

we are a big team 👍

After every 5-6 build on dev mode including--hot --inline --progress --colors --port 2000 , i got this FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

webpack v3.1.0

webpack-dev-server v2.5.1

Just upgraded to webpack 3.0 and experiencing leaks as well:

16:17:03 - File change detected. Starting incremental compilation...



<--- Last few GCs --->

[26089:0x3d30680]    47717 ms: Mark-sweep 1402.9 (1470.8) -> 1400.8 (1472.8) MB, 891.8 / 0.0 ms  allocation failure GC in old space requested
[26089:0x3d30680]    48960 ms: Mark-sweep 1400.8 (1472.8) -> 1400.8 (1472.8) MB, 1243.3 / 0.0 ms  allocation failure GC in old space requested
[26089:0x3d30680]    50225 ms: Mark-sweep 1400.8 (1472.8) -> 1400.8 (1442.8) MB, 1264.0 / 0.0 ms  last resort 
[26089:0x3d30680]    51476 ms: Mark-sweep 1400.8 (1442.8) -> 1400.8 (1442.8) MB, 1251.0 / 0.0 ms  last resort 


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x1c3674e29891 <JS Object>
    2: encodeLastRecordedSourceMapSpan [/home/rjansen/CODE/core/node_modules/typescript/lib/tsc.js:~50611] [pc=0xdef7f5f96d8](this=0x264e5bcb27a1 <JS Global Object>)
    3: emitPos [/home/rjansen/CODE/core/node_modules/typescript/lib/tsc.js:~50639] [pc=0xdef7f5525d1](this=0x264e5bcb27a1 <JS Global Object>,pos=0x2caa051c92e9 <Number: 5.37458e+06>)
    4: emitNodeWithSourceMap [/home/rjansen/CODE...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x13647ec [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewUninitializedFixedArray(int) [node]
 6: 0xe90ca3 [node]
 7: v8::internal::Runtime_GrowArrayElements(int, v8::internal::Object**, v8::internal::Isolate*) [node]
 8: 0xdef7de0437d
Aborted
npm ERR! code ELIFECYCLE
npm ERR! errno 134
npm ERR! core@1.0.0 watch-server: `tsc -p tsconfig.json -inlineSourceMap -outDir private --watch`
npm ERR! Exit status 134

The problem for me was a memory leak in postcss-loader v1.3.x, no more crashes so far after an upgrade to v.2.0.5

+1 same issue here deploying to heroku

I’m experiencing the same issue here. From what can I tell, it seems that Webpack is loading up files in the project directory even when they are technically “unreachable” from the dependency graph, starting from the given entry points.

I’m trying to integrate Webpack into an existing project with a large existing Javascript code (written as Typescript). This is where Webpack throws OOM errors.

So as an experiment, I try to make a minimal Webpack test example, as described in this gist. This compiles fine, so far so good.

But, when I copy the src directory of the large project into my minimal example, Webpack then throws OOM errors. Note that I just copied the src directory; none of the files in the given gist example were changed.

FWIW, I ran Webpack with strace and I get traces like

open("/node_modules/@types/zeroclipboard.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/node_modules/@types/zeroclipboard/package.json", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/node_modules/@types/zeroclipboard/index.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
...
open("/home/node_modules/zeroclipboard.js", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/node_modules/zeroclipboard.jsx", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/node_modules/zeroclipboard/package.json", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/node_modules/zeroclipboard/index.js", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/node_modules/zeroclipboard/index.jsx", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

I’m not sure if this is normal behavior or not but it is looking for node_modules way beyond where it is supposed to look.

strace also makes me sure that Webpack is indeed digging into the unreferenced src directory. I get traces like,

open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard.tsx", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard/package.json", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard/index.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard/index.tsx", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard/index.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/@types/zeroclipboard.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/@types/zeroclipboard/package.json", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/@types/zeroclipboard/index.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

Same issue has just started for me. Has there been any update on this?

@sokra In the documentation for css-loader it says to use hash for easier debugging wrt css modules. Does that have the same problem?

You can configure the generated ident with the localIdentName query parameter (default [hash:base64]). Example: css-loader?localIdentName=[path][name]---[local]---[hash:base64:5] for easier debugging.

We’ve also started to see this problem in dev (out of memory), but our only hash in dev mode is the css one.

EDIT: Of course it uses hash by default, so I’m guessing it’s not a problem.

Just adding my experience here. I did have issues with compilation on the t1.micro size but I did not on the t2.small.

@buildbreakdo what if build works great, and then after some hot module reloads, it keeps using more memory and then crashes

is this a bug on hot module reload?

What pushes a build over the 1400 MB ram allocated by node will be different for each project. There is no webpack bug here. However, there could be a bug in your build configuration (like compiling node_modules).

Or maybe, just maybe, the project is just that big and the build process requires more RAM than Nodes default 1400 MB. If you would like to increase the RAM allocated to Node see instructions above here: https://github.com/webpack/webpack/issues/1914#issuecomment-392660230

Same probleme here. I use Webpack (Dev Server) with Pug templates and HTMLWebpackPlugin.

 94% asset optimization
<--- Last few GCs --->

[5239:0x102801e00]   213342 ms: Mark-sweep 1364.4 (1412.2) -> 1364.2 (1412.2) MB, 104.2 / 0.0 ms  allocation failure GC in oldspace requested
[5239:0x102801e00]   213448 ms: Mark-sweep 1364.2 (1412.2) -> 1364.2 (1379.2) MB, 106.5 / 0.0 ms  last resort GC in old space requested
[5239:0x102801e00]   213542 ms: Mark-sweep 1364.2 (1379.2) -> 1364.2 (1379.2) MB, 94.3 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x15d329da55e9 <JSObject>
    2: replace(this=0x15d3dcff1d09 <Very long string[21267994]>,0x15d3ddf66541 <String[32]: var HTML_WEBPACK_PLUGIN_RESULT =>,0x15d333582431 <String[0]: >)
    3: evaluateCompilationResult [/Volumes/APPLE SD Card/Projets/.../node_modules/html-webpack-plugin/index.js:240] [bytecode=0x15d3ddf668c1 offset=40](this=0x15d3c80275d1 <HtmlWebpackPlugin map = 0x...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/.../.nvm/versions/node/v9.11.2/bin/node]
 2: node::FatalTryCatch::~FatalTryCatch() [/Users/.../.nvm/versions/node/v9.11.2/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/.../.nvm/versions/node/v9.11.2/bin/node]
 4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/Users/.../.nvm/versions/node/v9.11.2/bin/node]
 5: v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [/Users/.../.nvm/versions/node/v9.11.2/bin/node]
 6: v8::internal::String::IndexOf(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>, v8::internal::Handle<v8::internal::String>, int) [/Users/.../.nvm/versions/node/v9.11.2/bin/node]
 7: v8::internal::Runtime_StringIndexOfUnchecked(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/.../.nvm/versions/node/v9.11.2/bin/node]
 8: 0x1c0837c042fd
[1]    5238 abort      npm start

The problem is the RegEx, when it catch the files, it doesn’t filter well. If I build, removing the node_modules (from task_modules/primer_modulo/node_modules), it works well.

Same issue

<--- Last few GCs --->

 7076693 ms: Scavenge 1416.9 (1472.9) -> 1416.9 (1472.9) MB, 3.9 / 0 ms (+ 5.4 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
 7077971 ms: Mark-sweep 1416.9 (1472.9) -> 1403.9 (1459.8) MB, 1278.4 / 0 ms (+ 9.1 ms in 2 steps since start of marking, biggest step 5.4 ms) [last resort gc].
 7079257 ms: Mark-sweep 1403.9 (1459.8) -> 1403.9 (1459.8) MB, 1285.9 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2fb44c1e3ac1 <JS Object>
    2: toString [/home/abarrera/Proyectos/front-sae/node_modules/webpack/lib/Stats.js:411] [pc=0x24c131efa4ef] (this=0x3546c1922b51 <a Stats with map 0x38d80a68ee9>,options=0x2513409b01f9 <an Object with map 0x1ad3a812ccc1>)
    3: handler(aka compilerCallback) [/home/abarrera/Proyectos/front-sae/node_modules/webpack/bin/webpack.js:348] [pc=0x24c131efa08d] (this=0x2513409b3071 <a Watching with ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

Command used webpack --watch --progress

Webpack v3.2.0

Node v5.12.0

I have same issue, though I am using ionic app scripts, with webpack 3.0 the memory spikes up to 5gb and the process dies. With webpack 2 it used to be ok

Security context: 0x354f7b2a66a1 <JS Object>
    0: builtin exit frame: stringify(this=0x354f7b29a7d1 <a JSON with map 0x8ddf39053d9>,0xd2259a02311 <undefined>,0xd2259a02311 <undefined>,0x225fe8c83c11 <an Object with map 0x1460ed77b201>)

    1: arguments adaptor frame: 1->3
    2: /* anonymous */(aka /* anonymous */) [/Users/x/WebApp/node_modules/webpack/lib/SourceMapDevToolPlugin.js:~112] [pc=0xaf6d181150b](this=0xd2259a0231...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Update Node versión from v8.16.0 to v12.6.0 and that issue is gone!

upping the memory limit will only solve the problem temporarily but the RAM being used up the more you rebuild the application. You will notice your machine gets slower overtime.

Adding a directory with some large library files to .eslintignore fixed the issue for me.

Same problem, but using cheap-module-source-map at dev builds and false at production. @joshcomley that’s not solution, it’s falls too.

My problem was caused because I was using a large JSON file (multiple MBs) to mock a server-side HTTP response. This was a temporary workaround because the server-side endpoint was unders construction. Removing the large file solved the leak. So I would suggest to check the size of your files and split them into multiple files or remove them if possible if they are too large.

Had the same issue. Seems like source of the problem was I hadn’t allocated enough memory to my virtual machine (running ubuntu/trusty64 through Vagrant). I think Vagrant only allocates about 512 MB of memory by default.

Setting

config.vm.provider "virtualbox" do |vb|
  vb.memory = "2048"
end

in my Vagrantfile fixed the problem. I should note I’m also running node with --max_old_space_size=2048

Same issue here, in our case, we use webpack 3.5.6 and webpack-bundle-analyzer 2.9.0, if we use the bundle analyzer plugin and run webpack --watch --progress, the out of memory error will show up. Solution for us is simple, do not use the bundle analyzer when running webpack --watch --progress.

postcss-loader was causing this issue for me. Commenting it out would allow the builds to pass.

The issue was because the VPS I was running the build on had 1024MB RAM but only a 256MB swap disk. Increasing the swap disk size to 2048MB fixed the issue.

Changing devtool to eval-source-map from source-map solved it for me.

webpack 3.5.1

My error is directly related to: #5089

@eirikurn Alright. After looking into it, it looks like postcss-loader#2.0.0 did update to postcss#6.0, which appears to include some memory related improvements.

Not sure if this change is what impacted your improvement, but it’s at least good to know. Thanks again.

I think clearing them out like @SimenB said would be a nice option, or maybe there is some way to delete all in-memory files that are no longer in use?