webpack: out of memory

Webpack 2 terminates due to out of memory for our Project at work.

<--- Last few GCs --->

  117575 ms: Mark-sweep 1356.4 (1457.2) -> 1364.8 (1457.2) MB, 593.4 / 0 ms [allocation failure] [scavenge might not succeed].
  118168 ms: Mark-sweep 1364.8 (1457.2) -> 1364.8 (1457.2) MB, 592.5 / 0 ms [allocation failure] [scavenge might not succeed].
  118777 ms: Mark-sweep 1364.8 (1457.2) -> 1364.8 (1457.2) MB, 609.1 / 0 ms [last resort gc].
  119381 ms: Mark-sweep 1364.8 (1457.2) -> 1364.8 (1457.2) MB, 603.8 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2caa627e3ac1 <JS Object>
    1: /* anonymous */(aka /* anonymous */) [/Users/entwickler/programming/factroFrontend/node_modules/webpack/lib/optimize/RemoveParentModulesPlugin.js:43] [pc=0xe09107f767b] (this=0x2caa62704189 <undefined>,key=0x11a5e8dd09d9 <String[4]: 1084>)
    2: arguments adaptor frame: 3->1
    3: InnerArrayForEach(aka InnerArrayForEach) [native array.js:~942] [pc=0xe09109b31f0] (this=0x2caa62704189 <u...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
fish: 'node_modules/.bin/webpack' terminated by signal SIGABRT (Abort)

Sadly I’m not allowed to just give you our Project for debugging. I hope you can still do something about it.

Configuration:

eslint: {
    configFile: './.eslintrc',
    failOnWarning: false,
    failOnError: true,
  },
  context: __dirname,
  resolve: {
    modulesDirectories: ['node_modules/@marudor', 'dependency', 'node_modules'],
    alias: {
      theme: 'Theme/index.js',
      warning: 'fbjs/lib/warning.js',
      shallowequal: 'fbjs/lib/shallowEqual.js',
      vex: 'vex/js/vex.js',
      joyride: 'react-joyride/index.js',
      vexCSS: path.resolve('./dependency/vex/css'),
    },
  },
  entry,
  output: {
    path: outputPath,
    filename: name,
    chunkFilename: node_env === 'production' ? '[id]-[chunkhash].js' : '[id].js',
    publicPath: '/',
  },
  module: {
    loaders: [
      { test: /^((?!(CSS|css)\.js$).)*(\.jsx?)$/,
        exclude: /(node_modules|dependency|primusClient\.js|Unit\/Services)/,
        loader: jsLoader,
      },
      {
        test: /tinymce(\\|\/)(plugins|themes|skins|langs)/,
        loader: 'file?name=[path][name].[ext]&context=dependency/tinymce',
      },
      { test: /\.css$/, loader: ExtractTextPlugin.extract('style', 'css'), include: /Editor/, exclude: /tinymce(\\|\/)/ },
      { test: /\.css$/, loader: 'style!css', exclude: /(tinymce(\\|\/)|Editor)/ },
      { test: /\.(CSS|css)\.js$/, exclude: /(node_modules|dependency)/, loader: `inline-css!${jsLoader}` },
      { test: /\.(jpg|png|gif|jpeg|ico)$/, loader: 'url', exclude: /tinymce(\\|\/)/ },
      { test: /\.(eot|ttf|otf|svg|woff2?)(\?.*)?$/, loader: 'file', exclude: /tinymce(\\|\/)/ },
      { test: /\.pdf$/, loader: 'file', exclude: /tinymce(\\|\/)/ },
      { test: /\.json$/, loader: 'json' },
      {
        test: /-spec\.js/,
        exclude: /(node_modules|dependency)/,
        loader: 'mocha!babel!eslint',
      },
    ],
    noParse: [
      /primusClient\.js/,
      /.*primusClient.*/,
      /react\\dist\\react(-with-addons)?\.js/,
    ],
  },
  plugins: [
    new webpack.NoErrorsPlugin(),
    new webpack.DefinePlugin({
      'process.env': {
        NODE_ENV: JSON.stringify(node_env),
      },
      FACTRO_CONFIG: JSON.stringify(config),
      __DEV__: JSON.stringify(__DEV__),
      __PROD__: JSON.stringify(!__DEV__),
      __WEB__: JSON.stringify(true),
      __APP__: JSON.stringify(false),
    }),
    new ExtractTextPlugin('[name].css'),
    new HtmlWebpackPlugin({
      filename: 'index.html',
      template: 'html!src/Web/index.html',
      minify: {},
    }),
    new webpack.optimize.CommonsChunkPlugin({
      async: true,
      minChunks: 3,
      children: true,
    }),
  ]
  devServer: {
    proxy: {
      '/api/*': {
        target: 'http://localhost:65429',
        toProxy: true,
        changeOrigin: true,
        xfwd: true,
      },
      '/primus/*': {
        target: 'http://127.0.0.1:3000',
        ws: true,
      },
    },
  },

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 8
Comments: 43 (16 by maintainers)

Commits related to this issue

Generate sourcemaps in production Resolves instructure/canvas-lms#1101 In production, generate separate sourcemap files so that bundles appear split into their original, un-obfuscated source files i... — committed to sfu/canvas-lms by grahamb 7 years ago
Generate JavaScript sourcemaps in production Resolves instructure/canvas-lms#1101 In production, generate separate sourcemap files so that bundles appear split into their original, un-obfuscated sou... — committed to sfu/canvas-lms by grahamb 7 years ago
Generate JavaScript sourcemaps in production Resolves instructure/canvas-lms#1101 In production, generate separate sourcemap files so that bundles appear split into their original, un-obfuscated sou... — committed to sfu/canvas-lms by grahamb 7 years ago
Generate JavaScript sourcemaps in production Resolves instructure/canvas-lms#1101 In production, generate separate sourcemap files so that bundles appear split into their original, un-obfuscated sou... — committed to sfu/canvas-lms by grahamb 7 years ago
Generate JavaScript sourcemaps in production Resolves instructure/canvas-lms#1101 In production, generate separate sourcemap files so that bundles appear split into their original, un-obfuscated sou... — committed to instructure/canvas-lms by grahamb 7 years ago

Most upvoted comments

@skora: I tried. Also, there is a problem.

szagi3891 on Sep 9, 2016

on node v8+

instead of

  "scripts": {
    "build": "node --max_old_space_size=8192 node_modules/.bin/webpack "
  },

try to add this to your bash script (or .bash_profile or .bashrc):

export NODE_OPTIONS=--max_old_space_size=8192

it works for me on mac osx

more info on stackoverflow

akalitenya on Jun 24, 2018

Same issue here, webpack 2.2, happens often on 8GB RAM machine after ~20-30 sass/js rebuilds

tlenex on Feb 12, 2017

Update: My issue ended up being in the extract-text plugin(ETP)…To start the sourcemap dev tool plugin is a memory hog but the real issue was that ETP was generating some huge css files due to https://github.com/webpack-contrib/extract-text-webpack-plugin/pull/452 which increased the memory usage of the sourcemap plugin. Applying that patch to ETP fixed my issue.

sresant on Mar 6, 2017

Is it a big project? If so, you can increase the memory available to the node.js process…

sokra on Mar 9, 2016

I am also seeing this issue. I noticed that if i were to use say the cheap-source-map instead or any other devtool it works fine but this problems seems specific to source-map devtool for my case. I was able to build with webpack v1.14.0 but I have issues with webpack 2

sresant on Mar 3, 2017

I’m just leaving this error dump here. I’m using webpack-dev-server and after doing some work and emitting several changes with hot-module-reload the server crashes by using too much memory. I have been watching the memory usage and noticed it always grows after an emit and will never decrease (memory leak?)

I’m not certain that this memory leak might be related to webpack core, but I’m unable to determine what is causing it.

Emitted js build is about 9MB without source maps and about 26MB with sourcemap.

webpack 3.9.1
webpack-dev-server 2.9.5

<--- Last few GCs --->

[16:0x55a8aa6e0000] 18017361 ms: Mark-sweep 1358.1 (1415.2) -> 1358.0 (1415.2) MB, 160.8 / 0.1 ms  allocation failure GC in old space requested
[16:0x55a8aa6e0000] 18017524 ms: Mark-sweep 1358.0 (1415.2) -> 1357.9 (1378.7) MB, 162.6 / 0.1 ms  last resort GC in old space requested
[16:0x55a8aa6e0000] 18017699 ms: Mark-sweep 1357.9 (1378.7) -> 1357.9 (1378.7) MB, 175.6 / 0.1 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x1ffc11a5749 <JSObject>
    1: fromString(aka fromString) [buffer.js:340] [bytecode=0xfd9f52437d1 offset=161](this=0x13970f1822d1 <undefined>,string=0x2b66c3dddb49 <Very long string[25606263]>,encoding=0x1ffc11b49c1 <String[4]: utf8>)
    2: from [buffer.js:193] [bytecode=0xfd9f52432b1 offset=14](this=0xa4009d3b819 <JSFunction Buffer (sfi = 0x1ffc11fdd19)>,value=0x2b66c3dddb49 <Very long string[25606263]>,encodingOrOffs...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

hultberg on Dec 1, 2017

Has anybody tried to run the dev tools memory profiler on an incremental build? That’s how i found this problem with the source-map library a while ago: https://github.com/mozilla/source-map/issues/259

Don’t want to seem too negative, but source mapping is a critical part of webpack and the source-map lib from mozilla has piss poor performance. It was written in an era of tiny javascript. I think the ideal would be a native implementation. I remember seeing an incomplete rust one which was literally 100x faster…

MagicDuck on Sep 24, 2017

Same issue in my project. it’s happened only when devtool set to “source-map” (when devtool set to “cheap-source-map” it work fine).

liadoster on Jun 4, 2017

In our project a V8 issue was causing node to run out of memory. I’ve sent a PR to workaround the issue in webpack: https://github.com/webpack/webpack/pull/2497

achicu on May 16, 2016

Friendly advice: giving version numbers is usually not enough info to get help. You have to think about it from the perspective of the project maintainer. I know i was making this mistake before and wondering why no help was fothcoming 😁. Either provide a repro repository or if that’s not possible, try to get some kind of performance log/dump such as an allocation snapshot. It’s actually quite easy to do. Ex insructions here: https://www.toptal.com/nodejs/debugging-memory-leaks-node-js-applications

MagicDuck on Feb 15, 2018

@Anthony59273 Try a larger size might help? I’m using "webpack:dev": "NODE_ENV=development node --max_old_space_size=8096 ./node_modules/.bin/webpack --progress --colors",

artivilla on Dec 6, 2017

Given reports which indicate that the issue might relate to source maps I had a quick look at SourceMapDevToolPlugin.js. In line 60 array entries are pushed onto chunk.files within a map()-callback used when iterating the <del>same</del> array (see edit below). <del>Its an uneducated guess but modifying an array while iterating over it is always prone to side effects</del>. I could imagine chunk.files is stacking up over time, though can’t verify it at the moment. But since we push cached files onto the array it might fit into reports telling that the problem occurs only after a few rebuilds.

Edit (2017-10-01)

modifying an array while iterating over it is always prone to side effects

I have to correct me. Since map() is invoked on the result of filter() and filter() is an immutable operation generating a new array, saying that the same array is modified while being iterated with map() isn’t a valid statement.

about-code on Oct 1, 2017

same issue in ci.

ycjcl868 on Nov 11, 2020

Thanks everyone, we have a lot of difference issues here, it is hard to track and resolve their. When you have very very very big project you really need to increase node memory (i.e. node --max_old_space_size=4096). Also some loaders/plugins can have memory leak.

If you faced with FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory:

Try to update all deps to latest version (problem can be solved).
Try to increase node memory - node --max_old_space_size=4096
Try to disable plugins/loaders step by step.

If nothing has helped feel free create new issue with minimum reproducible test repo. Thanks!

alexander-akait on May 6, 2018

Yes second this!! We in fact won’t even look into this without a snapshot, heapdump or memtracing profile!

TheLarkInn on Feb 15, 2018

Further the filter().map().forEach() chain over chunk.files maps to tasks which when iterated later will further add to chunk.files in line 158.

These are only results from a superficial review for code smells made with a worst-case assumption. I’ve not attempted to fully understand the code (would have taken hours or days, probably). So read this as a disclaimer, please.

about-code on Aug 24, 2017

I was having this issue after adding a decent amount of json in a file (~3.6Mb) as a direct export. It seemed to actually have been caused by eslint via eslint-loader and not webpack directly, because adding the file to my .eslintignore solved the problem.

justinmc on Oct 21, 2016