webpack: Webpack 3 RC ModuleConcatenationPlugin crash, out of memory

Do you want to request a feature or report a bug?

Bug

What is the current behavior?

Webpack crashes trying to build our application when using Webpack.optimize.ModuleConcatenationPlugin().

Also note up until the crash the build time is also showing a >60% increase in regards to build time with this plugin.

If the current behavior is a bug, please provide the steps to reproduce.

Building a large application (3000-4000 modules), with several entry point, a few commons chunks, pretty aggressive code-splitting using import() everywhere and uglified with source-maps.

What is the expected behavior?

No crash without having to bump memory usage. Note the build time increase is pretty significant too - wouldn’t expect a single plugin to have such a drastic impact.

Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.

OS: Mac OSX Node.JS: v6.9.1 Webpack version: 3.0.0-rc.2

Stack:

<--- Last few GCs --->

  471727 ms: Mark-sweep 1343.8 (1443.3) -> 1343.2 (1443.3) MB, 1503.7 / 0.0 ms [allocation failure] [scavenge might not succeed].
  473238 ms: Mark-sweep 1343.2 (1443.3) -> 1343.2 (1443.3) MB, 1509.9 / 0.0 ms [allocation failure] [scavenge might not succeed].
  474819 ms: Mark-sweep 1343.2 (1443.3) -> 1343.2 (1427.3) MB, 1579.0 / 0.1 ms [last resort gc].
  476439 ms: Mark-sweep 1343.2 (1427.3) -> 1343.2 (1427.3) MB, 1619.3 / 0.0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x24ab363cfb51 <JS Object>
    1: stringify [native json.js:~176] [pc=0x138f1ce98049] (this=0x24ab363c9111 <a JSON with map 0x8bbe1d0a0e1>,E=0x362ddef793a9 <an Object with map 0x26615909c261>,F=0x24ab36304381 <undefined>,S=0x24ab36304381 <undefined>)
    2: arguments adaptor frame: 1->3
    3: source(aka getSource) [/Users/chris/Projects/frontend/node_modules/stats-webpack-plugin/index.js:25] [pc=0x138f1d931920] (this=0x...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
 4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
 5: v8::internal::IncrementalStringBuilder::Extend() [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
 6: v8::internal::BasicJsonStringifier::SerializeString(v8::internal::Handle<v8::internal::String>) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
 7: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
 8: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
 9: v8::internal::BasicJsonStringifier::SerializeJSArraySlow(v8::internal::Handle<v8::internal::JSArray>, unsigned int, unsigned int) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
10: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
11: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
12: v8::internal::BasicJsonStringifier::SerializeJSArraySlow(v8::internal::Handle<v8::internal::JSArray>, unsigned int, unsigned int) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
13: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
14: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
15: v8::internal::BasicJsonStringifier::SerializeJSArraySlow(v8::internal::Handle<v8::internal::JSArray>, unsigned int, unsigned int) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
16: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
17: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
18: v8::internal::Runtime_BasicJSONStringify(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/chris/.nvm/versions/node/v6.9.1/bin/node]
19: 0x138f16c092a7
sh: line 1: 43838 Abort trap: 6           webpack --config ./webpack/build.webpack.config.js --bail --profile

npm ERR! Darwin 14.5.0
npm ERR! argv "/Users/chris/.nvm/versions/node/v6.9.1/bin/node" "/Users/chris/.nvm/versions/node/v6.9.1/bin/npm" "run" "build"
npm ERR! node v6.9.1
npm ERR! npm  v3.10.8
npm ERR! code ELIFECYCLE
npm ERR! frontend@0.209.0 build: `npm run clean; webpack --config ./webpack/build.webpack.config.js --bail --profile`
npm ERR! Exit status 134
npm ERR! 
npm ERR! Failed at the frontend@0.209.0 build script 'npm run clean; webpack --config ./webpack/build.webpack.config.js --bail --profile'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the frontend package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     npm run clean; webpack --config ./webpack/build.webpack.config.js --bail --profile
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs frontend
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls frontend
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/chris/Projects/frontend/npm-debug.log

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 40
  • Comments: 18 (3 by maintainers)

Most upvoted comments

@JLHwung Yes. I’m on 3.5.2 using --max-old-space-size=10240. The process consumes up to around 8gb of memory on a relatively large app (single output bundle is ~5mb, minified). Watching the process in activity monitor, it seems to be fine for a while, slowly growing to around 2gb, then towards the end it shoots up in increments as high as 1gb at a time, probably when the ModuleConcatenationPlugin kicks in. Setting devtool to false (as suggested in the other thread) does not have a noticeable effect on memory consumption.

Without ModuleConcatenationPlugin, the process maxes out around 2.5gb towards the end.

Basically this means that we need to upgrade our CI boxes from 4gb to 8 or even 16. That expense is difficult to justify for the marginal gains from this plugin.

Build speed isn’t really an issue in a CI environment. I wonder if there could be a slower, disk-oriented version of the plugin that could run in the kind of low-memory environments common for CI?

+3
echenley on Aug 9, 2017

Ran into this issue as well. If you wonder how to give webpack more memory through that flag:

node --max-old-space-size=8192 node_modules/.bin/webpack
+2
tajo on Jul 19, 2017

Could you reproduce this issue on 3.5.2? It seems that #5471 is possibly related to this issue.

+1
JLHwung on Aug 9, 2017

My two cents: The project I’ve been testing ModuleConcatenationPlugin in, I’ve mostly gotten this on situations where I have cache enabled, e.g. on watch. I simply disabled it when I’m on HMR mode, for example, and only kept it when producing a final bundle, which is where the optimizations matter to me for the most part.

+1
emilio-martinez on Jun 23, 2017

@aseem2625 look at this html-webpack-plugin comment, it fixed my ‘undefined’ error (mainly back to webpack@2.6.1)

+1
ghostd on Jun 20, 2017

@aseem2625 then you should probably open another issue 😄

+1
christophehurpeau on Jun 20, 2017

For me it’s bundling fine but the browser does’t load the app giving error call of undefined

+1
aseem2625 on Jun 20, 2017

Think i’m experiencing the same without the error logs. After the first bundle, with ModuleConcatenationPlugin, it does not recognize any changes anymore, so i have to restart it. It works like a charme without this plugin

+1
Manuelandro on Jun 20, 2017
Read more comments on GitHub
← webpack: webpack 2.6.0 can't find variable: Promise on require async
webpack: Webpack 4 doesn't support sharing modules from entries →