gatsby: Gatsby@2.11.0 Build failure (JavaScript heap out of memory)

Description

Gatsby 2.11 breaks site build. I’m guessing it’s the new functionality around building node_modules as well as the site (#14111).

i was updating dependencies on master when build started breaking. I’ve updated to 2.10.5 on an update branch with no troubles but as soon as I move Gatsby to 2.11 the build breaks.

Steps to reproduce

  1. Clone master or update branches from https://github.com/moonmeister/moonmeister.net
  2. update gatsby dep to gatsby@2.11.0
  3. run build => see breakage.

Expected result

build should be successful.

Actual result

[BABEL] Note: The code generator has deoptimised the styling of /home/ajmoon/code/moonmeister/moonmeister.net/node_modules/netlify-cms-app/dist/netlify-cms-app.js as it
⠦ Building production JavaScript and CSS bundles

<--- Last few GCs --->

[19712:0x2dbca30]    45370 ms: Scavenge 1338.2 (1423.9) -> 1337.3 (1423.9) MB, 2.9 / 0.0 ms  (average mu = 0.163, current mu = 0.102) allocation failure 
[19712:0x2dbca30]    45374 ms: Scavenge 1338.4 (1423.9) -> 1337.5 (1423.9) MB, 2.8 / 0.0 ms  (average mu = 0.163, current mu = 0.102) allocation failure 
[19712:0x2dbca30]    45378 ms: Scavenge 1338.6 (1423.9) -> 1337.7 (1424.4) MB, 2.6 / 0.0 ms  (average mu = 0.163, current mu = 0.102) allocation failure 


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x34eb54adbe1d]
    1: StubFrame [pc: 0x34eb54a875c2]
Security context: 0x2caa7a21e6e9 <JSObject>
    2: /* anonymous */(aka /* anonymous */) [0x23804dd52ac9] [/home/ajmoon/code/moonmeister/moonmeister.net/node_modules/@babel/core/lib/transformation/file/merge-map.js:~155] [pc=0x34eb5534b963](this=0x01389b5022b1 <null>,m=0x1c2e6adbae29 <Object map = 0x11c6eb590b11>)
    3: arguments adaptor frame: 3->1
    4: forEach...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x8dc510 node::Abort() [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
 2: 0x8dc55c  [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
 3: 0xad9b5e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
 4: 0xad9d94 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
 5: 0xec7bf2  [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
 6: 0xec7cf8 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
 7: 0xed3dd2 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
 8: 0xed4704 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
 9: 0xed7371 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
10: 0xea07f4 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
11: 0x114018e v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/home/ajmoon/.nvm/versions/node/v10.15.3/bin/node]
12: 0x34eb54adbe1d 
Aborted (core dumped)
error Command failed with exit code 134.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Environment

  System:
    OS: Linux 5.0 Ubuntu 19.04 (Disco Dingo)
    CPU: (8) x64 Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
    Shell: 5.0.3 - /bin/bash
  Binaries:
    Node: 10.15.3 - ~/.nvm/versions/node/v10.15.3/bin/node
    Yarn: 1.15.2 - ~/.nvm/versions/node/v10.15.3/bin/yarn
    npm: 6.9.0 - ~/.nvm/versions/node/v10.15.3/bin/npm
  Browsers:
    Chrome: 75.0.3770.100
    Firefox: 67.0.4
  npmPackages:
    gatsby: 2.11.0 => 2.11.0 
    gatsby-image: ^2.0.34 => 2.2.3 
    gatsby-plugin-eslint: ^2.0.4 => 2.0.5 
    gatsby-plugin-manifest: ^2.0.24 => 2.2.0 
    gatsby-plugin-netlify: ^2.0.13 => 2.1.0 
    gatsby-plugin-netlify-cms: ^4.1.0 => 4.1.0 
    gatsby-plugin-offline: ^2.0.25 => 2.2.0 
    gatsby-plugin-react-helmet: ^3.0.10 => 3.1.0 
    gatsby-plugin-robots-txt: ^1.4.0 => 1.4.0 
    gatsby-plugin-sass: ^2.0.11 => 2.1.0 
    gatsby-plugin-sharp: ^2.0.30 => 2.2.1 
    gatsby-plugin-sitemap: ^2.0.11 => 2.2.0 
    gatsby-remark-images: ^3.0.10 => 3.1.2 
    gatsby-remark-relative-images: ^0.2.2 => 0.2.2 
    gatsby-source-filesystem: ^2.0.27 => 2.1.1 
    gatsby-transformer-remark: ^2.3.8 => 2.5.0 
    gatsby-transformer-sharp: ^2.1.17 => 2.2.0 
  npmGlobalPackages:
    gatsby-cli: 2.7.2
    gatsby-dev-cli: 2.5.0

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 9
  • Comments: 50 (34 by maintainers)

Commits related to this issue

Most upvoted comments

Yeah, affects Netlify too. I was able to get Netlify to build correctly by adding an environment variable:

[build.environment]
  NODE_OPTIONS = "--max_old_space_size=4096"

Add in netlify like this. Screen Shot 2019-06-28 at 2 26 47 AM

Okay, so this is definitely a node issue. It seems that the extra transpiling being done is just now making node hit it’s default limit in my case. I ran export NODE_OPTIONS=--max_old_space_size=4096 to expand the defaults and build is now happy.

this seems strange that node doesn’t just request more memory if it’s available, also I’m a little concerned how this might affect folks build servers.

gatsby-plugin-netlify-cms runs an independent Webpack build within Gatsby’s Webpack build (insert “yo dawg” reference here) using a modified version of the Gatsby Webpack config. Gatsby 2.11.0 no longer excludes node_modules, which causes the plugin’s Webpack build to attempt processing the Netlify CMS modules with Babel. The modules are massive and also prebuilt, so it will (and should) crash your system.

The PR I raised just adds that exclusion back in to the plugin’s Webpack config. Please don’t override your node max file size limit to fix this!

Added “(JavaScript heap out of memory)” to end of issue title just now (in addition to reopening yesterday) to try and prevent more duplicate issues like #15540

Seems like gatsby-netlify-cms being the issue. I’ll cook up a fix.

Sorry for the inconveniences

@moonmeister OMG thank you so much I just updated my package.json file and suddenly this issue crept up. And yes increasing the size solved the issue.

This seems like it might negatively affect a lot of people. Any way we can make this more memory efficient?

I’ll be closing this one in favor of #15256

Please move the discussion to that thread. I have created a PR that hopefully fixes the OOM issue https://github.com/gatsbyjs/gatsby/pull/15588

Yeah, affects Netlify too. I was able to get Netlify to build correctly by adding an environment variable:

[build.environment]
  NODE_OPTIONS = "--max_old_space_size=4096"

This worked for me ❤️

Thanks Alex, I updated to netlify-cms-app and everything works. Werner

Sent from my iPad

On Jul 4, 2019, at 19:20, Alex Moon notifications@github.com wrote:

@wernerglinka netlify-cms changed the name of it’s package to netlify-cms-app. That may be affecting you, not sure.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

@wernerglinka netlify-cms changed the name of it’s package to netlify-cms-app. That may be affecting you, not sure.

Yeah, build times have definitely increased. I’m not sure what value this adds, I’m sure it does but it seems to have some significant downsides too.