gatsby: Build fail with heap out of memory error in gatsby 2.11.6

Description

With Gatsby version 2.11.6 I can`t build a production version of the app because it fails with out of memory exception while building (same on my machine and on Netlify). If I downgrade to version 2.10.5, build is working as expected.

Steps to reproduce

Run rimraf .cache public && gatsby build

Expected result

Build should succeed

Actual result

Build fails with out of memory error :

⠦ Building production JavaScript and CSS bundles

<— Last few GCs —>

[10733:0x10264e000] 241422 ms: Scavenge 1331.0 (1423.1) -> 1330.1 (1423.6) MB, 12.4 / 0.0 ms (average mu = 0.123, current mu = 0.086) allocation failure [10733:0x10264e000] 241437 ms: Scavenge 1331.1 (1423.6) -> 1330.2 (1424.1) MB, 12.0 / 0.0 ms (average mu = 0.123, current mu = 0.086) allocation failure [10733:0x10264e000] 241458 ms: Scavenge 1331.3 (1424.1) -> 1330.4 (1424.6) MB, 14.7 / 0.0 ms (average mu = 0.123, current mu = 0.086) allocation failure

<— JS stacktrace —>

==== JS stack trace =========================================

0: ExitFrame [pc: 0x2e6bd9b5be3d]

Security context: 0x261c7219e6e9 <JSObject> 1: /* anonymous /(aka / anonymous */) [0x261c46282299] [/Users/ssijak/work/hocudabudem.fit/node_modules/gatsby/node_modules/@babel/core/lib/transformation/file/merge-map.js:~155] [pc=0x2e6bdaf5a89b](this=0x261c319822b1 <null>,m=0x261c8d43ed71 <Object map = 0x261cf880a9c9>) 2: arguments adaptor frame: 3->1 3: forEach [0x261c72186df1](this=0x261c…

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory 1: 0x10003c597 node::Abort() [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 2: 0x10003c7a1 node::OnFatalError(char const*, char const*) [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 3: 0x1001ad575 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 4: 0x100579242 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 5: 0x10057bd15 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 6: 0x100577bbf v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 7: 0x100575d94 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 8: 0x10058262c v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 9: 0x1005826af v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 10: 0x100551ff4 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 11: 0x1007da044 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/ssijak/.nvm/versions/node/v10.15.3/bin/node] 12: 0x2e6bd9b5be3d 13: 0x2e6bdaf5a89b /bin/sh: line 1: 10733 Abort trap: 6 gatsby build error Command failed with exit code 134.

Environment

System: OS: macOS 10.14.5 CPU: (8) x64 Intel® Core™ i7-6820HQ CPU @ 2.70GHz Shell: 5.3 - /bin/zsh Binaries: Node: 10.15.3 - ~/.nvm/versions/node/v10.15.3/bin/node Yarn: 1.15.2 - ~/.nvm/versions/node/v10.15.3/bin/yarn npm: 6.4.1 - ~/.nvm/versions/node/v10.15.3/bin/npm Languages: Python: 2.7.10 - /usr/bin/python Browsers: Chrome: 75.0.3770.100 Safari: 12.1.1 npmPackages: gatsby: 2.10.5 => 2.10.5 gatsby-awesome-pagination: ^0.3.4 => 0.3.4 gatsby-background-image: ^0.7.4 => 0.7.4 gatsby-image: ^2.2.3 => 2.2.3 gatsby-plugin-favicon: ^3.1.6 => 3.1.6 gatsby-plugin-feed: ^2.3.1 => 2.3.1 gatsby-plugin-google-tagmanager: ^2.1.0 => 2.1.0 gatsby-plugin-lunr: ^1.5.1 => 1.5.1 gatsby-plugin-manifest: ^2.2.0 => 2.2.0 gatsby-plugin-netlify: ^2.1.0 => 2.1.0 gatsby-plugin-netlify-cms: ^4.1.0 => 4.1.0 gatsby-plugin-offline: ^2.2.0 => 2.2.0 gatsby-plugin-postcss: ^2.1.0 => 2.1.0 gatsby-plugin-prefetch-google-fonts: ^1.4.2 => 1.4.2 gatsby-plugin-purgecss: ^4.0.0 => 4.0.0 gatsby-plugin-react-helmet: ^3.1.0 => 3.1.0 gatsby-plugin-robots-txt: ^1.4.0 => 1.4.0 gatsby-plugin-sharp: ^2.2.1 => 2.2.1 gatsby-plugin-sitemap: ^2.2.1 => 2.2.1 gatsby-remark-copy-linked-files: ^2.1.0 => 2.1.0 gatsby-remark-embed-video: ^1.7.1 => 1.7.1 gatsby-remark-images: ^3.1.2 => 3.1.2 gatsby-remark-relative-images: ^0.2.2 => 0.2.2 gatsby-remark-responsive-image: ^1.0.0-alpha13-alpha.435e0178 => 1.0.0-alpha13-alpha.435e0178 gatsby-source-filesystem: ^2.1.1 => 2.1.1 gatsby-transformer-remark: ^2.5.0 => 2.5.0 gatsby-transformer-sharp: ^2.2.0 => 2.2.0 npmGlobalPackages: gatsby: 2.10.0

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 7
  • Comments: 43 (26 by maintainers)

Most upvoted comments

Oh sweet! let us get a fix out asap!

Thanks for the reproduction.

Had the same problem with 2.12.1+. Netlify builds were getting out of memory errors - but local builds were working OK. We’re using gatsby-plugin-netlify-cms@4.1.1

Reverted to 2.12.0 and Netlify builds are now working as expected.

There is a feature in 2.12.1:

enable babel in deps (#15284) (4ea3fa1)

And also a similar looking feature in 2.11.0:

gatsby: enable babel-loader for all dependencies (#14111) (268ed27)

Have a feeling running babel on dependencies has something to do with this issue…

It looks like version 2.11.0 introduced “enable babel-loader for all dependencies” (https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/CHANGELOG.md#2110-2019-06-27) and netlify-cms-app.js is just too big for babel. I’m getting this warning before node crashes:

[BABEL] Note: The code generator has deoptimised the styling of /Users/mml/Repositories/av/node_modules/netlify-cms-app/dist/netlify-cms-app.js as it exceeds the max of 500KB.

@lorensr You need to update gatsby-plugin-netlify-cms to 4.1.2 to fix the issue

I’m getting the same error but it works when I disable gatsby-plugin-netlify-cms in gatsby-config.js. Do you experience the same thing?

That fixed it, thank you! ☺️🙏

On Thu, Jul 11, 2019 at 1:39 AM Tommy Ang notifications@github.com wrote:

@lorensr https://github.com/lorensr You need to update gatsby-plugin-netlify-cms to 4.1.2 to fix the issue

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gatsbyjs/gatsby/issues/15256?email_source=notifications&email_token=AAB5LGEVFLAZOMIUFBTW623P63BRPA5CNFSM4H4MQ532YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZVR7TA#issuecomment-510336972, or mute the thread https://github.com/notifications/unsubscribe-auth/AAB5LGCNQSBQCCZTVXK3DATP63BRPANCNFSM4H4MQ53Q .

Just chiming in, I’m following this thread because I am experiencing the same heap out-of-memory error trying to build one of the more popular gatsby blog starters (no changes): https://www.gatsbyjs.org/starters/alxshelepenok/gatsby-starter-lumen/

Sorry I realize that’s not a “small” example 😅but it’s one I can suggest for reproducing the error.

To see the error, run:

gatsby new blog https://github.com/alxshelepenok/gatsby-starter-lumen
cd blog
npm install gatsby-plugin-netlify-cms@latest
npm run build

The package versions that get installed are gatsby v2.13.2 and gatsby-plugin-netlify-cms v4.1.1. Disabling gatsby-plugin-netlify-cms in gatsby-config.js will result in a successful build.

After a little search, I figured out that ‘gatsby-plugin-netlify-cms’ is not the problem in my case. The issue is happening due to ‘gatsby-plugin-material-ui’ requiring ‘autoprefixer’.

Any I ideas why this might be happening?

@wardpeet I ended up solving it, so no need to open an issue. It was an inherited project that was querying way more data than it needed. It was overusing too many fragments that weren’t specific enough. I ended up simplifying the data structure and stopped using image sharp transforms and the problem went away. I’m thinking for sites that do have really heavy data querying requirements it will fail, but maybe those sites are rare.

yes, let me fix that issue in netlify-plugin so we don’t break. And i’ll open up a new pr to add a deprection message

@womcauliff Thank you so much!

I’ve been trying all day to make the bug-repo I have created, to fail consistently and I couldn’t, where in my own project it does every time.

I have figured out that it most probably has to do with the size of the files (imports), so I guess being a large example is a good thing in this case.

UPDATE

I upgraded gatsby to v2.13.1 and now the babel issue I mentioned above is solved but I’m back having problem in build when using the ‘gatsby-plugin-netlify-cms’ plugin.

I get this error (multiple times…):

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

After some investigation I realised that it only happens when I import a component in cms.js to be used in CMS.registerPreviewTemplate.

I hope this helps

I’m having the same issue and receive a similar error message but for @fortawesome/free-solid-svg-icons. I tried increasing the memory limit like node --max_old_space_size=12000 ./node_modules/.bin/gatsby build --prefix-paths but that didn’t accomplish anything.

When looking in htop I can see 4-5 gatsby processes running up to about 1GB of memory and then all jump down again only to slowly climb again and repeat. Not sure if that’s helpful.