aws-sdk-js: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Environment

React Native Environment Info: System: OS: macOS 10.14 CPU: (8) x64 Intel® Core™ i7-4770HQ CPU @ 2.20GHz Memory: 2.16 GB / 16.00 GB Shell: 5.3 - /bin/zsh Binaries: Node: 10.15.3 - ~/.nvm/versions/node/v10.15.3/bin/node Yarn: 1.16.0 - ~/.nvm/versions/node/v10.15.3/bin/yarn npm: 6.9.0 - ~/.nvm/versions/node/v10.15.3/bin/npm Watchman: 4.9.0 - /usr/local/bin/watchman SDKs: iOS SDK: Platforms: iOS 12.1, macOS 10.14, tvOS 12.1, watchOS 5.1 Android SDK: API Levels: 25, 27, 28 Build Tools: 27.0.3, 28.0.2, 28.0.3 IDEs: Android Studio: 3.2 AI-181.5540.7.32.5056338 Xcode: 10.1/10B61 - /usr/bin/xcodebuild npmPackages: react: 16.8.3 => 16.8.3 react-native: 0.59.8 => 0.59.8 npmGlobalPackages: react-native-cli: 2.0.1

Description

I am using aws-sdk (https://github.com/aws/aws-sdk-js) in my react-native project. After upgrading v0.56.1 to v0.59.8, metro bundler shows error below.

transform[stdout]:
transform[stdout]: <--- Last few GCs --->
transform[stdout]:
transform[stdout]: [26907:0x103800c00] 87297 ms: Mark-sweep 1319.0 (1447.2) -> 1316.2 (1450.7) MB, 2824.6 / 0.0 ms (average mu = 0.140, current mu = 0.015) allocation failure scavenge might not succeed
transform[stdout]: [26907:0x103800c00] 90564 ms: Mark-sweep 1325.3 (1450.7) -> 1322.4 (1452.7) MB, 3250.7 / 0.0 ms (average mu = 0.070, current mu = 0.005) allocation failure scavenge might not succeed
transform[stdout]:
transform[stdout]:
transform[stdout]: <--- JS stacktrace --->
transform[stdout]:
transform[stdout]: ==== JS stack trace =========================================
transform[stdout]:
transform[stdout]: 0: ExitFrame [pc: 0x323bb255be3d]
transform[stdout]: Security context: 0x1e631a09e6e9 
transform[stdout]: 1: new NodePath [0x1e63e18bb4a1] [/Users/O/Documents/dev/togather/rn59AwsDemo/node_modules/@babel/traverse/lib/path/index.js:~75] [pc=0x323bb2950df0](this=0x1e638de5e4e9 ,hub=0x1e6397c33a19 ,parent=0x1e637fe54981 )
transform[stdout]: 2: ConstructFrame [pc: 0x323bb250d02f]
transform[stdout]: 3: /...
transform[stdout]:
transform[stderr]: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
transform[stderr]: 1: 0x10003c597 node::Abort() [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 2: 0x10003c7a1 node::OnFatalError(char const, char const*) [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 3: 0x1001ad575 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 4: 0x100579242 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 5: 0x10057bd15 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 6: 0x100577bbf v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 7: 0x100575d94 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 8: 0x10058262c v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 9: 0x1005826af v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 10: 0x100551ff4 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 11: 0x1007da044 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/O/.nvm/versions/node/v10.15.3/bin/node]
transform[stderr]: 12: 0x323bb255be3d
transform[stderr]: 13: 0x323bb2950df0
transform[stderr]: 14: 0x323bb250d02f

Reproducible Demo

I tested on vanilla RN project (created by react-native init, and added only aws-sdk) and it shows same result.

https://github.com/artechventure/rn59AwsDemo

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 10
Comments: 30 (6 by maintainers)

Most upvoted comments

@amitgilad3 I have the same setup and I fixed it by using this import (aws-sdk 2.769.0)

import S3 from 'aws-sdk/clients/s3';

+16

StefanFeser on Oct 7, 2020

Hi @artechventure

Thank you for the reproduce repository, we will look into this issue. There are some solutions from previous issues worth trying. You can have a look at #1877 and see if they solve your problem.

AllanZhengYP on May 23, 2019

@amitgilad3 I have the same setup and I fixed it by using this import (aws-sdk 2.769.0)
import S3 from 'aws-sdk/clients/s3';

Can confirm! Cut around 350 mb on nextjs, "aws-sdk": "^2.784.0",

MonstraG on Dec 8, 2020

I was facing the same problem and after upgrading node from v10.15.3 to v12.18.0 the issue went away.

russpitre on Jul 16, 2020

I’m still getting this. I’m using "aws-sdk": "2.767.0" with expo 38 (bare workflow)

System:
    OS: Linux 5.4 Ubuntu 18.04.5 LTS (Bionic Beaver)
    CPU: (8) x64 Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
    Memory: 5.59 GB / 15.10 GB
    Shell: Unknown
  Binaries:
    Node: 12.13.0 - /usr/local/bin/node
    Yarn: 1.13.0 - /usr/local/bin/yarn
    npm: 6.12.0 - /usr/local/bin/npm
    Watchman: Not Found
  SDKs:
    Android SDK:
      Android NDK: 20.0.5594570
  IDEs:
    Android Studio: Not Found
  Languages:
    Java: Not Found
    Python: 2.7.17 - /usr/bin/python
  npmPackages:
    @react-native-community/cli: Not Found
    react: ~16.11.0 => 16.11.0 
    react-native: 0.62.2 => 0.62.2

I’m running node v12.13.0

I’ve tried modifying my .bashrc to include this line: export NODE_OPTIONS=--max_old_space_size=4096

I’ve tried modifying my app/build.gradle:

project.ext.react = [
    enableHermes: false,
    nodeExecutableAndArgs: ["node", "--max-old-space-size=8192"],
    inputExcludes: ["node_modules/aws-sdk/dist/aws-sdk-react-native.js"]
]

I’ve modified my package.json to include:

  "babel": {
    "ignore": [
      "node_modules/aws-sdk/dist/aws-sdk-react-native.js"
    ]
  },

I’ve tried using both import AWS from 'aws-sdk/dist/aws-sdk-react-native'; and var AWS = require('aws-sdk/dist/aws-sdk-react-native');

I’ve tried increasing my Android Studio memory - IDE max heap size, Gradle daemon max heap size, and Kotlin daemon max heap size are all at the maximum 4096mb.

Something that I haven’t seen suggested here is editing your gradle.properties to include:

org.gradle.daemon=true
org.gradle.jvmargs=-Xmx4096m

The above seems to help produce release builds more reliably (but only just)

Since installing aws-sdk, my release builds now take 2-3 times as long (at least) and fail 80% of the time, if they don’t completely lock up my laptop.

Unfortunately using another package is not an option for me as I need to put records directly into a Kinesis stream.

Ultimately, I don’t understand why I need to increase the maximum memory required to build my app just to accommodate this specific SDK.

Hopefully v3 will resolve this issue but it’s still a release candidate and so not an option for production use.

CantFayle on Oct 23, 2020

Thanks @ausshadu for verifying.

Is the issue fixed in version ~2.740.0 for others? cc @artechventure @4RGUS @Argus444 @tedi0722 @zholmes1 @SaveYourTime @aaadarshbisht

trivikr on Sep 9, 2020

This was fixed in 2.740.0. It works like charm now.

ausshadu on Sep 9, 2020

I have the same issue on my nestjs application, when I deploy it to heroku

splNovikov on Mar 15, 2020

Encountering this issue when adding aws-sdk to a NodeJS app

zholmes1 on Feb 10, 2020