newman: Fatal memory error with large response

Newman Version (can be found via newman -v): 4.1.0
OS details (type, version, and architecture): Alpine Linux 3.7.0, in docker container
Are you using Newman as a library, or via the CLI? CLI
Did you encounter this recently, or has this bug always been there: Recently
Expected behaviour: No timeouts during long collection or test script execution when options are omitted. No out-of-memory errors
Command / script used to run Newman: newman run ./collections/$COLLECTION_NAME.json -e ./env/$ENV_NAME.json
Sample collection, and auxiliary files (minus the sensitive details):
Screenshots (if applicable):

Steps to reproduce the problem:

Run attached collection (JSON, saved with bogus .txt extension). Randomly selected large JSON file from the internet.

→ get_all_episodes
  GET https://raw.githubusercontent.com/zemirco/sf-city-lots-json/master/citylots.json [200 OK, 21.79MB, 19.4s]

<--- Last few GCs --->

[1:0x55698c4e2000]    28027 ms: Mark-sweep 655.2 (666.6) -> 439.2 (450.6) MB, 280.1 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 3337 ms) allocation failure GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x4c364725ee1 <JSObject>
    1: toJSON [buffer.js:~938] [pc=0x15daddbc657d](this=0x2d8a4dcf68d1 <Uint8Array map = 0xee6886417b9>)
    3: /* anonymous */ [/usr/lib/node_modules/newman/node_modules/postman-collection/lib/collection/property-base.js:133] [bytecode=0x17720956b749 offset=100](this=0x3b2e4f48bde9 <JSGlobal Object>,accumulator=0x2d8a4dc84931 <Object map = 0x21f9f4330a39>,value=0x2d8a4dcf68d1 <Uint8Array map = 0...

FATAL ERROR: invalid array length Allocation failed - JavaScript heap out of memory

episode-editor.json.txt

I noticed this problem initially when a job was failing with Script execution timed out, but adjusting timeout settings and upgrading to 4.10.0 to use its default Infinite setting didn’t help.

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 15 (5 by maintainers)

Most upvoted comments

This issue still exists. This is a VM with 10GB of ram with NODE_OPTIONS=“–max-old-space-size=8192”. The response is a file of 137.98MB. The file is downloaded, but then we get the fatal memory error. Smaller responses are fine.

$ newman -v 5.2.2

$ node -v v14.16.0

<--- Last few GCs --->

[7232:000000BAE331C240]    93192 ms: Scavenge 1163.6 (1200.8) -> 1163.6 (1200.8) MB, 105.5 / 0.0 ms  (average mu = 0.985, current mu = 0.981) allocation failure
[7232:000000BAE331C240]    94383 ms: Scavenge 1546.2 (1583.3) -> 1546.2 (1583.3) MB, 161.4 / 0.0 ms  (average mu = 0.985, current mu = 0.981) allocation failure
[7232:000000BAE331C240]    96250 ms: Scavenge 2120.0 (2157.1) -> 2120.0 (2157.1) MB, 243.3 / 0.0 ms  (average mu = 0.985, current mu = 0.981) allocation failure

<--- JS stacktrace --->

FATAL ERROR: invalid array length Allocation failed - JavaScript heap out of memory
 1: 00007FF6E461052F napi_wrap+109311
 2: 00007FF6E45B5256 v8::internal::OrderedHashTable<v8::internal::OrderedHashSet,1>::NumberOfElementsOffset+33302
 3: 00007FF6E45B6026 node::OnFatalError+294
 4: 00007FF6E4E8163E v8::Isolate::ReportExternalAllocationLimitReached+94
 5: 00007FF6E4E664BD v8::SharedArrayBuffer::Externalize+781
 6: 00007FF6E4D1094C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1516
 7: 00007FF6E4D3547F v8::internal::Factory::NewUninitializedFixedArray+111
 8: 00007FF6E4BFB3C0 v8::Object::GetIsolate+8128
 9: 00007FF6E4A851F7 v8::internal::interpreter::JumpTableTargetOffsets::iterator::operator=+169671
10: 00007FF6E4F09FED v8::internal::SetupIsolateDelegate::SetupHeap+463949
11: 000003D543A02C98

swedkg on Mar 16, 2021

Another customer has experienced a similar issue. They have tried the following and the problem still persists. I have attached a .txt of the error output as well.

newman version: 4.5.6
Node version: v12.13.1

[NODE_OPTIONS="--max-old-space-size=2048"](url
[error.txt](https://github.com/postmanlabs/newman/files/3913004/error.txt)
) newman run ./collections/$COLLECTION_NAME.json -e ./env/$ENV_NAME.json

Set NODE_OPTIONS first
SET NODE_OPTIONS="--max-old-space-size=2048"

Run newman next
newman run <collection_filenam> -e <environment_filename>

asi-postman on Dec 2, 2019

@shamasis The mentioned problem is still reproducable, even with node v11 Do you have any other suggestions to us, to solve this unfortunate issue?

DieGraueEminenz on Oct 18, 2019