MeshCentral: Create Child instance crashing on web connection

I have not seen this issue brought up anywhere else, so it is highly possible that it is isolated to my specific environment.

I am using Ubuntu 18.04.5 LTS Just updated to the newest version of MeshCentral, upgraded the server fully, and updated node/npm packages, in hopes my issue would be remedied. We have about 50 devices on the system.

The meshcentral service seems to come up just fine, but the service crashes when trying to connect to the web portal, with the following:

Dec 29 21:53:32 rmm node[3086]: <--- Last few GCs --->
Dec 29 21:53:32 rmm node[3086]: [3184:0x3763490]    84016 ms: Mark-sweep 2044.5 (2050.2) -> 2044.2 (2049.9) MB, 3451.9 / 0.0 ms  (average mu = 0.126, current mu = 0.004) allocation failure scavenge might not succeed
Dec 29 21:53:32 rmm node[3086]: [3184:0x3763490]    87636 ms: Mark-sweep 2044.8 (2050.2) -> 2044.6 (2050.4) MB, 3611.6 / 0.0 ms  (average mu = 0.063, current mu = 0.002) allocation failure scavenge might not succeed
Dec 29 21:53:32 rmm node[3086]: <--- JS stacktrace --->
Dec 29 21:53:32 rmm node[3086]: ==== JS stack trace =========================================
Dec 29 21:53:32 rmm node[3086]: Security context: 0x1abc415008d1 <JSObject>
Dec 29 21:53:32 rmm node[3086]:     0: builtin exit frame: stringify(this=0x1abc4151ec89 <Object map = 0x185eefc43549>,0x221d53d404b1 <undefined>,0x2d454cae4e11 <JSFunction (sfi = 0x4a2de91e691)>,0x17672604c119 <Object map = 0x185eefc7df19>,0x1abc4151ec89 <Object map = 0x185eefc43549>)
Dec 29 21:53:32 rmm node[3086]:     1: arguments adaptor frame: 2->3
Dec 29 21:53:32 rmm node[3086]:     2: /* anonymous */ [0x1ff19d940119] [/opt/meshcentral/node_modules/nedb/lib/persistence.js:~128] [pc=...
Dec 29 21:53:32 rmm node[3086]: ERR: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
Dec 29 21:53:32 rmm node[3086]:  1: 0x9ef190 node::Abort() [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  2: 0x9f13b2 node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  3: 0xb5da9e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR:  4: 0xb5de19 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  5: 0xd0a765  [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  6: 0xd0adf6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR:  7: 0xd1760a v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR:  8: 0xd18515 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR:  9: 0xd1afcc v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR: 10: 0xce8c8c v8::internal::Factory::NewRawOneByteString(int, v8::internal::AllocationType) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR: 11: 0x10a2305 v8::internal::IncrementalStringBuilder::Extend() [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR: 12: 0xdff298 v8::internal::JsonStringifier::SerializeString(v8::internal::Handle<v8::internal::String>) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR: 13: 0xe009f1 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR: 14: 0xe04f06 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR: 15: 0xe058b4 v8::internal::JsonStringify(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR: 16: 0xc0f24b v8::internal::Builtin_JsonStringify(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: ERR: 17: 0x13a72b9  [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: Error: Command failed: /usr/local/bin/node /opt/meshcentral/node_modules/meshcentral --launch 3086
Dec 29 21:53:32 rmm node[3086]: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
Dec 29 21:53:32 rmm node[3086]:  1: 0x9ef190 node::Abort() [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  2: 0x9f13b2 node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  3: 0xb5da9e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  4: 0xb5de19 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  5: 0xd0a765  [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  6: 0xd0adf6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  7: 0xd1760a v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  8: 0xd18515 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:  9: 0xd1afcc v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: 10: 0xce8c8c v8::internal::Factory::NewRawOneByteString(int, v8::internal::AllocationType) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: 11: 0x10a2305 v8::internal::IncrementalStringBuilder::Extend() [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: 12: 0xdff298 v8::internal::JsonStringifier::SerializeString(v8::internal::Handle<v8::internal::String>) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: 13: 0xe009f1 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: 14: 0xe04f06 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: 15: 0xe058b4 v8::internal::JsonStringify(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: 16: 0xc0f24b v8::internal::Builtin_JsonStringify(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]: 17: 0x13a72b9  [/usr/local/bin/node]
Dec 29 21:53:32 rmm node[3086]:     at ChildProcess.exithandler (child_process.js:303:12)
Dec 29 21:53:32 rmm node[3086]:     at ChildProcess.emit (events.js:323:22)
Dec 29 21:53:32 rmm node[3086]:     at maybeClose (internal/child_process.js:1021:16)
Dec 29 21:53:32 rmm node[3086]:     at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5) {
Dec 29 21:53:32 rmm node[3086]:   killed: false,
Dec 29 21:53:32 rmm node[3086]:   code: null,
Dec 29 21:53:32 rmm node[3086]:   signal: 'SIGABRT',
Dec 29 21:53:32 rmm node[3086]:   cmd: '/usr/local/bin/node /opt/meshcentral/node_modules/meshcentral --launch 3086'
Dec 29 21:53:32 rmm node[3086]: }
Dec 29 21:53:32 rmm node[3086]: ERROR: MeshCentral failed with critical error, check MeshErrors.txt. Restarting in 5 seconds...
Dec 29 21:53:39 rmm node[3086]: MeshCentral HTTP redirection server running on port 80.
Dec 29 21:53:39 rmm node[3086]: MeshCentral v0.7.32, WAN mode, Production mode.
Dec 29 21:53:40 rmm node[3086]: MeshCentral Intel(R) AMT server running on rmm:4433.
Dec 29 21:53:40 rmm node[3086]: MeshCentral HTTPS server running on rmm:443.

I am at a loss, and completely unfamiliar with NodeJS. Any suggestions would be welcome.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 17 (10 by maintainers)

Most upvoted comments

Ok. Installed latest LTS Node version (14.15.3). Still not working with that one but I will reboot and check it out. I will play around with it and see what combination of mesh and node work correctly.