remix: [Bug]: JavaScript heap out of memory

Which Remix packages are impacted?

remix (Remix core)
@remix-run/dev
@remix-run/node
@remix-run/react
@remix-run/serve
@remix-run/vercel

What version of Remix are you using?

v1.1.1

What version of Node are you using? Minimum supported version is 14.

v16.13.1

Steps to Reproduce

It’s just a regular remix app and sometimes when saving something in the codebase the server crashes and gives me the error pasted below, relating to JS memory issues. Looks like something to do with esbuild but I’m not really sure, just wondering if anyone else ever has this?

Expected Behavior

Not crash

Actual Behavior

<--- Last few GCs --->

[37477:0x130008000]  7075356 ms: Mark-sweep 4025.7 (4134.4) -> 4015.9 (4140.4) MB, 3782.7 / 0.0 ms  (average mu = 0.694, current mu = 0.007) allocation failure scavenge might not succeed
[37477:0x130008000]  7095099 ms: Mark-sweep 4034.0 (4142.4) -> 4023.3 (4147.9) MB, 4517.8 / 0.0 ms  (average mu = 0.742, current mu = 0.771) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0x102d1846c node::Abort() [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
 2: 0x102d185f4 node::errors::TryCatchScope::~TryCatchScope() [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
 3: 0x102e667b4 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
 4: 0x102e66748 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
 5: 0x102fe9c08 v8::internal::Heap::GarbageCollectionReasonToString(v8::internal::GarbageCollectionReason) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
 6: 0x102fe8728 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
 7: 0x102ff3a04 v8::internal::Heap::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
 8: 0x102ff3a98 v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
 9: 0x102fbece4 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArrayWithFiller(v8::internal::Handle<v8::internal::Map>, int, v8::internal::Handle<v8::internal::Oddball>, v8::internal::AllocationType) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
10: 0x1031f5638 v8::internal::Handle<v8::internal::NameDictionary> v8::internal::HashTable<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::EnsureCapacity<v8::internal::Isolate>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::NameDictionary>, int, v8::internal::AllocationType) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
11: 0x1031f5fe0 v8::internal::Handle<v8::internal::NameDictionary> v8::internal::Dictionary<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::Add<v8::internal::Isolate>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::NameDictionary>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDetails, v8::internal::InternalIndex*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
12: 0x1031f5f54 v8::internal::BaseNameDictionary<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::Add(v8::internal::Isolate*, v8::internal::Handle<v8::internal::NameDictionary>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDetails, v8::internal::InternalIndex*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
13: 0x1031ba59c v8::internal::JSObject::SetNormalizedProperty(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDetails) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
14: 0x1031d7268 v8::internal::LookupIterator::TransitionToAccessorPair(v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
15: 0x1031d6b50 v8::internal::LookupIterator::TransitionToAccessorProperty(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
16: 0x1031b53d4 v8::internal::JSObject::DefineAccessor(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
17: 0x1031b50dc v8::internal::JSReceiver::ValidateAndApplyPropertyDescriptor(v8::internal::Isolate*, v8::internal::LookupIterator*, bool, v8::internal::PropertyDescriptor*, v8::internal::PropertyDescriptor*, v8::Maybe<v8::internal::ShouldThrow>, v8::internal::Handle<v8::internal::Name>) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
18: 0x1031b4428 v8::internal::JSReceiver::OrdinaryDefineOwnProperty(v8::internal::LookupIterator*, v8::internal::PropertyDescriptor*, v8::Maybe<v8::internal::ShouldThrow>) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
19: 0x1031b3f64 v8::internal::JSReceiver::OrdinaryDefineOwnProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSObject>, v8::internal::PropertyKey const&, v8::internal::PropertyDescriptor*, v8::Maybe<v8::internal::ShouldThrow>) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
20: 0x1031b3b74 v8::internal::JSReceiver::DefineOwnProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDescriptor*, v8::Maybe<v8::internal::ShouldThrow>) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
21: 0x1031b3a0c v8::internal::JSReceiver::DefineProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
22: 0x102ef1164 v8::internal::Builtin_ObjectDefineProperty(int, unsigned long*, v8::internal::Isolate*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
23: 0x10361042c Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
24: 0x1082b4198
25: 0x1082b37e4
26: 0x1035a3f78 Builtins_InterpreterEntryTrampoline [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
27: 0x107e38a4c
28: 0x107731ddc
29: 0x107922624
30: 0x107f51e98
31: 0x107a714cc
32: 0x107cab644
33: 0x1079f1d20
34: 0x1079f1be8
35: 0x1079f1be8
36: 0x1079f1be8
37: 0x1079f1be8
38: 0x1079f1be8
39: 0x1079f1be8
40: 0x10784b71c
41: 0x107cab644
42: 0x1079dcba0
43: 0x107acb0e0
44: 0x107acb40c
45: 0x107e20784
46: 0x1079d61a0
47: 0x107a72e0c
48: 0x107e20a08
49: 0x1079d61a0
50: 0x107ad59b4
51: 0x107b70e74
52: 0x1079dcf34
53: 0x107bae498
54: 0x107b15d1c
55: 0x107b70940
56: 0x107af1ef4
57: 0x107832bec
58: 0x1035a1d6c Builtins_JSEntryTrampoline [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
59: 0x1035a1a04 Builtins_JSEntry [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
60: 0x102f76488 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
61: 0x102f75b1c v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
62: 0x102e830c0 v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
63: 0x102c68280 node::InternalMakeCallback(node::Environment*, v8::Local<v8::Object>, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*, node::async_context) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
64: 0x102c7d4ac node::AsyncWrap::MakeCallback(v8::Local<v8::Function>, int, v8::Local<v8::Value>*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
65: 0x102d1c628 node::fs::FSReqCallback::Reject(v8::Local<v8::Value>) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
66: 0x102d1cdf4 node::fs::FSReqAfterScope::Reject(uv_fs_s*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
67: 0x102d1cf1c node::fs::AfterStat(uv_fs_s*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
68: 0x1035823b0 uv__work_done [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
69: 0x103585b10 uv__async_io [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
70: 0x103597824 uv__io_poll [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
71: 0x103585fa0 uv_run [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
72: 0x102c68e00 node::SpinEventLoop(node::Environment*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
73: 0x102d51638 node::NodeMainInstance::Run(int*, node::Environment*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
74: 0x102d51304 node::NodeMainInstance::Run(node::EnvSerializeInfo const*) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
75: 0x102cea2ec node::Start(int, char**) [/Users/jclackett/.fnm/node-versions/v16.13.1/installation/bin/node]
76: 0x1075f50f4
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x140000b4c28)
	runtime/sema.go:56 +0x38
sync.(*WaitGroup).Wait(0x140000b4c20)
	sync/waitgroup.go:130 +0xa4
main.runService(0x1)
	github.com/evanw/esbuild/cmd/esbuild/service.go:138 +0x494
main.main()
	github.com/evanw/esbuild/cmd/esbuild/main.go:203 +0x1f0

goroutine 18 [chan receive]:
main.runService.func1(0x140000cc140, 0x140000b4c20)
	github.com/evanw/esbuild/cmd/esbuild/service.go:66 +0x3c
created by main.runService
	github.com/evanw/esbuild/cmd/esbuild/service.go:64 +0x1a8

goroutine 19 [chan receive]:
main.(*serviceType).sendRequest(0x140000cc140, {0x102c3b320, 0x140044368a0})
	github.com/evanw/esbuild/cmd/esbuild/service.go:163 +0x104
main.runService.func2(0x140000cc140)
	github.com/evanw/esbuild/cmd/esbuild/service.go:92 +0x40
created by main.runService
	github.com/evanw/esbuild/cmd/esbuild/service.go:89 +0x2b8
error Command failed with signal "SIGABRT".

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 27
Comments: 37 (10 by maintainers)

Commits related to this issue

fix(remix-dev/cli): JavaScript heap out of memory closing #1555 I'm not sure of the fix though because I don't understand the whole logic there but the bug was a pain in the *ss before that "f... — committed to arnaudambro/remix by arnaudambro 2 years ago
fix(remix-dev/cli): JavaScript heap out of memory closing #1555 I'm not sure of the fix though because I don't understand the whole logic there but the bug was a pain in the *ss before that "fi... — committed to arnaudambro/remix by arnaudambro 2 years ago

Most upvoted comments

Happening on 1.7.5 too, remix is really bad at this. Development experience with this issue is terrible.

harisb2012 on Nov 5, 2022

Yes I plan on submitting a PR to fix this. I’m flying back today from Remix Conf which was awesome.

kiliman on May 26, 2022

@kentcdodds @jacob-ebey sorry for being pushy, but is this on your radar?

viczam on Feb 14, 2022

Fixed by #5133, which will make the new dev server available in the next minor release under the unstable_dev future flag.

Note that we needed to rearchitect the dev server as moving the require cache purging to onRebuildFinished in the current dev server caused race conditions

MichaelDeBoey on Jan 22, 2023

On every page load memory usage grows by a few megabytes, consistently. I’ve been trying to figure it out using Node’s heap dumps, but it’s just to much in there that I cannot figure it out.

However, I found a lot of identical cases of… I think it’s cached code.

Trying to dig down in the cache I can see a lot of cached node_module files in there.

Is there some cache built-in somewhere that isn’t flushed or reused on a reload so it just sticks around forever? I don’t know, and I don’t understand what I’m looking at.

If you want to have a look, run node --inspect ./node_modules/.bin/remix dev and open a Chromium-based browser to chrome://inspect and make memory snapshots. Perhaps someone else can make sense of this.

Mange on Nov 25, 2022

Getting this constantly on a large app, seems like after a couple save/LiveReload cycles, it blows up. In active development, it’s happening every 5-10 minutes.

Setting NODE_OPTIONS="--max-old-space-size=8192" seems to have temporarily fixed this – working for a couple hours now with no out of memory errors.

dmarkow on May 26, 2022

Still happening on 1.3.4

nicksrandall on Apr 4, 2022

I found that the source of this error for me was importing a large (20mb) json file as

import Messages from '../messages.json' - the way I fixed it was making a new messages.server.ts file and doing a node file read instead.

import path from "path";
import fs from "fs/promises";

const messagePath = path.join(__dirname, "..", "messages.json");

export async function getMessages() {
  const file = await fs.readFile(messagePath, "utf8");
  return JSON.parse(file);
}

I guess this is happening because using an import is continually adding the json to memory and the garbage collector either isn’t keeping up or isn’t freeing up that memory at all. But the node file system read does. Just speculation as I stopped looking in to it once I found my fix.

jondcallahan on Mar 26, 2022

@JClackett I’ve just tested it on your remix boilerplate and it used half the memory was using before (I had to remove the sentry imports though). I used autocannon for that: autocannon -d 30 -c 50 http://localhost:3000/posts

viczam on Feb 15, 2022