next.js: Next 9.5.1 out of memory after some hot reloads

Bug report

Describe the bug

Since updating to 9.5.x (from 9.4.x), i get an out of memory error after 10 something hot reloads:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

it did rarely happen in 9.4.1, but it happens very consistantly in 9.5.x

To Reproduce

thats probably tricky. it happens on big projects and might be related to some bug in the hot reload / rebuild. Maybe it happens when there are some import circles?

Expected behavior

nextjs should not go out-of-memory

System information

  • OS:macOS
  • Browser: chrome
  • Version of Next.js: 9.5.1
  • Version of Node.js: 12.13.1

Additional information

we are using a custom server with typescript

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 177
  • Comments: 144 (18 by maintainers)

Commits related to this issue

Most upvoted comments

Please provide a reproduction so that we can investigate

this is extremly hard to do, but i am 100% sure others have the same issue. Do you hav any idea how this could happen? Its a new problem and happens very consistantly.

maybe circular dependencies? I investigate now in that

Just a small update:

Used node-oom-heapdump to grab a heap snapshot of our app when it crashed.

Screenshot 2020-11-13 at 12 18 22

It seems to call out one of Webpack’s dependencies, watchpack (and its DirectoryWatcher.js), which makes sense to me because my crashes happen the most when I’m in dev mode, switching between branches that have a large number of changed files between them.

I have sent the heap snapshot to the Next.js team so hopefully it will help them look into this. I think I’m potentially going to open an issue on the Watchpack repo as well.

but i am 100% sure others have the same issue

Haven’t heard anything from our development teams or others, vercel.com is a Next.js app with 250 unique Next.js pages so we would have definitely have run into it if it was as trivial as you’re making it to be.

Do you hav any idea how this could happen?

Issues with memory are practically impossible to investigate without having the application so now I can’t give you pointers on where to look. You’ll have to run tooling to track down where memory is allocated (e.g. the Node.js debugger profiler).

If it makes it any easier to share the application we can also investigate it under (paid) enterprise support if that makes the company you work for share the application, which will likely save your development team significant amounts of time.

I have been experiencing memory leak since 9.5.2 (I updated yesterday). I was working on the /api/ files yesterday and this morning when it will randomly crash during hot reloads.

Since today afternoon, I was working on material-ui and some api

After I kept making changes to makeStyles, and changing the classes back and forth (because I’m trying to get the layout right), then it will crash after quite a few hot reload compiling.

Below, shows the number of hot reloads (in between the 2 crashes, I was working on material-ui’s makeStyles and adding/removing classes to different component) and compiles while I’m playing with makeStyles and material-ui’s Accordion component.

I’m not sure whether the logs below help, but I have to say the crashes are all pretty random, so far crashed about 10-15 times in the span of 24 hours, sometimes when working on /api/ sometimes while working on material-ui.

wait - compiling…

<— Last few GCs —>

[3511:0x4ac6840] 1722755 ms: Scavenge 454.2 (462.5) -> 451.2 (463.0) MB, 3.2 / 0.1 ms (average mu = 0.984, current mu = 0.618) allocation failure [3511:0x4ac6840] 1722796 ms: Scavenge 455.0 (463.0) -> 451.5 (463.0) MB, 1.8 / 0.0 ms (average mu = 0.984, current mu = 0.618) allocation failure [3511:0x4ac6840] 1723246 ms: Mark-sweep 474.1 (484.8) -> 461.6 (482.1) MB, 400.5 / 0.1 ms (average mu = 0.987, current mu = 0.989) allocation failure scavenge might not succeed

<— JS stacktrace —>

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory 1: 0xa3ac30 node::Abort() [node] 2: 0x98a45d node::FatalError(char const*, char const*) [node] 3: 0xbae25e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node] 4: 0xbae5d7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node] 5: 0xd56125 [node] 6: 0xd8178e v8::internal::EvacuateNewSpaceVisitor::Visit(v8::internal::HeapObject, int) [node] 7: 0xd8da96 v8::internal::FullEvacuator::RawEvacuatePage(v8::internal::MemoryChunk*, long*) [node] 8: 0xd7b4ef v8::internal::Evacuator::EvacuatePage(v8::internal::MemoryChunk*) [node] 9: 0xd7b768 v8::internal::PageEvacuationTask::RunInParallel(v8::internal::ItemParallelJob::Task::Runner) [node] 10: 0xd70dc9 v8::internal::ItemParallelJob::Run() [node] 11: 0xd8fbfe void v8::internal::MarkCompactCollectorBase::CreateAndExecuteEvacuationTasks<v8::internal::FullEvacuator, v8::internal::MarkCompactCollector>(v8::internal::MarkCompactCollector*, v8::internal::ItemParallelJob*, v8::internal::MigrationObserver*, long) [node] 12: 0xd90494 v8::internal::MarkCompactCollector::EvacuatePagesInParallel() [node] 13: 0xd90665 v8::internal::MarkCompactCollector::Evacuate() [node] 14: 0xda1857 v8::internal::MarkCompactCollector::CollectGarbage() [node] 15: 0xd63dd9 v8::internal::Heap::MarkCompact() [node] 16: 0xd64c0b v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node] 17: 0xd65684 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node] 18: 0xd680fc v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node] 19: 0xd2f3aa v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [node] 20: 0xd29254 v8::internal::FactoryBasev8::internal::Factory::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Map, v8::internal::AllocationAlignment) [node] 21: 0xd2b5a1 v8::internal::FactoryBasev8::internal::Factory::NewRawTwoByteString(int, v8::internal::AllocationType) [node] 22: 0xf89565 v8::internal::String::SlowFlatten(v8::internal::Isolate*, v8::internal::Handlev8::internal::ConsString, v8::internal::AllocationType) [node] 23: 0xbc1369 v8::String::Utf8Length(v8::Isolate*) const [node] 24: 0xa13067 [node] 25: 0xc19bab [node] 26: 0xc1b156 [node] 27: 0xc1b7d6 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [node] 28: 0x13f5159 [node] Aborted (core dumped) npm ERR! code ELIFECYCLE npm ERR! errno 134 npm ERR! project@1.0.0 dev: next dev -p 6000 npm ERR! Exit status 134 npm ERR! npm ERR! Failed at the project@1.0.0 dev script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in: npm ERR! /home/ubuntu/.npm/_logs/2020-08-15T17_43_39_015Z-debug.log ubuntu@ip-172-31-35-196:~/project2$ npm run dev

project@1.0.0 dev /home/ubuntu/project next dev -p 6000

info - Loaded env from /home/ubuntu/project2/.env.local ready - started server on http://localhost:6000 event - compiled successfully event - build page: /next/dist/pages/_error wait - compiling… event - compiled successfully event - build page: /dashboard/customer/[pid] wait - compiling… event - compiled successfully event - build page: /api/dashboard/[[…path]] wait - compiling… event - compiled successfully wait - compiling… event - compiled successfully wait - compiling… event - compiled successfully event - build page: /api/dashboard/[[…path]] wait - compiling… event - compiled successfully wait - compiling… event - compiled successfully wait - compiling… event - compiled successfully wait - compiling… error - ./src/components/Dashboard/Activity.js:25:25 Syntax error: Identifier directly after number

23 | accordionExpandedNoExtraMargin: { 24 |

25 | minHeight: 48px !important’ | ^ 26 | }, 27 | expandNothing: { 28 | wait - compiling… event - compiled successfully event - build page: /dashboard wait - compiling… event - compiled successfully wait - compiling… event - compiled successfully wait - compiling… event - compiled successfully wait - compiling… <— Last few GCs —>

[3573:0x503a840] 357009 ms: Scavenge 454.7 (462.2) -> 452.0 (462.9) MB, 3.1 / 0.0 ms (average mu = 0.950, current mu = 0.668) allocation failure [3573:0x503a840] 357025 ms: Scavenge 456.3 (464.6) -> 454.2 (465.1) MB, 2.7 / 0.0 ms (average mu = 0.950, current mu = 0.668) allocation failure [3573:0x503a840] 357472 ms: Mark-sweep 476.2 (486.9) -> 456.4 (481.3) MB, 382.0 / 0.1 ms (average mu = 0.990, current mu = 0.994) allocation failure scavenge might not succeed

<— JS stacktrace —>

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory 1: 0xa3ac30 node::Abort() [node] 2: 0x98a45d node::FatalError(char const*, char const*) [node] 3: 0xbae25e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node] 4: 0xbae5d7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]

Same issue with latest 12.0.0

Still happening with latest v10, in case anyone was hoping it could’ve been solved by chance.

Quick follow up: these are some messages I’ve been getting in my terminal warning of a memory leak.

Screenshot 2020-08-19 at 18 45 48 Screenshot 2020-08-19 at 18 46 11

It points specifically to this file in our frontend repo, which has an unnamed default export. It’s one of the oldest files in our repo and definitely wasn’t causing issues before. Perhaps there’s been a change in the way Next.js handles these default unnamed exports that would cause the memory leak? This for example seems to be a new addition released within the last few weeks that was flagging that particular error, and warning of a memory leak.

I played around making some changes to a component I’ve been working on this week while the crashes have been most notable, and was able to trigger one again just now.

Here’s the output from my terminal:

wait  - compiling...
event - compiled successfully
event - build page: /editCollective
wait  - compiling...
event - compiled successfully
event - build page: /collective-page
wait  - compiling...
event - compiled successfully
wait  - compiling...

<--- Last few GCs --->

[15737:0x102925000]  5850165 ms: Mark-sweep 2052.8 (2068.5) -> 2051.9 (2068.5) MB, 419.6 / 0.2 ms  (average mu = 0.783, current mu = 0.000) last resort GC in old space requested
[15737:0x102925000]  5850866 ms: Mark-sweep 2051.9 (2068.5) -> 2051.5 (2067.8) MB, 701.2 / 0.1 ms  (average mu = 0.602, current mu = 0.000) last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2bc03c1408d1 <JSObject>
    0: builtin exit frame: byteLength(aka byteLengthUtf8)(this=0x2bc07bf6fd89 <Object map = 0x2bc0a06a4ba9>,0x2bc09252c761 <Very long string[34224816]>,0x2bc07bf6fd89 <Object map = 0x2bc0a06a4ba9>)

    1: fromString(aka fromString) [0x2bc07bf67d79] [buffer.js:~424] [pc=0x2784c0fe4651](this=0x2bc0bd7c04b1 <undefined>,0x2bc09252c761 <Very long string[34224816]>,0x2bc0eba27999 <String[#4]: utf8>)
...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: 0x100080c68 node::Abort() [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
 2: 0x100080dec node::errors::TryCatchScope::~TryCatchScope() [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
 3: 0x100185167 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
 4: 0x100185103 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
 5: 0x10030b2f5 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
 6: 0x1003130ed v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
 7: 0x1002e273c v8::internal::Factory::NewRawTwoByteString(int, v8::internal::AllocationType) [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
 8: 0x100533c81 v8::internal::String::SlowFlatten(v8::internal::Isolate*, v8::internal::Handle<v8::internal::ConsString>, v8::internal::AllocationType) [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
 9: 0x1001a42ad v8::String::Utf8Length(v8::Isolate*) const [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
10: 0x100064354 node::Buffer::(anonymous namespace)::ByteLengthUtf8(v8::FunctionCallbackInfo<v8::Value> const&) [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
11: 0x1001f08f0 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
12: 0x1001efecf v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
13: 0x1001ef5d0 v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
14: 0x100950a19 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/Users/katebeard/.nvm/versions/node/v12.16.1/bin/node]
15: 0x2784c0fe4651 
zsh: abort      npm run dev
~/coding/opencollective/opencollective-frontend  (git)-[master]- -> 

Node had been using around ~1.9 GB for a while and I’m not sure if it helped trigger it or if it was coincidence but when I switched branches and made a few more changes that seemed to push it over the edge of the available 2GB of memory.

i wonder if this PR will fix some of the issues we are seeing here: #24282 Can someone try that patch locally and see if that makes a difference?

Hey, it seems that going to Next 10.2.1 (including #24282) + using webpack5 has solved the issue in my case. The memory is now stable and in particular the (string) instances (source-map ?) in the heap snapshot that used to grow to several Gb are now stable and less than 300 Mb.

I’m using Next 10 (@latest) with Material-UI. After reading/skimming through this topic I incrementally raised the Node memory to 4, 6 and 8 GB. But even at 8 it will eventually run out of memory, although this is only after several days without restarting my PC and who normally codes that long in one go…?

Anyway, I do agree with @Dunky45 on this:

From my point of view trying to “fix” it with giving access to more RAM is just pointless, THAS IS JUST NO FIX AT ALL and you just keep increasing the cost of your app.

So I want to add a demo here to show this behavior. In this demo I artificially lowered the amount of memory for Node to 128MB, which is obviously unrealistically low. But at least it consistently shows the behavior we see at higher memory amounts. If you increase the amount of memory the crash won’t happen directly, unless you keep adding to and changing the style object in styles.js, but it takes some time (e.g. ~10 recompiles at 160MB).

@timneutkens Let me know if I missed or forgot to supply some information, as this is the first time ever I share a demo.

  1. Clone demo and install (https://github.com/joeplaa/nextjs-mui-out-of-memory)
  2. Run the application
  3. Click the link to the “about” page which uses JSS styles from Material-UI.
  4. Next dev server crashes:
$ yarn dev
yarn run v1.22.5
$ set NODE_OPTIONS=--max-old-space-size=128 && next
ready - started server on 0.0.0.0:3000, url: http://localhost:3000
info  - Using webpack 5. Reason: no next.config.js https://nextjs.org/docs/messages/webpack5
event - compiled successfully
event - build page: /next/dist/pages/_error
wait  - compiling...
event - compiled successfully
event - build page: /
wait  - compiling...
event - compiled successfully
event - build page: /about
wait  - compiling...

<--- Last few GCs --->

[16532:000001AF3D803260]    13863 ms: Mark-sweep 125.4 (134.4) -> 124.9 (129.7) MB, 14.5 / 0.0 ms  (+ 29.0 ms in 33 s
teps since start of marking, biggest step 4.8 ms, walltime since start of marking 53 ms) (average mu = 0.286, current
 mu = 0.268) finalize[16532:000001AF3D803260]    13921 ms: Mark-sweep 126.9 (129.7) -> 126.4 (130.7) MB, 21.1 / 0.0 m
s  (+ 25.9 ms in 29 steps since start of marking, biggest step 8.0 ms, walltime since start of marking 57 ms) (averag
e mu = 0.234, current mu = 0.178) allocati

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 00007FF7A266BB0D]
    1: StubFrame [pc: 00007FF7A266CA3D]
Security context: 0x03c64b1008d1 <JSObject>
    2: a [0000033AA58F86C9] [C:\Users\Joepje\bitbucket\nextjs-mui-out-of-memory\node_modules\next\dist\compiled\sourc
e-map\source-map.js:~1] [pc=0000010274E4AC10](this=0x03c582d82149 <JSGlobal Object>)
    3: /* anonymous */ [0000037485742711] [C:\Users\Joepje\bitbucket\nextjs-mui-out-of-memory\node_modules\next\dist.
..

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF7A1A3193F napi_wrap+119119
 2: 00007FF7A19D8A36 v8::internal::OrderedHashTable<v8::internal::OrderedHashSet,1>::NextTableOffset+38102
 3: 00007FF7A19D9836 node::OnFatalError+438
 4: 00007FF7A221694E v8::Isolate::ReportExternalAllocationLimitReached+94
 5: 00007FF7A21FEB01 v8::SharedArrayBuffer::Externalize+833
 6: 00007FF7A20B02DC v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1436
 7: 00007FF7A20BB520 v8::internal::Heap::ProtectUnprotectedMemoryChunks+1312
 8: 00007FF7A20B8034 v8::internal::Heap::PageFlagsAreConsistent+3204
 9: 00007FF7A20AD833 v8::internal::Heap::CollectGarbage+1283
10: 00007FF7A20ABEA4 v8::internal::Heap::AddRetainedMap+2500
11: 00007FF7A20CD1ED v8::internal::Factory::NewFillerObject+61
12: 00007FF7A1E307C1 v8::internal::interpreter::JumpTableTargetOffsets::iterator::operator=+1665
13: 00007FF7A266BB0D v8::internal::SetupIsolateDelegate::SetupHeap+546925
14: 00007FF7A266CA3D v8::internal::SetupIsolateDelegate::SetupHeap+550813
15: 0000010274E4AC10
error Command failed with exit code 134.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

It’s often not possible to send over reproductions, so you probably need to debug these memory problems on your own.

So here are some hints how to find them:

  1. Make sure you are using the latest version of all build-related dependencies: next, next plugins, webpack plugins, loaders, node.
  2. To find memory leaks you need a heap snapshot.
  • Node.js 15: Use NODE_OPTIONS=--heapsnapshot-near-heap-limit=1 to automatically create a heapsnapshot just before the process went OOM.
  • Other versions: node --inspect node_modules/next/dist/bin/next allows to attach Node.js devtools (chrome://inspect/edge://inspect, click on the Open dedicated DevTools for Node link)
    • The Memory tab is your fried. Here you can take a heapsnapshot. Click the GC button before doing that (trash can).
    • Make sure to run a few compilation to fill up all the cache and optimize code, etc.
  1. Set class filter to Compilation. Are there many Compilation instances? There should only be two or very few.
  2. If that doesn’t lead directly to the leak, take 3 heapsnapshots after each rebuild. Select the last one and select Objects allocated between Snapshot 1 and 2 right to the Class Filter. (Leave the class filter empty for this bullet point)
  • That are objects that are created during the 2 rebuild and are still alive after the 3 rebuild. Theoretically that should be none. In practice the javaascript engine keeps a lot stuff around for optimization or stuff is kept in caches.
  • Sort by Retained Size. Investigate into the biggest leaks.
  1. When expanding the class type you will see the instances. You can further expand them and look into their content.
  2. Select an Instance. On the bottom you will see the Retainers. That’s how these instance is referenced.
  3. You can expand paths, but it’s also auto-expanded. Make sure to only follow paths where Distance is shrinking. Do not look at paths where the Distance is increasing. That’s not relevant and will only confuse you.
  4. If you found something, you could post a screenshot here.
  5. If you path contains a WeakMap: Having the Devtools open will lead to leaking WeakMaps in some older node.js version. I think node.js <= 12.
  6. Look at multiple instances of the same type. Is there a pattern?
  7. During watch mode next/webpack will hold on the last Compilation. It will also keep all modules in cache. That’s expected.

fwiw posting memory stacks is not going to solve this issue, we need full repositories including all customizations you’ve done in order to be able to even determine if this is caused by Next.js in your particular application or if it’s caused by any of your other dependencies (e.g. you might have customized the webpack configuration). If you’re not able to provide these please just use GitHub reactions on the initial issue (add a 👍) as the memory stacktrace is not helpful in tracking down what the issue is for your application.

Note that memory leaks can be introduced by your own code (e.g. if you render nested components in an infinite loop, if you create an object in the module scope and keep adding to it etc.) and by dependencies (e.g. React components from npm, libraries from npm etc.). Hence why I keep asking for full reproductions, as it’s practically impossible to track these down for your particular case otherwise.

Also note that I want to solve this if it is related to Next.js, we just need more pointers from applications that experience this. We run a Next.js application with hundreds of pages and thousands of components with many developers and haven’t experienced OOMs both during dev/build. The cases that we have investigated before mostly lead back to a certain dependency being imported that had a memory leak, this is also the reason why I brought up enterprise support (feel free to email enterprise@vercel.com) as a means of investigating this particular issue in your application, given I can’t justify investigating days/weeks on an issue with no reproduction provided to date and that hasn’t been reproducible to start investigating.

I hope you all understand and I’m looking forward to seeing a reproduction provided that can be investigated.

I can also confirm that this is happening. It seems like it may be related to page navigation somehow, as, when trying to come up with reproducible steps, we noticed that if we stayed on the same page and tried a series of change+save loops to get the hot reloading to fire quickly, it wouldn’t crash. But, if we tried navigating through our app for a bit, and then doing a few change+save loops we would quickly get to a point where it crashed.

A few properties of the codebase I’m working on:

  • We use styled-components (currently at v5.1.0) and we have a babel.config.js file that we use so we can properly SSR the styles (mentioning this because of what was reported https://github.com/vercel/next.js/issues/7929#issuecomment-707222434)
  • We use Typescript in the codebase (currently at v4.0.3)
  • We are currently on NextJS v9.5.5 (upgraded recently from v9.3.6)
  • We have a number of circular dependencies in our codebase
  • There are a lot of components in our codebase, ranging from ones we created ourselves to ones we import via component libraries that were created in house

As far as I know, no one on our team had this issue on NextJS v9.3.6.

I’ve been getting this error with 9.5 as my site has grown in size. It happens whenever I fast refresh and have been in dev mode for over 15 minutes. I could share my repo privately if it’s helpful @timneutkens

Update, fixed

I refactored my entire app to make sure that there was absolutely no circular importing. I also changed my package.json script to run with this:

{
  "scripts": {
    "start": "NODE_OPTIONS=--max_old_space_size=8192 next"
  }
}

Now it’s working just fine. Not sure what did it exactly, but I have a feeling it was a circular imports. So I recommend making sure you don’t have circular imports in your app.

For context, I’m on next@9.5.3. I’m also using Expo + Next.js, and I upgraded to Expo SDK39 from 38 and react-native-web 0.14. Unsure if that had anything to do with it or not. 🤷🏼‍♂️

it is getting pretty annoying

react@17.0.1 next@10.0.9 emotion@11.1.5 ant.design@4.15.0 framer-motion@4.0.0 reduxjs/toolkit@1.5.0 prisma@2.20.1


<--- Last few GCs --->

[29520:00000226913763F0]  3383394 ms: Mark-sweep 1375.7 (1423.2) -> 1374.8 (1422.2) MB, 525.8 / 0.0 ms  (average mu = 0.101, current mu = 0.043) allocation failure scavenge might not succeed
[29520:00000226913763F0]  3383402 ms: Scavenge 1375.9 (1422.7) -> 1375.0 (1423.2) MB, 4.7 / 0.0 ms  (average mu = 0.101, current mu = 0.043) allocation failure


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 000002E9189DC5C1]
Security context: 0x00c8f769e6e9 <JSObject>
    1: /* anonymous */ [000000A24938EA69] [B:\_PROJEKTY\_PRODUCTS\_REACT-CMS\node_modules\next\dist\compiled\source-map\source-map.js:~1] [pc=000002E91AF4A219](this=0x0200df80d481 <JSGlobal Object>,e=0x0207e7a4c029 <String[54]\:  context The context to use when calling the function\n>,o=0x0200bae15d79 <Object map = 0000004AA3635B59>)
    2: SourceNode_walk...

 1: 00007FF707A17DDA v8::internal::GCIdleTimeHandler::GCIdleTimeHandler+4506
 2: 00007FF7079F2876 node::MakeCallback+4534
 3: 00007FF7079F31F0 node_module_register+2032
 4: 00007FF707D0B6BE v8::internal::FatalProcessOutOfMemory+846
 5: 00007FF707D0B5EF v8::internal::FatalProcessOutOfMemory+639
 6: 00007FF707EF1DE4 v8::internal::Heap::MaxHeapGrowingFactor+9620
 7: 00007FF707EE8DC6 v8::internal::ScavengeJob::operator=+24550
 8: 00007FF707EE741C v8::internal::ScavengeJob::operator=+17980
 9: 00007FF707EF0167 v8::internal::Heap::MaxHeapGrowingFactor+2327
10: 00007FF707EF01E6 v8::internal::Heap::MaxHeapGrowingFactor+2454
11: 00007FF70801A487 v8::internal::Factory::NewFillerObject+55
12: 00007FF708097676 v8::internal::operator<<+73494
13: 000002E9189DC5C1
error Command failed with exit code 134.

Only thing that is annoying is here are these 0 value comments. Stop the spamming, please!

(Sorry for bumping the thread myself)

I no longer are affected by this problem, but its hard to tell what ultimatly fixed it. We are now on nextjs 11, but already in some version of nextjs 10, the issue was resolved

I observed that once the memory crash in development did no longer ocure, i had instead some fash refresh hicups, where nextjs would force reload the whole page. I cannot tell whether this is related though.

Its important to state that this issue here was never about memory leaks in production. I think this is a completly different problem that might have different reasons.

for those who are affected by memory leaks in production: open up a different issue, please!

Despite the activity here I would suggest to close this issue here. Reason is that a lot has changed since version 9.5.1. and information might be missleading here. Also i am no longer affected, so the initial issue is resolve for me and the information is no longer vaid.

  • if you are still affected by memory leaks / crashses in development: open up a new issue. You can still reference this issue here.
  • if you are affected by memory leaks / crashes in production: also open up a separate issue.

Here’s what (I believe) solved this for me.

1. Circular imports

yarn add -D madge

yarn madge --circular packages --extensions ts,tsx,js,jsx

If any circular imports come up, fix them. Make sure that you isolate variables into their own files if needed.

2. Re-exports

My other issue was I was doing this in packages

Example:

packages/api/index.ts

// 🚨 I stopped doing this
export * from './artist'

// ✅ I do this instead
export {}

Instead, I now import directly:

import { createArtist } from '@beatgig/api/artist'

I’m using a (massive) monorepo and next-transpile-modules, and 10.1.4-canary.2.

3. Memory leak listeners

My final problem was that I set up a listener in pages/_app.js outside of render code. I was listening to Router.events. I fixed this by moving this to useEffect.

Be very cautious to not include any listeners outside of an effect. And make sure you clean them up.

This is not an issue with Next.js, rather an issue with how Tailwind handles class compilation.

To solve this issue, use the new @tailwindcss/jit just in time compilation module. You can install this via. NPM or Yarn,

// NPM
npm i @tailwindcss/jit

// Yarn
yarn add @tailwindcss/jit

and then by adding this to your postcss.config.js file:

module.exports = {
  plugins: {
    '@tailwindcss/jit': {},
    autoprefixer: {}
  }
};

@timneutkens, this issue can be marked as resolved (closed).

No, multiple people in the thread (including me) are not using Tailwind.

We’re also seeing the exact same out of memory crashes both in local development and in production.

Our cluster of Next.js containers keeps kicking over randomly under load with these out of memory errors.

Typescript/Styled-Components + custom server. No circular dependencies in our codebase.

I’m starting to consider adding something like nodemon to restart the process faster than our container orchestrator, but that’s a bit of a hack, no?

So, https://github.com/vercel/next.js/issues/12557 is probably relevant to this issue.

[…] does anybody by any chance know if there’s an automated way to do this […]

I ended up building a codemod for our codebase using ts-morph to refactor away all of our barrels. The end result was an improvement to our jest runtimes (improved by 200s), our NextJS build times (20-30s improvements), a reduction in the number of circular dependencies (269 to 127), improved bundle chunking and an overall reduction in First Load JS (about 1MB to 840KB or so) for most pages controlled by our codebase.

I’ll try checking if I can release the codemod to the general public.

I would recommend getting rid of barrel files if all of the consumers of your code are within the same codebase. If there are consumers of your code outside of your codebase, and you really want those barrel files, have something to generate the barrels as a build artifact to help you avoid the creation of unintended circular dependencies, among other issues, while also providing a nice public api for your code.

Just want to add my experience with this and spill some of my toughts. As the others said here. I’m also experiencing this issue in dev since upgrading to 9.5 and in other project to latest 10.x. . It often crashes during some CSS work or just when we navigate throughout the project (maybe some hot-reload cache accumulating). On 9.5 I use SASS with next-sass lib and on 10.x styled components. I just keep my fingers crossed it won’t happen in production 😄 , so far it hasn’t but you never know.

To be honest I don’t feel confident that the leaks can’t be somewhere in my code. I tried debugging snapshots but it’s just too much information for me and I would say that orienting in this kind of deep debugging is for real senior hot-shots. (I’ve been working with react since 2017 and can’t rly understand much).

From my point of view trying to “fix” it with giving access to more RAM is just pointless, THAS IS JUST NO FIX AT ALL and you just keep increasing the cost of your app.

I totally understand that guys from Vercel are in tough spot here, since helping with issue that you hardly can reproduce is almost impossible. But we can all agree that something has changed guys, too many devs have this problem and it shouldn’t be overlooked.

Btw. Thanks for your work nevertheless, best framework out here still! 👍

This is not an issue with Next.js, rather an issue with how Tailwind handles class compilation.

To solve this issue, use the new @tailwindcss/jit just in time compilation module. You can install this via. NPM or Yarn,

// NPM
npm i @tailwindcss/jit

// Yarn
yarn add @tailwindcss/jit

and then by adding this to your postcss.config.js file:

module.exports = {
  plugins: {
    '@tailwindcss/jit': {},
    autoprefixer: {}
  }
};

@timneutkens, this issue can be marked as resolved (closed).

I would be willing to share our apps private source code with a Next.js core-member to help get to the bottom of this.

@mankybansal There are plenty of articles out there that tell you how to use Node debugger’s memory snapshots to track down memory leaks.

I just tried building Next.js locally and linking into our app as described here: https://github.com/vercel/next.js/blob/canary/contributing.md

I wanted to see if there was any truth to the idea that Next.js is using more memory in 9.5 vs 9.4, as per the OP:

it did rarely happen in 9.4.1, but it happens very consistantly in 9.5.x

However, I just get this error, and the server never seems to start:

(node:1926) UnhandledPromiseRejectionWarning: TypeError: (0 , _middleware.default) is not a function
    at HotReloader.assignBuildTools (/directory_of/next.js/packages/next/dist/server/hot-reloader.js:16:126)
    at HotReloader.start (/directory_of/next.js/packages/next/dist/server/hot-reloader.js:15:2084)
    at async DevServer.prepare (/directory_of/next.js/packages/next/dist/server/next-dev-server.js:10:1565)

Since that error doesn’t reference any of our own code, I don’t know where to start debugging, so I’m giving up for now.

Looking at memory use in htop though, I don’t see much difference, so I’m not convinced there’s been a big change in Next.js here. It’s possible that those of us who have started seeing this issue more often have simply increased the size of our apps: more dependencies, more pages, etc.

Between 9.4 and 9.5, there were changes in how Next.js handled Webpack hot middleware + some other fast refresh stuff so that’s probably where your error is coming from. I’ve spent countless hours tracking down changes between 9.4 and 9.5 both in what changed in Webpack, and what changed in Next.js, and sadly I just don’t think I know enough about bundlers/hash chunks (chunk hashes?? it’s been a few weeks since I looked 😅 )/Source maps and source nodes/all this stuff I’m encountering to really make sense of it.

But it does feel like it’s to do with Webpack + Next.js not properly cleaning up old chunk hashes/sources and essentially leaving these orphan chunks to accumulate in memory. All the error messages and heap snapshots we’ve posted seem to point to that (anything to do with emit or EventEmitters generally comes from Webpack code that comes from watchpack/DirectoryWatcher/etc spawning a new watcher for a chunk/source that might not have gotten garbage collected, so it stays open… or something like that).

What I find particularly intriguing though is that - for Open Collective at least - this memory leak issue is not present in prod, only on the Next.js dev server, which is where I’ve focused my investigations. Maybe I’ll try to condense all the notes I’ve taken and post them and hopefully someone with more knowledge about bundler internals can use that as a springboard.

We managed to solve (at least I think so) this on Next 10.0.5 by removing tailwind from our project. Removing it reduced our heap size to around 150Mb from 2Gb I need to keep an eye out to make sure that the heap size doesn’t keep increasing until I get a crash, but so far it’s looking good.

Thanks for the tip, that’s a useful data point.

We aren’t running tailwind, but I have a suspicion that there are other dependencies which will have a similar effect.

I find this interesting because I felt like the times I was having the server crash the most in development is when I was working on CSS (OC doesn’t use Tailwind, but does use the styled- packages) rather than other types of frontend work. Perhaps something to do with the way Webpack hashes CSS and how that interacts with the Next.js updates? 🤔

I’ve been getting this issue consistently after 1-2 hot reloads after upgrading to 9.5. It makes the dev experience slow (constantly restarting) and barely usable. The OOM occurred on 9.4 but much less frequently (once an hour) Has anyone been able to improve / resolve this issue?

@timneutkens it looks like we have a few repros, I can also throw mine into the mix but would need to be under an NDA.

@thereactguys deleted your spam.

I’ve noticed this issue happen 4 -5 times in the past month. It has happened when switching branches, but it also just happened when I updated my JSX to have an inline style. I tend to run yarn dev for multiple days on end.

wait  - compiling...
event - compiled successfully
wait  - compiling...
event - compiled successfully
wait  - compiling...

<--- Last few GCs --->
nce start of marking 305 ms) (average mu = 0.778, current mu = 0.186) allocatio[20536:0x104091000] 57744365 ms: Mark-sweep 2083.8 (2090.9) -> 2083.8 (2090.9) MB, 346.4 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 394 ms) (average mu = 0.651, current mu = 0.121) last reso[20536:0x104091000] 57744720 ms: Mark-sweep 2083.8 (2090.9) -> 2083.7 (2090.9) MB, 354.6 / 0.0 ms  (average mu = 0.483, current mu = 0.000) last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x1007538f9]
    1: StubFrame [pc: 0x1007964e0]
Security context: 0x3eb61e880921 <JSObject>
    2: replace [0x3eb61e88cdd1](this=0x3eb6896a1cc9 <Very long string[29761]>,0x3eb61dbdef61 <JSRegExp <String[#10]: \n(?=.|\s)>>,0x3eb6c6363451 <String[9]\: \n/******/>)
    3: source [0x3eb61dbdfaf1] [/Users/helson/Development/polyops/ui/node_modules/webpack-sources/lib/PrefixSource.js:43] [bytecode=0x3eb6c634d381 offset=60]...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: 0x100080a4c node::Abort() [/usr/local/Cellar/node/13.13.0_2/bin/node]
 2: 0x100080bc2 node::OnFatalError(char const*, char const*) [/usr/local/Cellar/node/13.13.0_2/bin/node]
 3: 0x100180be5 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node/13.13.0_2/bin/node]
 4: 0x100180b8f v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node/13.13.0_2/bin/node]
 5: 0x10029a5c3 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/Cellar/node/13.13.0_2/bin/node]
 6: 0x10029facc v8::internal::Heap::SetUp() [/usr/local/Cellar/node/13.13.0_2/bin/node]
 7: 0x10027ca79 v8::internal::Factory::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Map, v8::internal::AllocationAlignment) [/usr/local/Cellar/node/13.13.0_2/bin/node]
 8: 0x10027ee6c v8::internal::Factory::NewRawOneByteString(int, v8::internal::AllocationType) [/usr/local/Cellar/node/13.13.0_2/bin/node]
 9: 0x100441a9f v8::internal::String::SlowFlatten(v8::internal::Isolate*, v8::internal::Handle<v8::internal::ConsString>, v8::internal::AllocationType) [/usr/local/Cellar/node/13.13.0_2/bin/node]
10: 0x1004bb6d9 v8::internal::RegExpImpl::IrregexpExec(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSRegExp>, v8::internal::Handle<v8::internal::String>, int, v8::internal::Handle<v8::internal::RegExpMatchInfo>) [/usr/local/Cellar/node/13.13.0_2/bin/node]
11: 0x100504307 v8::internal::Runtime_RegExpExec(int, unsigned long*, v8::internal::Isolate*) [/usr/local/Cellar/node/13.13.0_2/bin/node]
12: 0x1007538f9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/usr/local/Cellar/node/13.13.0_2/bin/node]
13: 0x1007964e0 Builtins_RegExpReplace [/usr/local/Cellar/node/13.13.0_2/bin/node]
14: 0x100740ae2 Builtins_StringPrototypeReplace [/usr/local/Cellar/node/13.13.0_2/bin/node]
error Command failed with signal "SIGABRT".

Agreed. We’re not using tailwind however is still having this issue. We’re using styled-components and I’m wondering what are the percentage of people having this issue on 9.5 that are using at least one of the css libraries (tailwind, styled-components, etc)

We are using Next with Material-UI CSS-in-JS.

Next 10.0.7 Circular dependency detected: node_modules/next/dist/client/with-router.js -> node_modules/next/dist/client/router.js -> node_modules/next/dist/client/with-router.js

I’m not seeing any issue with memory usage either. What I am seeing is slowly ballooning CPU usage until the process becomes completely unresponsive. If I start a load test at a steady number of concurrent connections, the connections count on my load balancer report as steady throughout the duration of the test, memory usage stays normal and does not show any GC spikes or leaks, but CPU constantly climbs until all hosts completely lock up.

This issue is to do with memory use while hot-reloading is enabled. If neither of those are relevant to your situation (I assume you aren’t load testing with hot-reloading enabled…?), maybe you should create a new issue. Have I misunderstood what you’re saying?

@sambecker Interesting - we honestly started seeing this after upgrading from Node 10 to 13. Upgrading to v14 is on the todo list and I’m really crossing my fingers that the issue just goes away when we do.

FWIW @timneutkens I appreciate your patience with this and the fact that you’re not closings this outright. Getting bug reports like this must be frustrating, and I’m sure I’m not the only one when I say I’d love to share our source code with you but are legally unable to. I’m circling up with my team about potentially reaching out for enterprise support, but even that is less than ideal as the red tape to get that NDA signed will probably take us months.

My team is doing a lot of investigation into this right now and I’ll be sure to share anything we find. The only smoking gun we’ve found is that a lot of the out of memory crash stacktraces point back to the etag header generation code that hashes the response before sending it. I don’t think the actual issue is that code, but it seems to be indicative of Node poorly managing memory. If anything, this is quite possibly a Node bug.

I don’t know if this is useful, but I found that some React components were loading the Link element with bad urls. The errors weren’t really exposed until I removed most of my pages from generation. There were some other issues but this was a big one. It might be good to have Next.js report more of it’s lifecycle data so it’s easier to pinpoint where things are happening, or some way to die on errors so they can be located.

Anyways, seems one thing I learned was to cut down your generated pages to as few as possible and add them back in until you see errors so you see them.

Update 1: We have a big json file that’s a mapping of zip codes to coordinates. This was require d in one of our API files. I changed it to a .js file and imported it import { zipMap } from 'lib/zipToCoordinates.js' and that resolved one more OOM error that had zero context.

I have improved this situation a lot (as in I need to test more to consider it fixed, but it stopped crashing) by replacing tailwindcss with @tailwindcss/jit, I guess it is leaking memory in CSS processing and Tailwind does pull in a huge amount of CSS. If that’s the case, it’s probably not a Next.js problem but in some dependency.

May not be a global solution since people here didn’t report using Tailwind

We experienced the same problem: started after upgrading to 9.5 and never stopped.

We don’t use a custom server (we have no API code in the Next project, it’s just frontend) and we also make no use of Tailwind, so whatever the problem is Tailwind is not the cause.

Our library stack is Typescript, Material-UI and heavy CSS module usage. However, the problem started before we adopted typescript.

It got to the point where the server restarts every few minutes and regularly uses 4GB of RAM.

Maybe Material-UI is also affected? Unfortunately I can’t open up the codebase for this project.

@mmmeff Upgraded to Node 14 last week and we’re still experiencing the same problem so I wouldn’t count on that making a difference, unfortunately.

In our case it started happening when upgrading from Next 9.4.2 to 9.5.2 (without touching the Node version) so I’m not sure if it makes sense that it’s a Node bug either.

I seem to get this crash after I change some CSS in the same file as tailwind imports, I am also using typescript. I am currently trying to create a test repo. Repo based on blog-starter-typescript but slightly modified https://github.com/lclc98/NextOOM

I’ve been having the same issue for about 1-2 weeks as well. After starting up my local dev environment, I work for a bit and then end up having to restart due to a crash. I also get these report.*.json files put into my root directory. This has happened most days.

System information

  • OS: MacOS Catalina 10.15.6
  • Browser: Firefox 79
  • Version of Next.js: 9.5.1 (upgraded 15 days ago) & continuing in 9.5.2 (upgraded 8 days ago)
  • Version of Node.js: 12.16.1 (with NVM)

(I can provide the full Node-generated report JSON if needed).

A coworker and I also noticed we were getting this error too:

TypeError: Cannot destructure property 'components' of 'object null' as it is null.
    at DevServer.renderToHTMLWithComponents (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:60:246)
    at DevServer.renderErrorToHTML (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:101:332)
    at async DevServer.renderErrorToHTML (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/server/next-dev-server.js:30:1204)
    at async DevServer.renderError (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:100:1173)
TypeError: Cannot destructure property 'components' of 'object null' as it is null.
    at DevServer.renderToHTMLWithComponents (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:60:246)
    at DevServer.renderErrorToHTML (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:101:332)
    at async DevServer.renderErrorToHTML (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/server/next-dev-server.js:30:1204)
    at async DevServer.renderToHTML (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:100:974)
    at async DevServer.renderToHTML (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/server/next-dev-server.js:30:578)
    at async DevServer.render (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:55:236)
TypeError: Cannot destructure property 'components' of 'object null' as it is null.
    at DevServer.renderToHTMLWithComponents (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:60:246)
    at DevServer.renderErrorToHTML (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:101:332)
    at async DevServer.renderErrorToHTML (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/server/next-dev-server.js:30:1204)
    at async DevServer.renderToHTML (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:100:974)
    at async DevServer.renderToHTML (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/server/next-dev-server.js:30:578)
    at async DevServer.render (/Users/katebeard/coding/opencollective/opencollective-frontend/node_modules/next/dist/next-server/server/next-server.js:55:236)

Please provide a reproduction so that we can investigate

When you say this do you mean a link to a PR with a live app on Vercel? We might be able to provide since Open Collective is open source and all our repos are open. (Edited to add: here’s the PR I’ve been working on where I experienced the latest crashes, you can pull the code from our repo and see the Vercel app link in the PR.)

I think this is a critical issue that strongly hinder adoption of Next framework in production environments.

I am a big fan of the framework, so please have a DEEP look into it, after almost 1 year without a fix this deserve a focused investigation and a definitive fix,

It is probably worth checking if you have fast refresh working properly.

https://nextjs.org/docs/basic-features/fast-refresh

We had a huge index file for our @ecologi/react-components package

After changing the package.json main field in our react components package to the following…

Screenshot 2021-04-20 at 14 50 31

Which was needed to get Next Transpile Modules to pick up changes in each component folder properly.

And also updating all the imports to have imports for each component.

Screenshot 2021-04-20 at 14 51 56

We now have fast refresh for all the components which makes it possible to develop on our apps using yarn dev

However there is still issues navigating around the site. After compiling more than 3 or 4 different pages in dev mode it becomes unbearably slow and will crash or need restarting.

The issue seems occur when there is any refresh of the browser window or navigation to another NextJS page in yarn dev mode.

If you don’t have fast refresh enabled it will reload the browser window for most changes and this will cause it to crash all the time.

it is getting pretty annoying

react@17.0.1 next@10.0.9 emotion@11.1.5
ant.design@4.15.0 framer-motion@4.0.0 reduxjs/toolkit@1.5.0 prisma@2.20.1


<--- Last few GCs --->

[29520:00000226913763F0]  3383394 ms: Mark-sweep 1375.7 (1423.2) -> 1374.8 (1422.2) MB, 525.8 / 0.0 ms  (average mu = 0.101, current mu = 0.043) allocation failure scavenge might not succeed
[29520:00000226913763F0]  3383402 ms: Scavenge 1375.9 (1422.7) -> 1375.0 (1423.2) MB, 4.7 / 0.0 ms  (average mu = 0.101, current mu = 0.043) allocation failure


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 000002E9189DC5C1]
Security context: 0x00c8f769e6e9 <JSObject>
    1: /* anonymous */ [000000A24938EA69] [B:\_PROJEKTY\_PRODUCTS\_REACT-CMS\node_modules\next\dist\compiled\source-map\source-map.js:~1] [pc=000002E91AF4A219](this=0x0200df80d481 <JSGlobal Object>,e=0x0207e7a4c029 <String[54]\:  context The context to use when calling the function\n>,o=0x0200bae15d79 <Object map = 0000004AA3635B59>)
    2: SourceNode_walk...

 1: 00007FF707A17DDA v8::internal::GCIdleTimeHandler::GCIdleTimeHandler+4506
 2: 00007FF7079F2876 node::MakeCallback+4534
 3: 00007FF7079F31F0 node_module_register+2032
 4: 00007FF707D0B6BE v8::internal::FatalProcessOutOfMemory+846
 5: 00007FF707D0B5EF v8::internal::FatalProcessOutOfMemory+639
 6: 00007FF707EF1DE4 v8::internal::Heap::MaxHeapGrowingFactor+9620
 7: 00007FF707EE8DC6 v8::internal::ScavengeJob::operator=+24550
 8: 00007FF707EE741C v8::internal::ScavengeJob::operator=+17980
 9: 00007FF707EF0167 v8::internal::Heap::MaxHeapGrowingFactor+2327
10: 00007FF707EF01E6 v8::internal::Heap::MaxHeapGrowingFactor+2454
11: 00007FF70801A487 v8::internal::Factory::NewFillerObject+55
12: 00007FF708097676 v8::internal::operator<<+73494
13: 000002E9189DC5C1
error Command failed with exit code 134.

I am still encountering this error in 10.1.x.

Log:

@courselit/app: event - compiled successfully
@courselit/app: wait  - compiling...
@courselit/app: event - compiled successfully
@courselit/app: wait  - compiling...
@courselit/app: event - compiled successfully
@courselit/app: wait  - compiling...
@courselit/app: <--- Last few GCs --->
@courselit/app: [33530:0x575bc80] 11140174 ms: Mark-sweep 4008.3 (4136.6) -> 3985.7 (4130.3) MB, 1966.6 / 0.2 ms  (average mu = 0.194, current mu = 0.104) task scavenge might not succeed
@courselit/app: [33530:0x575bc80] 11143486 ms: Mark-sweep 4000.9 (4131.9) -> 3990.9 (4132.2) MB, 3222.1 / 0.3 ms  (average mu = 0.102, current mu = 0.027) task scavenge might not succeed
@courselit/app: <--- JS stacktrace --->
@courselit/app: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
@courselit/app:  1: 0xa877f0 node::Abort() [/usr/local/bin/node]
@courselit/app:  2: 0x9abe29 node::FatalError(char const*, char const*) [/usr/local/bin/node]
@courselit/app:  3: 0xc6ea6e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
@courselit/app:  4: 0xc6ede7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
@courselit/app:  5: 0xe38865  [/usr/local/bin/node]
@courselit/app:  6: 0xe3940c  [/usr/local/bin/node]
@courselit/app:  7: 0xe46d9b v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
@courselit/app:  8: 0xeb01bc v8::internal::ScavengeJob::Task::RunInternal() [/usr/local/bin/node]
@courselit/app:  9: 0xd6573b non-virtual thunk to v8::internal::CancelableTask::Run() [/usr/local/bin/node]
@courselit/app: 10: 0xaf7994 node::PerIsolatePlatformData::RunForegroundTask(std::unique_ptr<v8::Task, std::default_delete<v8::Task> >) [/usr/local/bin/node]
@courselit/app: 11: 0xaf97f9 node::PerIsolatePlatformData::FlushForegroundTasksInternal() [/usr/local/bin/node]
@courselit/app: 12: 0x14789e6  [/usr/local/bin/node]
@courselit/app: 13: 0x148b775  [/usr/local/bin/node]
@courselit/app: 14: 0x1479318 uv_run [/usr/local/bin/node]
@courselit/app: 15: 0x9d2025 node::SpinEventLoop(node::Environment*) [/usr/local/bin/node]
@courselit/app: 16: 0xac8b90 node::NodeMainInstance::Run(node::EnvSerializeInfo const*) [/usr/local/bin/node]
@courselit/app: 17: 0xa4f73a node::Start(int, char**) [/usr/local/bin/node]
@courselit/app: 18: 0x7f6089dcf2e1 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
@courselit/app: 19: 0x9ce26c  [/usr/local/bin/node]
@courselit/app: Aborted (core dumped)
@courselit/app: error Command failed with exit code 134.
@courselit/app: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
lerna ERR! yarn run dev exited 134 in '@courselit/app'
error Command failed with exit code 134.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

This is not an issue with Next.js, rather an issue with how Tailwind handles class compilation. To solve this issue, use the new @tailwindcss/jit just in time compilation module. You can install this via. NPM or Yarn,

// NPM
npm i @tailwindcss/jit

// Yarn
yarn add @tailwindcss/jit

and then by adding this to your postcss.config.js file:

module.exports = {
  plugins: {
    '@tailwindcss/jit': {},
    autoprefixer: {}
  }
};

@timneutkens, this issue can be marked as resolved (closed).

No, multiple people in the thread (including me) are not using Tailwind.

My mistake. Nonetheless, the issue here isn’t in of itself NextJS. Alone, the framework is completely functional.

@ranisalt I experience the memory leak with the tw v2 only, works just fine with the v1.

We are also having this problem.

Next version: next@9.5.5

Reading this issue has been somewhat frustrating. I understand that it is difficult to diagnose these issues without a reproduction, but consider that most of us simply won’t be able to hand over our closed source repos. If it was easy for us to create a minimal reproduction, I’m sure someone would have done so, but this also is not easy.

I also understand that since this is likely due to some third party, it must be frustrating being lumped with the problem. I’m not really sure what else we can do in this case except work together to find the culprit.

Other than requests for reproductions, it hasn’t been stated what we can do to help you! I’m sure many of us would gladly give more information if we knew what you might find useful.

Here are some suggestions:

  • lists of dependencies and versions, taken from package.json - this would help narrow down possible dependency culprits
  • heap snapshots, potentially some redacted version or overview - I’m not experienced with what’s possible here, tips welcome
  • EDIT 2: someone could bisect the known working/broken versions to help identify the commit where this got worse?
  • EDIT 3: information about number of pages and built page sizes

One piece of information I haven’t seen here (I might have missed it) is that looking at heap snapshots makes me think that it is related to this problem in some way: https://nextjs.org/blog/webpack-memory

While the changes noted in that post seem to have fixed the memory problems when building, I see similar symptoms: huge amounts of memory attributed to CachedSource. Now, I’m a complete novice at reading heap snapshots, so take that with some scepticism!

I tried setting output.futureEmitAssets in our config, but it had no effect. I don’t understand what this does enough to predict what should happen though.

I’m going to echo what I think I’ve seen others say, which is that I’m happy to talk privately and show you what’s happening with our setup. I understand this might not be ideal for you, but it’s probably the best compromise I can offer.

EDIT: I can also offer some small bits of analysis about memory and file sizes.

Looking at the heap snapshot, there seems to be 47MB of memory taken up by CachedSource (from the webpack-sources module) for static/chunks/pages/_app.js alone. 35MB for static/chunks/pages/[region]/places/[id].js, and decreasing amounts for other pages. Again, my interpretation of the snapshot information might be off, but that’s how it appears to me.

The size of the files on disk in the built server/pages directory seems to be around 900kB each, with _app.js being 1.7MB.

I don’t know what numbers are expected here or even if they’re helpful, but there they are!

@TheReactGuys please stop spamming. I’ve reported the account to GitHub.

@malimccalla I also have a similar setup to you, and have been experiencing the same Out of Memory errors as have been previously described. I’ll see if removing these from my app mitigates the errors in any way. Thanks for the lead!

Tried with next 11.1.2 and node 16.9.0 and have exactly the same problem:

<--- Last few GCs --->

[57:0x7f1dcae51310]   739770 ms: Mark-sweep (reduce) 4073.9 (4143.6) -> 4072.7 (4143.6) MB, 4612.8 / 0.0 ms  (average mu = 0.120, current mu = 0.001) allocation failure scavenge might not succeed
[57:0x7f1dcae51310]   744338 ms: Mark-sweep (reduce) 4074.1 (4143.9) -> 4073.1 (4143.9) MB, 4565.2 / 0.0 ms  (average mu = 0.064, current mu = 0.001) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

This issue is still actual!

I bet you tried to find something before you started this endeavour, but does anybody by any chance know if there’s an automated way to do this (using VSCode)?

I did, but i was worried that if i would let some script do the works I would end up with many errors and loose my head around. By doing it myself I was sure every import is correct, of course human error is a biggie here. But thankfully I am kinda good at this “copy-paste-robot like” things 😆. Anyway, there must be for sure a way to script this process quite easily.

Also like others said, thanks to this there are improvements in build size, build speed and jest runtimes.

Just want to add my experience with this and spill some of my toughts. As the others said here. I’m also experiencing this issue in dev since upgrading to 9.5 and in other project to latest 10.x. . It often crashes during some CSS work or just when we navigate throughout the project (maybe some hot-reload cache accumulating). On 9.5 I use SASS with next-sass lib and on 10.x styled components. I just keep my fingers crossed it won’t happen in production 😄 , so far it hasn’t but you never know.

To be honest I don’t feel confident that the leaks can’t be somewhere in my code. I tried debugging snapshots but it’s just too much information for me and I would say that orienting in this kind of deep debugging is for real senior hot-shots. (I’ve been working with react since 2017 and can’t rly understand much).

From my point of view trying to “fix” it with giving access to more RAM is just pointless, THAS IS JUST NO FIX AT ALL and you just keep increasing the cost of your app.

I totally understand that guys from Vercel are in tough spot here, since helping with issue that you hardly can reproduce is almost impossible. But we can all agree that something has changed guys, too many devs have this problem and it shouldn’t be overlooked.

Btw. Thanks for your work nevertheless, best framework out here still! 👍

Ok so for me the issue was it seem this:

I have folder named components, where i have a lot of small components that app re-uses. In that folder i had index.js file, which exported all those components, so i could import them in pages/views like this:

import {ComponentA, ComponentB, ComponentC} from './components'

I removed index.js and changed all the imports, to direct imports e.g. :

import ComponentA from './components/ComponentA'

After i did this in whole project (maaan was this a painfull job for many many hours 🤮 😭 ) it seems that the problem with memory leak no longer occurs (haven’t got a single error in a month).

Hope that someone has the problem as I did and it will help him 😃. Happy coding!

I exactly had the same problems, after fixing all of them my projects works just fine again, many thanks!

Here’s what (I believe) solved this for me.

1. Circular imports

yarn add -D madge

yarn madge --circular packages --extensions ts,tsx,js,jsx

If any circular imports come up, fix them. Make sure that you isolate variables into their own files if needed.

2. Re-exports

My other issue was I was doing this in packages

Example:

packages/api/index.ts

// 🚨 I stopped doing this
export * from './artist'

// ✅ I do this instead
export {}

Instead, I now import directly:

import { createArtist } from '@beatgig/api/artist'

I’m using a (massive) monorepo and next-transpile-modules, and 10.1.4-canary.2.

3. Memory leak listeners

My final problem was that I set up a listener in pages/_app.js outside of render code. I was listening to Router.events. I fixed this by moving this to useEffect.

Be very cautious to not include any listeners outside of an effect. And make sure you clean them up.

Only thing that is annoying is here are these 0 value comments. Stop the spamming, please!

Why is it spamming? He provided a stack trace (different than the others, mind you) and the list of packages and versions involved. If you don’t want to get notified, ignore the issue or fix it.

Those tracebacks don’t help to solve the issue. Vercel guys have said pretty clearly that don’t help: https://github.com/vercel/next.js/issues/15855#issuecomment-760187734 and that they need reproducible examples: https://github.com/vercel/next.js/issues/15855#issuecomment-668478456

Only thing that is annoying is here are these 0 value comments. Stop the spamming, please!

Why is it spamming? He provided a stack trace (different than the others, mind you) and the list of packages and versions involved. If you don’t want to get notified, ignore the issue or fix it.

Agreed. We’re not using tailwind however is still having this issue. We’re using styled-components and I’m wondering what are the percentage of people having this issue on 9.5 that are using at least one of the css libraries (tailwind, styled-components, etc)

We are also having this issue with 10+ Next and using styled components.

I have the same problem. The project has 14 static pages + x dynamic pages + roughly a total of 100 components (small to large scale).

Base FE stacks

  • Next 9.5.x(last year) and 10.0.7(current)
  • React 16.x.x(last year) and 17.0.1(current)
  • Redux
  • theejs (WebGL)
  • and few other 3rd party services

The log:

<--- Last few GCs --->

[16497:0x102816000]    57372 ms: Mark-sweep 1323.8 (1403.1) -> 1323.8 (1398.6) MB, 367.2 / 0.0 ms  (average mu = 0.320, current mu = 0.000) last resort GC in old space requested
[16497:0x102816000]    57708 ms: Mark-sweep 1323.8 (1398.6) -> 1323.8 (1393.6) MB, 335.9 / 0.0 ms  (average mu = 0.192, current mu = 0.000) last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x180a2901e6e9 <JSObject>
    0: builtin exit frame: parse(this=0x180a29011a21 <Object map = 0x180ab85042a9>,0x180a3c1acf49 <Very long string[183289289]>,0x180a29011a21 <Object map = 0x180ab85042a9>)

    1: onread [0x180abb1159c9] [internal/child_process.js:~497] [pc=0x89d84652b4c](this=0x180ab7d686a9 <Pipe map = 0x180aba1c2cb1>,nread=1482,pool=0x180a3c1ad829 <Uint8Array map = 0x180a06f4d629>)
    2: InternalFrame [pc: ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: 0x10003d035 node::Abort() [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
 2: 0x10003d23f node::OnFatalError(char const*, char const*) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
 3: 0x1001b8e15 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
 4: 0x100586d72 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
 5: 0x100590274 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
 6: 0x10055f576 v8::internal::Factory::NewFixedArrayWithFiller(v8::internal::Heap::RootListIndex, int, v8::internal::Object*, v8::internal::PretenureFlag) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
 7: 0x10050b1eb v8::internal::(anonymous namespace)::ElementsAccessorBase<v8::internal::(anonymous namespace)::FastHoleyObjectElementsAccessor, v8::internal::(anonymous namespace)::ElementsKindTraits<(v8::internal::ElementsKind)3> >::ConvertElementsWithCapacity(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::FixedArrayBase>, v8::internal::ElementsKind, unsigned int, unsigned int, unsigned int, int) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
 8: 0x10050b0b7 v8::internal::(anonymous namespace)::ElementsAccessorBase<v8::internal::(anonymous namespace)::FastHoleyObjectElementsAccessor, v8::internal::(anonymous namespace)::ElementsKindTraits<(v8::internal::ElementsKind)3> >::GrowCapacityAndConvertImpl(v8::internal::Handle<v8::internal::JSObject>, unsigned int) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
 9: 0x10050b7cc v8::internal::(anonymous namespace)::FastElementsAccessor<v8::internal::(anonymous namespace)::FastHoleyObjectElementsAccessor, v8::internal::(anonymous namespace)::ElementsKindTraits<(v8::internal::ElementsKind)3> >::AddImpl(v8::internal::Handle<v8::internal::JSObject>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, unsigned int) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
10: 0x10069f49c v8::internal::JSObject::AddDataElement(v8::internal::Handle<v8::internal::JSObject>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::ShouldThrow) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
11: 0x1006a31ea v8::internal::JSObject::DefineOwnPropertyIgnoreAttributes(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::ShouldThrow, v8::internal::JSObject::AccessorInfoHandling) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
12: 0x1006a352e v8::internal::JSObject::SetOwnElementIgnoreAttributes(v8::internal::Handle<v8::internal::JSObject>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
13: 0x100659fee v8::internal::JsonParser<true>::ParseElement(v8::internal::Handle<v8::internal::JSObject>) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
14: 0x100658de8 v8::internal::JsonParser<true>::ParseJsonObject() [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
15: 0x100657cde v8::internal::JsonParser<true>::ParseJsonValue() [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
16: 0x100658f7d v8::internal::JsonParser<true>::ParseJsonObject() [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
17: 0x100657cde v8::internal::JsonParser<true>::ParseJsonValue() [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
18: 0x100659b0f v8::internal::JsonParser<true>::ParseJsonArray() [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
19: 0x100657a06 v8::internal::JsonParser<true>::ParseJsonValue() [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
20: 0x100657511 v8::internal::JsonParser<true>::ParseJson() [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
21: 0x1006573f1 v8::internal::JsonParser<true>::Parse(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>, v8::internal::Handle<v8::internal::Object>) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
22: 0x10027a043 v8::internal::Builtin_Impl_JsonParse(v8::internal::BuiltinArguments, v8::internal::Isolate*) [/Users/donghyukjang/.nvm/versions/node/v10.16.3/bin/node]
23: 0x89d8295bf9d 

I have been facing this issue since last year and still see it. I tried other suggestions above; like ensuring no circular importing and play with NODE_OPTIONS. but no luck. I believed it was because the WebGL component is very heavy.


It stops happening when removing the .next build directory and restart the local server.

I am also afraid of sharing the project application or the repository due to the project is private and confidential. I understand that it is just really tricky to give insight into this problem without looking at the actual project application.

Does anyone have any “feeling” or assumption of why refreshing the .next build directory folder remedies this problem immediately?

I have improved this situation a lot (as in I need to test more to consider it fixed, but it stopped crashing) by replacing tailwindcss with @tailwindcss/jit, I guess it is leaking memory in CSS processing and Tailwind does pull in a huge amount of CSS. If that’s the case, it’s probably not a Next.js problem but in some dependency.

May not be a global solution since people here didn’t report using Tailwind

CSS processing does seem to be a common theme. Thanks for the information.

Next 10.0.7 Circular dependency detected: node_modules/next/dist/client/with-router.js -> node_modules/next/dist/client/router.js -> node_modules/next/dist/client/with-router.js

@timneutkens I had also spotted/reported this circular dep in my attempts to debug this. Is this of any relevance to the out of memory dev server bug?

it’s reproducible by adding https://www.npmjs.com/package/next-circular-dependency to a NextJS app

fyi, this is still an issue as of next 10.0.7, I’ve experienced it with two projects with very different setups(one is using TS + apollo client, the other is plain JS + Redux). Both have been having severe issues with their dev servers, making working with nextJS much more difficult than it should be.

Obviously, i can’t post reproductions on this, since these are private projects.

Just wanted to give an update on our memory leak - we root caused it to a dependency, graphql-tag: apollographql/graphql-tag#356

But yeah, we’re still seeing dev servers crash regularly after a few changes. We’ve remedied by invoking the Next dev server with forever so it auto-restarts every time it dies.

@mmmeff, can you give me some insights on how you narrowed it down to graphql-tag for your app?

Between 9.4 and 9.5, there were changes in how Next.js handled Webpack hot middleware + some other fast refresh stuff so that’s probably where your error is coming from. I’ve spent countless hours tracking down changes between 9.4 and 9.5 both in what changed in Webpack, and what changed in Next.js, and sadly I just don’t think I know enough about bundlers/hash chunks (chunk hashes?? it’s been a few weeks since I looked sweat_smile )/Source maps and source nodes/all this stuff I’m encountering to really make sense of it.

Really interesting! Certainly looks like a likely culprit. Both for the errors I saw, and the introduction of these memory issues. Similarly, I know little about Webpack/bundlers, so investigating is hard. 😞

I find this interesting because I felt like the times I was having the server crash the most in development is when I was working on CSS (OC doesn’t use Tailwind, but does use the styled- packages) rather than other types of frontend work. Perhaps something to do with the way Webpack hashes CSS and how that interacts with the Next.js updates?

We’re using styled-components so that fits.

Does anyone have a good feel for how much work it would be to side-by-side compare projects with/without some CSS-heavy library?

I’m really unfamiliar with Next and the associated technologies, so I find the idea a bit daunting 😅

I just tried building Next.js locally and linking into our app as described here: https://github.com/vercel/next.js/blob/canary/contributing.md

I wanted to see if there was any truth to the idea that Next.js is using more memory in 9.5 vs 9.4, as per the OP:

it did rarely happen in 9.4.1, but it happens very consistantly in 9.5.x

However, I just get this error, and the server never seems to start:

(node:1926) UnhandledPromiseRejectionWarning: TypeError: (0 , _middleware.default) is not a function
    at HotReloader.assignBuildTools (/directory_of/next.js/packages/next/dist/server/hot-reloader.js:16:126)
    at HotReloader.start (/directory_of/next.js/packages/next/dist/server/hot-reloader.js:15:2084)
    at async DevServer.prepare (/directory_of/next.js/packages/next/dist/server/next-dev-server.js:10:1565)

Since that error doesn’t reference any of our own code, I don’t know where to start debugging, so I’m giving up for now.

Looking at memory use in htop though, I don’t see much difference, so I’m not convinced there’s been a big change in Next.js here. It’s possible that those of us who have started seeing this issue more often have simply increased the size of our apps: more dependencies, more pages, etc.

I have added Next Circular Dependency to my next.config.js https://www.npmjs.com/package/next-circular-dependency

This library uses https://www.npmjs.com/package/circular-dependency-plugin under the hood

I fixed any circular dependencies in my application code. The following remaining few circular dependencies highlighted in both NextJS and Shopify Polaris. Our team is working on replacing Shopify Polaris. I would highly recommend the Circular Dependency plugin to help determine if you have any memory leaks in your code.

➜  web-client git:(staging) yarn dev
yarn run v1.22.5
$ next dev
Browserslist: caniuse-lite is outdated. Please run:
npx browserslist@latest --update-db
Loaded env from /Users/ziagrosvenor/Work/ecologi/platform/web-client/.env.local
ready - started server on http://localhost:3000
info  - Using external babel configuration from /Users/ziagrosvenor/Work/ecologi/platform/web-client/babel.config.js
warn  - Circular dependency detected:
../node_modules/@shopify/app-bridge/actions/Error/actions.js -> ../node_modules/@shopify/app-bridge/actions/helper.js -> ../node_modules/@shopify/app-bridge/actions/Error/index.js -> ../node_modules/@shopify/app-bridge/actions/Error/actions.js

Circular dependency detected:
../node_modules/@shopify/app-bridge/actions/Error/index.js -> ../node_modules/@shopify/app-bridge/actions/Error/actions.js -> ../node_modules/@shopify/app-bridge/actions/helper.js -> ../node_modules/@shopify/app-bridge/actions/Error/index.js

Circular dependency detected:
../node_modules/@shopify/app-bridge/actions/helper.js -> ../node_modules/@shopify/app-bridge/actions/Error/index.js -> ../node_modules/@shopify/app-bridge/actions/Error/actions.js -> ../node_modules/@shopify/app-bridge/actions/helper.js

Circular dependency detected:
../node_modules/next/dist/client/router.js -> ../node_modules/next/dist/client/with-router.js -> ../node_modules/next/dist/client/router.js

Circular dependency detected:
../node_modules/next/dist/client/with-router.js -> ../node_modules/next/dist/client/router.js -> ../node_modules/next/dist/client/with-router.js
info  - ready on http://localhost:3000
event - build page: /

My next.config.js is the following…

const path = require("path");

const withPWA = require("next-pwa");
const withNextCircularDeps = require("next-circular-dependency");

// https://www.npmjs.com/package/next-transpile-modules
const withTM = require("next-transpile-modules")([
  "@ecologi/react-components",
  "@ecologi/assets",
]);

const withBundleAnalyzer = require("@next/bundle-analyzer")({
  enabled: !!process.env.BUNDLE_VISUALIZE,
});

const nodeModulesSassPath = path.resolve(
  __dirname,
  "../node_modules/@ecologi/react-components/src/style"
);

const prodConfig = {
  async headers() {
    return [
      {
        source: "/(.*?)",
        headers: [
          {
            key: "X-Frame-Options",
            value: "SAMEORIGIN",
          },
        ],
      },
    ];
  },
  async redirects() {
    return [
      {
        source: "/pages/faqs",
        destination: "/faqs",
        permanent: true,
      },
      {
        source: "/pages/about",
        destination: "/about",
        permanent: true,
      },
      {
        source: "/contact",
        destination: "/pages/contact",
        permanent: true,
      },
      {
        source: "/leaderboard",
        destination: "/leaderboards",
        permanent: true,
      },
      {
        source: "/adventurejunky",
        destination: "/plan",
        permanent: true,
      },
    ];
  },
};

const baseConfig = withNextCircularDeps(
  withTM({
    sassOptions: {
      includePaths: [nodeModulesSassPath],
    },
    failOnError: false,
    allowAsyncCycles: false,
    cwd: process.cwd(),
    webpack: (config, options) => {
      // Fixes npm packages that depend on `fs` module
      if (!options.isServer) {
        config.node = {
          fs: "empty",
        };
      }

      config.module.rules.push({
        test: /\.(svg)$/,
        include: [
          path.resolve(__dirname, "../node_modules/@ecologi/assets/icons"),
        ],
        loader: "svg-react-loader",
      });

      return config;
    },
  })
);

module.exports =
  process.env.NODE_ENV === "development"
    ? baseConfig
    : withBundleAnalyzer(
        withPWA({
          pwa: {
            dest: "public",
          },
          ...prodConfig,
          ...baseConfig,
        })
      );

I’ve tried to minimise the amount of webpack customisation in dev mode as shown in the above example, Disabling the headers and redirects features. However I am still seeing this issue with the dev server crashing frequently. It does seem to be related to navigation and switching pages in dev mode and also for example switch from one branch to another. It seems like these are both things which put a strain on NextJS and webpack.

These are the main libraries and webpack customisations I’m using in dev mode as follows…

    "next": "9.5.5",
    "react": "16.13.1",
    "react-dom": "16.13.1",
    "next-circular-dependency": "^1.0.1",
    "next-transpile-modules": "4.1.0",
    "@emotion/react": "11.1.1",
    "@emotion/styled": "11.0.0",
    "svg-react-loader": "0.4.6"

I also have a custom babel config as follows

module.exports = {
  presets: ["next/babel"],
  plugins: ["@emotion"],
  babelrcRoots: ["packages/*"],
};

I’ve tried NextJS version 10.0.6 but the dev server also crashes there. I’ve also tried Node LTS which didn’t fix the issue. I’m currently using v12.18.4

Adding the circular dependency plugin and fixing those issues seemed to help a little. Is there any insight on the circular dependency reported in Next JS?

../node_modules/next/dist/client/with-router.js -> ../node_modules/next/dist/client/router.js -> ../node_modules/next/dist/client/with-router.js

I can’t share the source code publicly however I would be happy to share it privately if there was any potential for some advice on fixing this issue. We are paying for the a Vercel hosting plan for all our websites and this dev server problem is really holding us back.

It would be great if there was first class support for TypeScript mono repos as this is the main custom webpack I am relying on in dev mode. I would love to remove all my webpack customisation, however it is not practical at this point.

Any advice on this would be greatly appreciated.

I had a massive memory leak and performance issue, which occurred with both the dev server and the production build. Turned out the reason was that I had set target: 'serverless' in my next.config.js, even though I was using a standard containerized “serverful” deployment.

I still have some memory leak issues when running the the dev server, but it doesn’t ever crash anymore since I’ve set 4GB max heap size for Node. With the 'serverless' target the dev server would crash even with 8GB of heap memory after loading around 10-20 pages.

I get this virtually every time after less than 10 reloads on Next 10

FWIW haven’t seen this happen since upgrading my system Node.js from 13 to 14. Will keep an eye on it.

I’m also getting this issue and I have customised my directory structure to look something like the following… Thought i’d share incase this may be a common theme amongst occurrences.

- pages/
    index.tsx
    a.tsx
    b.tsx
    c.tsx
- views/
    index/
     index.tsx
    a/
     index.tsx
    b/
     index.tsx
    c/
     index.tsx

Each file under the pages directory exports the index from the corresponding views directory. For example, pages/a.tsx just contains.

export { default } from '../views/a';

I use this structure as I like to locate all related code under one directory (also see https://github.com/vercel/next.js/issues/4789#issuecomment-476758132). I’m not sure if this is even related to the OOM issue but felt it like it might be.

i tried now setting NODE_OPTIONS='--max_old_space_size=4096' and it seems to be better at first glance

I can’t provide a reproduction due to sensitivity, but I can provide a next.config and plugins/env if that would be acceptable?

Unlikely we’ll be able to track it down based on just a next.config.js.

As said before if the only concern is that the code is sensitive we can look at enterprise support which would include a NDA.