next.js: [Next 9] out of memory when building the app

Bug report

Describe the bug

Moving an application from Next 8 to Next 9. When I run next build the process goes out of memory and it cannot build the application.

FYI, it’s an application with 20 routes more or less.

To Reproduce

That’s hard to reproduce as I don’t have any idea what went wrong. Next 8 compiles without issue but not Next9.

Here’s the stack trace. If you know how to get a more descriptive output, let me know and I will provide:



<--- Last few GCs --->

[28143:0x102634000]   231190 ms: Mark-sweep 1278.7 (1441.0) -> 1263.4 (1438.5) MB, 838.4 / 0.0 ms  (average mu = 0.290, current mu = 0.268) allocation failure scavenge might not succeed
[28143:0x102634000]   231308 ms: Scavenge 1278.1 (1438.5) -> 1266.6 (1440.0) MB, 8.2 / 0.0 ms  (average mu = 0.290, current mu = 0.268) allocation failure 
[28143:0x102634000]   231365 ms: Scavenge 1279.3 (1440.0) -> 1271.6 (1443.0) MB, 21.6 / 0.0 ms  (average mu = 0.290, current mu = 0.268) allocation failure 


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x3547db9dbe3d]
Security context: 0x3a36ebf9e6e1 <JSObject>
    1: DoJoin(aka DoJoin) [0x3a36ebf85e89] [native array.js:~87] [pc=0x3547dcd9a7d0](this=0x3a366f3826f1 <undefined>,l=0x3a368f6eb2b9 <JSArray[10]>,m=10,A=0x3a366f3828c9 <true>,w=0x3a36c3aafc69 <String[1]: />,v=0x3a366f3829a1 <false>)
    2: Join(aka Join) [0x3a36ebf85ed9] [native array.js:~112] [pc=0x3547dd85bb38](this=0x3a366f3826f1 <undefined>,l=0x3a368f6...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003ae75 node::Abort() [/usr/local/bin/node]
 2: 0x10003b07f node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x1001a7ae5 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x100572ef2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/bin/node]
 5: 0x1005759c5 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/bin/node]
 6: 0x10057186f v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
 7: 0x10056fa44 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 8: 0x10057c2dc v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
 9: 0x10057c35f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
10: 0x10054e1e4 v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
11: 0x10082784d v8::internal::Runtime_StringBuilderJoin(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
12: 0x3547db9dbe3d 
[1]    28142 abort      npm run build:next

Expected behavior

It should build

Screenshots

If applicable, add screenshots to help explain your problem.

System information

  • OS: macOS
  • Node: 10.13.0 (it will have to work with node 8.X too)
  • Version of Next.js: 9.0.1

Additional context

Add any other context about the problem here.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 98
  • Comments: 87 (16 by maintainers)

Most upvoted comments

@timneutkens could please reopen the issue now that you have a repo?

I’m getting same issue, even tried set NODE_OPTIONS=16384 on my 32GB physic memory MacBook Pro but no effect.

Logs are listed:

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x335bafb5be3d]
Security context: 0x0fe28be1e6e9 <JSObject>
    1: createPrinter [0xfe25fe2c0b9] [/Users/my-name/my-project/node_modules/typescript/lib/typescript.js:~86713] [pc=0x335bb12e86f3](this=0x0fe280c03329 <Object map = 0xfe2df152cc9>,printerOptions=0x0fe2f39e8ae1 <Object map = 0xfe20ac205a9>,handlers=0x0fe28e1026f1 <undefined>)
    2: arguments adaptor frame: 1->2
    3: reportImplicitAny(aka reportImpli...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003d035 node::Abort() [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 2: 0x10003d23f node::OnFatalError(char const*, char const*) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 3: 0x1001b8e15 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 4: 0x100586d72 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 5: 0x100589845 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 6: 0x1005856ef v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 7: 0x1005838c4 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 8: 0x10059015c v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 9: 0x1005901df v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
10: 0x10055fb24 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
11: 0x1007e7e04 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
12: 0x335bafb5be3d
13: 0x335bb12e86f3
14: 0x335bafb0a5c3

Having similar problem with "next": "^10.0.4":

Getting error:

Uncaught Error: Rendered more hooks than during the previous render.
    at updateWorkInProgressHook (react-dom.development.js?1843:15002)
...

Then dev falls:

<--- Last few GCs --->

[5228:0x148008000]  3435941 ms: Scavenge 907.3 (1022.9) -> 905.5 (1022.7) MB, 3.0 / 0.0 ms  (average mu = 0.996, current mu = 0.929) allocation failure 
[5228:0x148008000]  3435955 ms: Scavenge 914.6 (1023.2) -> 907.3 (1023.0) MB, 3.8 / 0.0 ms  (average mu = 0.996, current mu = 0.929) allocation failure 
[5228:0x148008000]  3435979 ms: Scavenge 921.0 (1023.2) -> 907.8 (1023.0) MB, 2.6 / 0.0 ms  (average mu = 0.996, current mu = 0.929) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: wasm code commit Allocation failed - process out of memory
 1: 0x1040ca9d8 node::Abort() [/Users/user/.nvm/versions/node/v14.15.4/bin/node]
 2: 0x1040cab58 node::errors::TryCatchScope::~TryCatchScope() [/Users/user/.nvm/versions/node/v14.15.4/bin/node]
...
  • Macbook M1 - Big Sur - 16GB RAM
  • Tried cross-env NODE_OPTIONS=--max_old_space_size=8192 node server.js - Didn’t help

Yep, we’re seeing this too…

Time to re-open I think @timneutkens

I’m facing the same problem with next@9.0.2.

I’m using next@11.0.1.

<— Last few GCs —>

[10292:000001AAE5E16820] 5217393 ms: Mark-sweep (reduce) 4068.4 (4105.0) -> 4067.6 (4105.2) MB, 2035.7 / 0.1 ms (average mu = 0.065, current mu = 0.002) allocation failure scavenge might not succeed [10292:000001AAE5E16820] 5219164 ms: Mark-sweep (reduce) 4069.0 (4102.2) -> 4068.3 (4103.7) MB, 1766.8 / 0.1 ms (average mu = 0.033, current mu = 0.003) allocation failure scavenge might not succeed

<— JS stacktrace —>

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory 1: 00007FF70B974BDF napi_wrap+111007 2: 00007FF70B918166 v8::base::CPU::has_sse+59910 3: 00007FF70B919066 node::OnFatalError+294 4: 00007FF70C1F217E v8::Isolate::ReportExternalAllocationLimitReached+94 5: 00007FF70C1D6F4D v8::SharedArrayBuffer::Externalize+781 6: 00007FF70C08008C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1516 7: 00007FF70C08B4AA v8::internal::Heap::ProtectUnprotectedMemoryChunks+1258 8: 00007FF70C0885E9 v8::internal::Heap::PageFlagsAreConsistent+2457 9: 00007FF70C07D181 v8::internal::Heap::CollectGarbage+2049 10: 00007FF70C07B385 v8::internal::Heap::AllocateExternalBackingStore+1349 11: 00007FF70C09B7FB v8::internal::Factory::NewFillerObject+203 12: 00007FF70BDCA2D1 v8::internal::interpreter::JumpTableTargetOffsets::iterator::operator=+1409 13: 00007FF70C27B03D v8::internal::SetupIsolateDelegate::SetupHeap+465325 14: 00007FF70C27BEED v8::internal::SetupIsolateDelegate::SetupHeap+469085 15: 0000034A2DB54C85

Adding another witness. I inherited an app with Next version 4.2.3.

  • Updated it to Next version 9.1.3
  • Updated all other packages version in package.json;
  • Deleted the node_modules folder and package.lock.json
  • ran npm install

When I did a local serve of the project it failed with the following error: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory The failure usually happened within less than ten minutes of starting the app and could be hastened by triggering more navigation in the app.

I have reverted to version 8.1.0 and cannot reproduce the same failure–even with more earnest navigation (and thus page-builds).

As far as I understood, Next.js (since v9.0.0) is not able to optimize the memory of large web applications. Our build process hits the memory limit when it starts to compile the server side part of the project.

During development mode, the local server is fine initially but after some time the allocated memory blows up and the memory heap error shows up.

Could someone from the core team please acknowledge this issue and help us to triage the problem?

Me and my team, even with powerful laptops, are not able to build locally the application sometimes. This issue is almost 1 year old but there hasn’t been a clear update around the issue. Personally, the only workaround that I can think of is to split the application in two sub Next.js projects because I am concerned that one day we won’t even be able to build the application even on the CI (and we already using powerful VMs for doing it).

I am so sorry to be the bad guy here, I am just seeking for help.

EDIT: we updated to Next.js 9.3.3 but there haven’t been improvements

@timneutkens please reopen this issue, or open a new one. This memory leak as well as some weird infinite HMR requests is making our app unusable in development mode and eating up a huge amount of our time.

In case any of you is using next-transpile-modules, my solution was to enable resolveSymlinks-setting (since v4.1.0) like so:

// next.config.js
const withTM = require("next-transpile-modules")([
	"somepackage"
], {
	resolveSymlinks: true
})
module.exports = {
	...withTM()
}

I first encountered the issue described here when somepackage became “too big” (from 500kb to 700kb), now all of a sudden, all of my memory (also tried with the 8gb option) was not enough to handle this.

I believe that there are some symlink-caused loops which cause memory to get bloated exponentially.

Not entirely sure if this will be the same for everyone, but make sure to check that any resource you are requesting can actually be found. I had an image that my app couldn’t find and it kept looking for it until it gave me the heap error. I removed the mention of that image and it hasn’t given me the error again.

Also seeing this with Next 10.0.3 while running next dev in a local dev environment (2019 Macbook Pro).

I don’t have specific steps to reproduce. This seems to happen for me in a small project we’re running on Next, where the server behaves fine for around 30 minutes or so, then eventually runs out of memory and crashes.

I’ve put a copy of my stack trace here if anyone wants it, rather than bloat the thread more.

@timneutkens, I have the latest version (9.3.5). I created the project this morning and faced that error some minutes later.

I had this problem too. As bukinoshita mentioned it, I looked into the static folder. I removed a folder of 674 json files (for testing purposes) out of the public/static/ folder. I did not had the problem since.

I’m guessing these application(s) have simply hit their size bounds to start running out of memory. You’ll need to run your build with more via NODE_OPTIONS=--max-old-space-size=4096.

Alternatively, you can enable our new chunking feature:

// next.config.js
module.exports = {
  experimental: { granularChunks: true }
}

@timneutkens I’m running out of memory using Next.js 9 as well. It could be because of the number of components we are compiling. You mentioned you needed a full repro of it. You can take a look at my branch. https://github.com/stramel/styled-icons/tree/ms/add-material-variants

Here is the failing build: https://travis-ci.org/jacobwgillespie/styled-icons/builds/582987078

Travis has NODE_OPTIONS=--max-old-space-size=4096 set

I’m using next@11.0.1.

<— Last few GCs —>

[10292:000001AAE5E16820] 5217393 ms: Mark-sweep (reduce) 4068.4 (4105.0) -> 4067.6 (4105.2) MB, 2035.7 / 0.1 ms (average mu = 0.065, current mu = 0.002) allocation failure scavenge might not succeed [10292:000001AAE5E16820] 5219164 ms: Mark-sweep (reduce) 4069.0 (4102.2) -> 4068.3 (4103.7) MB, 1766.8 / 0.1 ms (average mu = 0.033, current mu = 0.003) allocation failure scavenge might not succeed

<— JS stacktrace —>

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory 1: 00007FF70B974BDF napi_wrap+111007 2: 00007FF70B918166 v8::base::CPU::has_sse+59910 3: 00007FF70B919066 node::OnFatalError+294 4: 00007FF70C1F217E v8::Isolate::ReportExternalAllocationLimitReached+94 5: 00007FF70C1D6F4D v8::SharedArrayBuffer::Externalize+781 6: 00007FF70C08008C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1516 7: 00007FF70C08B4AA v8::internal::Heap::ProtectUnprotectedMemoryChunks+1258 8: 00007FF70C0885E9 v8::internal::Heap::PageFlagsAreConsistent+2457 9: 00007FF70C07D181 v8::internal::Heap::CollectGarbage+2049 10: 00007FF70C07B385 v8::internal::Heap::AllocateExternalBackingStore+1349 11: 00007FF70C09B7FB v8::internal::Factory::NewFillerObject+203 12: 00007FF70BDCA2D1 v8::internal::interpreter::JumpTableTargetOffsets::iterator::operator=+1409 13: 00007FF70C27B03D v8::internal::SetupIsolateDelegate::SetupHeap+465325 14: 00007FF70C27BEED v8::internal::SetupIsolateDelegate::SetupHeap+469085 15: 0000034A2DB54C85

confirming the same issue in 11.1.2 also. I observed that after executing npm run dev, project compiles successfully and if you open HTOP in the meanwhile, you will see that RAM usage is constantly increasing and when it reaches 7.1 GB in my 8GB RAM laptop, development server is crashed with the above error.

It feels that no single component can easily trigger this, from my experience it happens more frequently when a thick stack of plugins (UI frameworks, sass, typescript, apollo/axios) works with mid sized projects (e.g. 30+ routes).

For anyone who might be dealing with a runaway next build running out of memory, try deleting your .babelrc or babel.config.js. I have a .babelrc for a separate part of my project not related to Next.js, and it doesn’t agree with next build. In my situation, the issue was only happening in Docker. I fixed it by changing my Dockerfile from this

FROM node:12
WORKDIR /app
COPY package.json package-lock.json /app/
ENV NODE_ENV production
RUN npm install
COPY . /app
RUN npm run build
RUN npm run next-build
EXPOSE 8081
CMD ["npm", "start"]

to this

FROM node:12
WORKDIR /app
COPY package.json package-lock.json /app/
ENV NODE_ENV production
RUN npm install
COPY . /app
RUN npm run build
# Let Next.js use its own Babel config
RUN rm .babelrc
RUN npm run next-build
EXPOSE 8081
CMD ["npm", "start"]
▲  styled-icons (master) ✗ yarn build:ci
yarn run v1.22.4
$ lerna run build
lerna notice cli v3.21.0
lerna info Executing command in 25 packages: "yarn run build"
lerna info run Ran npm script 'build' in '@styled-icons/styled-icon' in 6.6s:
$ tsc --project tsconfig.esm.json && mv dist/index.js dist/index.esm.js && tsc --project tsconfig.json
lerna ERR! yarn run build exited 1 in '@styled-icons/boxicons-logos'
lerna ERR! yarn run build stdout:
$ build-pack
Reading icon packs...
Error reading icons from pack
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

lerna ERR! yarn run build stderr:
error Command failed with exit code 1.

lerna ERR! yarn run build exited 1 in '@styled-icons/boxicons-logos'
lerna WARN complete Waiting for 4 child processes to exit. CTRL-C to exit immediately.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

The reproduction can’t be executed, hence why I asked every single person that reported here to also provide a reproduction to avoid cases like this. Also the potential for these issues to be 100% the same is quite low.

Is this feature opt-in/opt-out? I removed the generation of source maps (it was done via plugin) after I opened the issue (so long time ago) but it didn’t help much.

Sourcemaps are always enabled in development, we switched to eval source maps though.

That’s why the issue was reopened and that’s why it doesn’t have the label that says it requires a repository to reproduce the issue.

I re-opened it because more reports kept coming in and I kept asking for reproductions: https://github.com/zeit/next.js/issues/7929#issuecomment-568760542

There’s only 2 reproductions provided in this thread. One is completely unrelated to Next.js (now dev memory consumption) and the other can’t be executed/built.

Hence why I’m once again asking for a full reproduction otherwise this will continue to be impossible to investigate.

I can’t provide a repository because the web application belongs to my client and the source code is closed.

Feel free to reach out to enterprise@vercel.com and we can set up a NDA, potentially this does mean we’ll have to do consulting for you though given it’ll take a significant amount of time to set all this up just to help you with your particular application and you’re building the application for a client also.

Keep in mind that posting “I still experience this” won’t solve the issue. Provide a full reproduction for people to look into, otherwise your comment is the same as doing a 👍 on the initial issue but pinging everyone in the issue for no reason.

Always make sure your comments are actionable or useful. Eg like ematipico sharing things based on investigating the issue is useful. Saying “I still have this issue” is not useful.

Have same problem here as well. It seems to crash when the file size in build are too big. For example, I imported something from Typescript and the file size in build went up to 2.41 mb. Then my CI with 2GB ram started to crash. After I removed the Typescript import the file size went down to 100kb and it worked again.

Nextjs 9 has been very slow in CI from start. It takes very long to build and I dont have any special needs… just react stuff, material ui, some packages here and there. I have CI:s in same cluster with node/express, some dotnet core, php etc all these has 1GB memory in CI and builds pretty fast. I dont know more than my feeling about the build process has some issue maybe?

This issue happens for one of our repos when we change the build to perform minification, which is disabled by default now.

Update: I determined that in our codebase the issue was using the withSourceMaps plugin:

"@zeit/next-source-maps": "^0.0.4-canary.1"

This makes sense since we have it set to hidden-source-map which is very slow. Though, I don’t know why it’s exponentially slower than NextJS 8.

@vicary import type does resolve my problem as well. thanks for the tip!

@ematipico the problem is that I dont think anyone found a clear path to reproduce the issue at this point, it just happens from time to time, the time in between doesnt seem to follow a pattern. You’d need to run it and keep checking it until it happens. Also I dont know if it’s related to accessing pages or not, so there’s a chace that just running the project and waiting doesn’t reproduce it. Also, because this happens in production, with actual people entering the website, to have a more reliable reproduction one should probably consider a non-cache behavior from browsers accessing it to make sure every request is like a new user in your website.

@szaza can you try next@canary, the most likely case is that you have the issue described here: https://github.com/zeit/next.js/issues/12280

I started getting this error after updating to next 9.3.6 today:

<--- Last few GCs --->

[66325:0x108008000]    17639 ms: Mark-sweep 1936.2 (2071.4) -> 1923.6 (2073.4) MB, 283.4 / 0.0 ms  (average mu = 0.157, current mu = 0.048) allocation failure scavenge might not succeed
[66325:0x108008000]    17937 ms: Mark-sweep 1938.1 (2073.4) -> 1925.4 (2075.4) MB, 285.8 / 0.0 ms  (average mu = 0.108, current mu = 0.042) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x0fb9ac667d09 <JSObject>
    0: builtin exit frame: stringify(this=0x0fb9ac6598a9 <Object map = 0xfb921b01449>,0x0fb958e804d1 <undefined>,0x0fb958e804d1 <undefined>,0x0fb910180ec1 <Object map = 0xfb9d8491399>,0x0fb9ac6598a9 <Object map = 0xfb921b01449>)

    1: arguments adaptor frame: 1->3
    2: callAsync [0xfb9320a6cf9] [0x0fb958e804d1 <undefined>:~1] [pc=0x1aa09fe84627](this=0x0fb9320a6909 <Hook map = 0xfb9d8495179>...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
...

Fixed it by modifiying paths in tsconfig.json as suggested in #12280

Same problem here. Mac OS / Github Actions. No actions mentioned here helps, can’t find any files that are linked but not existent.

However, project builds (and is in production) in Zeit Now.

Would love to help if knew how.

We too have noticed that after upgrading to Next 9 we are getting out of memory errors whereas with Next 8 this wasn’t an issue.

In our case we are building our app in CircleCi as part of our CI flow and encountering the following:

Exited with code 137
Hint: Exit code 137 typically means the process is killed because it was running out of memory
Hint: Check if you can optimize the memory usage in your app
Hint: Max memory usage of this container is 4284268544
 according to /sys/fs/cgroup/memory/memory.max_usage_in_bytes

4GB is memory limit for a CircleCI container, hence the error. It appears building with Next 9 we are hitting that limit.

I don’t have a reproducible case to share at the moment.