next.js: [Next 9] out of memory when building the app
Bug report
Describe the bug
Moving an application from Next 8 to Next 9.
When I run next build the process goes out of memory and it cannot build the application.
FYI, it’s an application with 20 routes more or less.
To Reproduce
That’s hard to reproduce as I don’t have any idea what went wrong. Next 8 compiles without issue but not Next9.
Here’s the stack trace. If you know how to get a more descriptive output, let me know and I will provide:
<--- Last few GCs --->
[28143:0x102634000] 231190 ms: Mark-sweep 1278.7 (1441.0) -> 1263.4 (1438.5) MB, 838.4 / 0.0 ms (average mu = 0.290, current mu = 0.268) allocation failure scavenge might not succeed
[28143:0x102634000] 231308 ms: Scavenge 1278.1 (1438.5) -> 1266.6 (1440.0) MB, 8.2 / 0.0 ms (average mu = 0.290, current mu = 0.268) allocation failure
[28143:0x102634000] 231365 ms: Scavenge 1279.3 (1440.0) -> 1271.6 (1443.0) MB, 21.6 / 0.0 ms (average mu = 0.290, current mu = 0.268) allocation failure
<--- JS stacktrace --->
==== JS stack trace =========================================
0: ExitFrame [pc: 0x3547db9dbe3d]
Security context: 0x3a36ebf9e6e1 <JSObject>
1: DoJoin(aka DoJoin) [0x3a36ebf85e89] [native array.js:~87] [pc=0x3547dcd9a7d0](this=0x3a366f3826f1 <undefined>,l=0x3a368f6eb2b9 <JSArray[10]>,m=10,A=0x3a366f3828c9 <true>,w=0x3a36c3aafc69 <String[1]: />,v=0x3a366f3829a1 <false>)
2: Join(aka Join) [0x3a36ebf85ed9] [native array.js:~112] [pc=0x3547dd85bb38](this=0x3a366f3826f1 <undefined>,l=0x3a368f6...
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 0x10003ae75 node::Abort() [/usr/local/bin/node]
2: 0x10003b07f node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
3: 0x1001a7ae5 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
4: 0x100572ef2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/bin/node]
5: 0x1005759c5 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/bin/node]
6: 0x10057186f v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
7: 0x10056fa44 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
8: 0x10057c2dc v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
9: 0x10057c35f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
10: 0x10054e1e4 v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
11: 0x10082784d v8::internal::Runtime_StringBuilderJoin(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
12: 0x3547db9dbe3d
[1] 28142 abort npm run build:next
Expected behavior
It should build
Screenshots
If applicable, add screenshots to help explain your problem.
System information
- OS: macOS
- Node: 10.13.0 (it will have to work with node 8.X too)
- Version of Next.js: 9.0.1
Additional context
Add any other context about the problem here.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 98
- Comments: 87 (16 by maintainers)
@timneutkens could please reopen the issue now that you have a repo?
I’m getting same issue, even tried set NODE_OPTIONS=16384 on my 32GB physic memory MacBook Pro but no effect.
Logs are listed:
Having similar problem with
"next": "^10.0.4":Getting error:
Then dev falls:
cross-env NODE_OPTIONS=--max_old_space_size=8192 node server.js- Didn’t helpYep, we’re seeing this too…
Time to re-open I think @timneutkens
I’m facing the same problem with next@9.0.2.
I’m using next@11.0.1.
<— Last few GCs —>
[10292:000001AAE5E16820] 5217393 ms: Mark-sweep (reduce) 4068.4 (4105.0) -> 4067.6 (4105.2) MB, 2035.7 / 0.1 ms (average mu = 0.065, current mu = 0.002) allocation failure scavenge might not succeed [10292:000001AAE5E16820] 5219164 ms: Mark-sweep (reduce) 4069.0 (4102.2) -> 4068.3 (4103.7) MB, 1766.8 / 0.1 ms (average mu = 0.033, current mu = 0.003) allocation failure scavenge might not succeed
<— JS stacktrace —>
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory 1: 00007FF70B974BDF napi_wrap+111007 2: 00007FF70B918166 v8::base::CPU::has_sse+59910 3: 00007FF70B919066 node::OnFatalError+294 4: 00007FF70C1F217E v8::Isolate::ReportExternalAllocationLimitReached+94 5: 00007FF70C1D6F4D v8::SharedArrayBuffer::Externalize+781 6: 00007FF70C08008C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1516 7: 00007FF70C08B4AA v8::internal::Heap::ProtectUnprotectedMemoryChunks+1258 8: 00007FF70C0885E9 v8::internal::Heap::PageFlagsAreConsistent+2457 9: 00007FF70C07D181 v8::internal::Heap::CollectGarbage+2049 10: 00007FF70C07B385 v8::internal::Heap::AllocateExternalBackingStore+1349 11: 00007FF70C09B7FB v8::internal::Factory::NewFillerObject+203 12: 00007FF70BDCA2D1 v8::internal::interpreter::JumpTableTargetOffsets::iterator::operator=+1409 13: 00007FF70C27B03D v8::internal::SetupIsolateDelegate::SetupHeap+465325 14: 00007FF70C27BEED v8::internal::SetupIsolateDelegate::SetupHeap+469085 15: 0000034A2DB54C85
Adding another witness. I inherited an app with Next version 4.2.3.
When I did a local serve of the project it failed with the following error:
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memoryThe failure usually happened within less than ten minutes of starting the app and could be hastened by triggering more navigation in the app.I have reverted to version 8.1.0 and cannot reproduce the same failure–even with more earnest navigation (and thus page-builds).
As far as I understood, Next.js (since v9.0.0) is not able to optimize the memory of large web applications. Our build process hits the memory limit when it starts to compile the server side part of the project.
During development mode, the local server is fine initially but after some time the allocated memory blows up and the memory heap error shows up.
Could someone from the core team please acknowledge this issue and help us to triage the problem?
Me and my team, even with powerful laptops, are not able to build locally the application sometimes. This issue is almost 1 year old but there hasn’t been a clear update around the issue. Personally, the only workaround that I can think of is to split the application in two sub Next.js projects because I am concerned that one day we won’t even be able to build the application even on the CI (and we already using powerful VMs for doing it).
I am so sorry to be the bad guy here, I am just seeking for help.
EDIT: we updated to Next.js 9.3.3 but there haven’t been improvements
@timneutkens please reopen this issue, or open a new one. This memory leak as well as some weird infinite HMR requests is making our app unusable in development mode and eating up a huge amount of our time.
In case any of you is using
next-transpile-modules, my solution was to enableresolveSymlinks-setting (since v4.1.0) like so:I first encountered the issue described here when
somepackagebecame “too big” (from 500kb to 700kb), now all of a sudden, all of my memory (also tried with the 8gb option) was not enough to handle this.I believe that there are some symlink-caused loops which cause memory to get bloated exponentially.
Not entirely sure if this will be the same for everyone, but make sure to check that any resource you are requesting can actually be found. I had an image that my app couldn’t find and it kept looking for it until it gave me the heap error. I removed the mention of that image and it hasn’t given me the error again.
Also seeing this with Next 10.0.3 while running
next devin a local dev environment (2019 Macbook Pro).I don’t have specific steps to reproduce. This seems to happen for me in a small project we’re running on Next, where the server behaves fine for around 30 minutes or so, then eventually runs out of memory and crashes.
I’ve put a copy of my stack trace here if anyone wants it, rather than bloat the thread more.
@timneutkens, I have the latest version (9.3.5). I created the project this morning and faced that error some minutes later.
I had this problem too. As bukinoshita mentioned it, I looked into the static folder. I removed a folder of 674 json files (for testing purposes) out of the
public/static/folder. I did not had the problem since.I’m guessing these application(s) have simply hit their size bounds to start running out of memory. You’ll need to run your build with more via
NODE_OPTIONS=--max-old-space-size=4096.Alternatively, you can enable our new chunking feature:
@timneutkens I’m running out of memory using Next.js 9 as well. It could be because of the number of components we are compiling. You mentioned you needed a full repro of it. You can take a look at my branch. https://github.com/stramel/styled-icons/tree/ms/add-material-variants
Here is the failing build: https://travis-ci.org/jacobwgillespie/styled-icons/builds/582987078
Travis has
NODE_OPTIONS=--max-old-space-size=4096setconfirming the same issue in 11.1.2 also. I observed that after executing
npm run dev, project compiles successfully and if you open HTOP in the meanwhile, you will see that RAM usage is constantly increasing and when it reaches 7.1 GB in my 8GB RAM laptop, development server is crashed with the above error.It feels that no single component can easily trigger this, from my experience it happens more frequently when a thick stack of plugins (UI frameworks, sass, typescript, apollo/axios) works with mid sized projects (e.g. 30+ routes).
For anyone who might be dealing with a runaway
next buildrunning out of memory, try deleting your.babelrcorbabel.config.js. I have a.babelrcfor a separate part of my project not related to Next.js, and it doesn’t agree withnext build. In my situation, the issue was only happening in Docker. I fixed it by changing my Dockerfile from thisto this
The reproduction can’t be executed, hence why I asked every single person that reported here to also provide a reproduction to avoid cases like this. Also the potential for these issues to be 100% the same is quite low.
Sourcemaps are always enabled in development, we switched to eval source maps though.
I re-opened it because more reports kept coming in and I kept asking for reproductions: https://github.com/zeit/next.js/issues/7929#issuecomment-568760542
There’s only 2 reproductions provided in this thread. One is completely unrelated to Next.js (
now devmemory consumption) and the other can’t be executed/built.Hence why I’m once again asking for a full reproduction otherwise this will continue to be impossible to investigate.
Feel free to reach out to enterprise@vercel.com and we can set up a NDA, potentially this does mean we’ll have to do consulting for you though given it’ll take a significant amount of time to set all this up just to help you with your particular application and you’re building the application for a client also.
Keep in mind that posting “I still experience this” won’t solve the issue. Provide a full reproduction for people to look into, otherwise your comment is the same as doing a 👍 on the initial issue but pinging everyone in the issue for no reason.
Always make sure your comments are actionable or useful. Eg like ematipico sharing things based on investigating the issue is useful. Saying “I still have this issue” is not useful.
Have same problem here as well. It seems to crash when the file size in build are too big. For example, I imported something from Typescript and the file size in build went up to 2.41 mb. Then my CI with 2GB ram started to crash. After I removed the Typescript import the file size went down to 100kb and it worked again.
Nextjs 9 has been very slow in CI from start. It takes very long to build and I dont have any special needs… just react stuff, material ui, some packages here and there. I have CI:s in same cluster with node/express, some dotnet core, php etc all these has 1GB memory in CI and builds pretty fast. I dont know more than my feeling about the build process has some issue maybe?
This issue happens for one of our repos when we change the build to perform minification, which is disabled by default now.
Update: I determined that in our codebase the issue was using the
withSourceMapsplugin:"@zeit/next-source-maps": "^0.0.4-canary.1"This makes sense since we have it set to
hidden-source-mapwhich is very slow. Though, I don’t know why it’s exponentially slower than NextJS 8.@vicary
import typedoes resolve my problem as well. thanks for the tip!@ematipico the problem is that I dont think anyone found a clear path to reproduce the issue at this point, it just happens from time to time, the time in between doesnt seem to follow a pattern. You’d need to run it and keep checking it until it happens. Also I dont know if it’s related to accessing pages or not, so there’s a chace that just running the project and waiting doesn’t reproduce it. Also, because this happens in production, with actual people entering the website, to have a more reliable reproduction one should probably consider a non-cache behavior from browsers accessing it to make sure every request is like a new user in your website.
@szaza can you try
next@canary, the most likely case is that you have the issue described here: https://github.com/zeit/next.js/issues/12280I started getting this error after updating to next 9.3.6 today:
Fixed it by modifiying
pathsintsconfig.jsonas suggested in #12280Same problem here. Mac OS / Github Actions. No actions mentioned here helps, can’t find any files that are linked but not existent.
However, project builds (and is in production) in Zeit Now.
Would love to help if knew how.
We too have noticed that after upgrading to Next 9 we are getting out of memory errors whereas with Next 8 this wasn’t an issue.
In our case we are building our app in CircleCi as part of our CI flow and encountering the following:
4GB is memory limit for a CircleCI container, hence the error. It appears building with Next 9 we are hitting that limit.
I don’t have a reproducible case to share at the moment.