next.js: Next 9.5.1 out of memory after some hot reloads
Bug report
Describe the bug
Since updating to 9.5.x (from 9.4.x), i get an out of memory error after 10 something hot reloads:
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
it did rarely happen in 9.4.1, but it happens very consistantly in 9.5.x
To Reproduce
thats probably tricky. it happens on big projects and might be related to some bug in the hot reload / rebuild. Maybe it happens when there are some import circles?
Expected behavior
nextjs should not go out-of-memory
System information
- OS:macOS
- Browser: chrome
- Version of Next.js: 9.5.1
- Version of Node.js: 12.13.1
Additional information
we are using a custom server with typescript
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 177
- Comments: 144 (18 by maintainers)
Commits related to this issue
- Upgrade jest-worker (#23077) This PR upgrades `jest-worker` and `jest-cli` to the latest pre-release version, also removed `jest-circus` which is included in Jest by default. `jest-worker@next` inclu... — committed to vercel/next.js by shuding 3 years ago
- Upgrade jest-worker (#23077) This PR upgrades `jest-worker` and `jest-cli` to the latest pre-release version, also removed `jest-circus` which is included in Jest by default. `jest-worker@next` inclu... — committed to blitz-js/next.js by shuding 3 years ago
- Correctly clean up router listeners https://nextjs.org/docs/api-reference/next/router#usage-6 Didn't seem to affect the memory usage of the app even thought this suggested it should've: https://gith... — committed to skoleapp/skole-frontend by ruohola 3 years ago
- Correctly clean up router listeners https://nextjs.org/docs/api-reference/next/router#usage-6 Didn't seem to affect the memory usage of the app even thought this suggested it should've: https://gith... — committed to skoleapp/skole-frontend by ruohola 3 years ago
- Correctly clean up router listeners https://nextjs.org/docs/api-reference/next/router#usage-6 Didn't seem to affect the memory usage of the app, even thought this suggested it should've: https://git... — committed to skoleapp/skole-frontend by ruohola 3 years ago
this is extremly hard to do, but i am 100% sure others have the same issue. Do you hav any idea how this could happen? Its a new problem and happens very consistantly.
maybe circular dependencies? I investigate now in that
Just a small update:
Used
node-oom-heapdumpto grab a heap snapshot of our app when it crashed.It seems to call out one of Webpack’s dependencies,
watchpack(and itsDirectoryWatcher.js), which makes sense to me because my crashes happen the most when I’m in dev mode, switching between branches that have a large number of changed files between them.I have sent the heap snapshot to the Next.js team so hopefully it will help them look into this. I think I’m potentially going to open an issue on the Watchpack repo as well.
Haven’t heard anything from our development teams or others, vercel.com is a Next.js app with 250 unique Next.js pages so we would have definitely have run into it if it was as trivial as you’re making it to be.
Issues with memory are practically impossible to investigate without having the application so now I can’t give you pointers on where to look. You’ll have to run tooling to track down where memory is allocated (e.g. the Node.js debugger profiler).
If it makes it any easier to share the application we can also investigate it under (paid) enterprise support if that makes the company you work for share the application, which will likely save your development team significant amounts of time.
I have been experiencing memory leak since 9.5.2 (I updated yesterday). I was working on the /api/ files yesterday and this morning when it will randomly crash during hot reloads.
Since today afternoon, I was working on material-ui and some api
After I kept making changes to makeStyles, and changing the classes back and forth (because I’m trying to get the layout right), then it will crash after quite a few hot reload compiling.
Below, shows the number of hot reloads (in between the 2 crashes, I was working on material-ui’s makeStyles and adding/removing classes to different component) and compiles while I’m playing with makeStyles and material-ui’s Accordion component.
I’m not sure whether the logs below help, but I have to say the crashes are all pretty random, so far crashed about 10-15 times in the span of 24 hours, sometimes when working on /api/ sometimes while working on material-ui.
Same issue with latest 12.0.0
Still happening with latest v10, in case anyone was hoping it could’ve been solved by chance.
Quick follow up: these are some messages I’ve been getting in my terminal warning of a memory leak.
It points specifically to this file in our frontend repo, which has an unnamed default export. It’s one of the oldest files in our repo and definitely wasn’t causing issues before. Perhaps there’s been a change in the way Next.js handles these default unnamed exports that would cause the memory leak? This for example seems to be a new addition released within the last few weeks that was flagging that particular error, and warning of a memory leak.
I played around making some changes to a component I’ve been working on this week while the crashes have been most notable, and was able to trigger one again just now.
Here’s the output from my terminal:
Node had been using around ~1.9 GB for a while and I’m not sure if it helped trigger it or if it was coincidence but when I switched branches and made a few more changes that seemed to push it over the edge of the available 2GB of memory.
Hey, it seems that going to Next 10.2.1 (including #24282) + using webpack5 has solved the issue in my case. The memory is now stable and in particular the
(string)instances (source-map ?) in the heap snapshot that used to grow to several Gb are now stable and less than 300 Mb.I’m using Next 10 (@latest) with Material-UI. After reading/skimming through this topic I incrementally raised the Node memory to 4, 6 and 8 GB. But even at 8 it will eventually run out of memory, although this is only after several days without restarting my PC and who normally codes that long in one go…?
Anyway, I do agree with @Dunky45 on this:
So I want to add a demo here to show this behavior. In this demo I artificially lowered the amount of memory for Node to 128MB, which is obviously unrealistically low. But at least it consistently shows the behavior we see at higher memory amounts. If you increase the amount of memory the crash won’t happen directly, unless you keep adding to and changing the style object in
styles.js, but it takes some time (e.g. ~10 recompiles at 160MB).@timneutkens Let me know if I missed or forgot to supply some information, as this is the first time ever I share a demo.
It’s often not possible to send over reproductions, so you probably need to debug these memory problems on your own.
So here are some hints how to find them:
NODE_OPTIONS=--heapsnapshot-near-heap-limit=1to automatically create a heapsnapshot just before the process went OOM.node --inspect node_modules/next/dist/bin/nextallows to attach Node.js devtools (chrome://inspect/edge://inspect, click on theOpen dedicated DevTools for Nodelink)Memorytab is your fried. Here you can take a heapsnapshot. Click the GC button before doing that (trash can).Compilation. Are there manyCompilationinstances? There should only be two or very few.Objects allocated between Snapshot 1 and 2right to theClass Filter. (Leave the class filter empty for this bullet point)Retained Size. Investigate into the biggest leaks.Retainers. That’s how these instance is referenced.Distanceis shrinking. Do not look at paths where theDistanceis increasing. That’s not relevant and will only confuse you.fwiw posting memory stacks is not going to solve this issue, we need full repositories including all customizations you’ve done in order to be able to even determine if this is caused by Next.js in your particular application or if it’s caused by any of your other dependencies (e.g. you might have customized the webpack configuration). If you’re not able to provide these please just use GitHub reactions on the initial issue (add a 👍) as the memory stacktrace is not helpful in tracking down what the issue is for your application.
Note that memory leaks can be introduced by your own code (e.g. if you render nested components in an infinite loop, if you create an object in the module scope and keep adding to it etc.) and by dependencies (e.g. React components from npm, libraries from npm etc.). Hence why I keep asking for full reproductions, as it’s practically impossible to track these down for your particular case otherwise.
Also note that I want to solve this if it is related to Next.js, we just need more pointers from applications that experience this. We run a Next.js application with hundreds of pages and thousands of components with many developers and haven’t experienced OOMs both during dev/build. The cases that we have investigated before mostly lead back to a certain dependency being imported that had a memory leak, this is also the reason why I brought up enterprise support (feel free to email enterprise@vercel.com) as a means of investigating this particular issue in your application, given I can’t justify investigating days/weeks on an issue with no reproduction provided to date and that hasn’t been reproducible to start investigating.
I hope you all understand and I’m looking forward to seeing a reproduction provided that can be investigated.
I can also confirm that this is happening. It seems like it may be related to page navigation somehow, as, when trying to come up with reproducible steps, we noticed that if we stayed on the same page and tried a series of change+save loops to get the hot reloading to fire quickly, it wouldn’t crash. But, if we tried navigating through our app for a bit, and then doing a few change+save loops we would quickly get to a point where it crashed.
A few properties of the codebase I’m working on:
babel.config.jsfile that we use so we can properly SSR the styles (mentioning this because of what was reported https://github.com/vercel/next.js/issues/7929#issuecomment-707222434)As far as I know, no one on our team had this issue on NextJS v9.3.6.
I’ve been getting this error with 9.5 as my site has grown in size. It happens whenever I fast refresh and have been in dev mode for over 15 minutes. I could share my repo privately if it’s helpful @timneutkens
Update, fixed
I refactored my entire app to make sure that there was absolutely no circular importing. I also changed my
package.jsonscript to run with this:Now it’s working just fine. Not sure what did it exactly, but I have a feeling it was a circular imports. So I recommend making sure you don’t have circular imports in your app.
For context, I’m on
next@9.5.3. I’m also using Expo + Next.js, and I upgraded to Expo SDK39 from 38 and react-native-web 0.14. Unsure if that had anything to do with it or not. 🤷🏼♂️Only thing that is annoying is here are these 0 value comments. Stop the spamming, please!
(Sorry for bumping the thread myself)
I no longer are affected by this problem, but its hard to tell what ultimatly fixed it. We are now on nextjs 11, but already in some version of nextjs 10, the issue was resolved
I observed that once the memory crash in development did no longer ocure, i had instead some fash refresh hicups, where nextjs would force reload the whole page. I cannot tell whether this is related though.
Its important to state that this issue here was never about memory leaks in production. I think this is a completly different problem that might have different reasons.
for those who are affected by memory leaks in production: open up a different issue, please!
Despite the activity here I would suggest to close this issue here. Reason is that a lot has changed since version 9.5.1. and information might be missleading here. Also i am no longer affected, so the initial issue is resolve for me and the information is no longer vaid.
Here’s what (I believe) solved this for me.
1. Circular imports
yarn add -D madgeIf any circular imports come up, fix them. Make sure that you isolate variables into their own files if needed.
2. Re-exports
My other issue was I was doing this in
packagesExample:
packages/api/index.tsInstead, I now import directly:
I’m using a (massive) monorepo and next-transpile-modules, and 10.1.4-canary.2.
3. Memory leak listeners
My final problem was that I set up a listener in
pages/_app.jsoutside of render code. I was listening toRouter.events. I fixed this by moving this touseEffect.Be very cautious to not include any listeners outside of an effect. And make sure you clean them up.
No, multiple people in the thread (including me) are not using Tailwind.
We’re also seeing the exact same out of memory crashes both in local development and in production.
Our cluster of Next.js containers keeps kicking over randomly under load with these out of memory errors.
Typescript/Styled-Components + custom server. No circular dependencies in our codebase.
I’m starting to consider adding something like nodemon to restart the process faster than our container orchestrator, but that’s a bit of a hack, no?
So, https://github.com/vercel/next.js/issues/12557 is probably relevant to this issue.
I ended up building a codemod for our codebase using
ts-morphto refactor away all of our barrels. The end result was an improvement to ourjestruntimes (improved by 200s), our NextJS build times (20-30s improvements), a reduction in the number of circular dependencies (269 to 127), improved bundle chunking and an overall reduction in First Load JS (about 1MB to 840KB or so) for most pages controlled by our codebase.I’ll try checking if I can release the codemod to the general public.
I would recommend getting rid of barrel files if all of the consumers of your code are within the same codebase. If there are consumers of your code outside of your codebase, and you really want those barrel files, have something to generate the barrels as a build artifact to help you avoid the creation of unintended circular dependencies, among other issues, while also providing a nice public api for your code.
Just want to add my experience with this and spill some of my toughts. As the others said here. I’m also experiencing this issue in dev since upgrading to 9.5 and in other project to latest 10.x. . It often crashes during some CSS work or just when we navigate throughout the project (maybe some hot-reload cache accumulating). On 9.5 I use SASS with next-sass lib and on 10.x styled components. I just keep my fingers crossed it won’t happen in production 😄 , so far it hasn’t but you never know.
To be honest I don’t feel confident that the leaks can’t be somewhere in my code. I tried debugging snapshots but it’s just too much information for me and I would say that orienting in this kind of deep debugging is for real senior hot-shots. (I’ve been working with react since 2017 and can’t rly understand much).
From my point of view trying to “fix” it with giving access to more RAM is just pointless, THAS IS JUST NO FIX AT ALL and you just keep increasing the cost of your app.
I totally understand that guys from Vercel are in tough spot here, since helping with issue that you hardly can reproduce is almost impossible. But we can all agree that something has changed guys, too many devs have this problem and it shouldn’t be overlooked.
Btw. Thanks for your work nevertheless, best framework out here still! 👍
This is not an issue with Next.js, rather an issue with how Tailwind handles class compilation.
To solve this issue, use the new
@tailwindcss/jitjust in time compilation module. You can install this via. NPM or Yarn,and then by adding this to your
postcss.config.jsfile:@timneutkens, this issue can be marked as resolved (closed).
I would be willing to share our apps private source code with a Next.js core-member to help get to the bottom of this.
@mankybansal There are plenty of articles out there that tell you how to use Node debugger’s memory snapshots to track down memory leaks.
Between 9.4 and 9.5, there were changes in how Next.js handled Webpack hot middleware + some other fast refresh stuff so that’s probably where your error is coming from. I’ve spent countless hours tracking down changes between 9.4 and 9.5 both in what changed in Webpack, and what changed in Next.js, and sadly I just don’t think I know enough about bundlers/hash chunks (chunk hashes?? it’s been a few weeks since I looked 😅 )/Source maps and source nodes/all this stuff I’m encountering to really make sense of it.
But it does feel like it’s to do with Webpack + Next.js not properly cleaning up old chunk hashes/sources and essentially leaving these orphan chunks to accumulate in memory. All the error messages and heap snapshots we’ve posted seem to point to that (anything to do with
emitorEventEmittersgenerally comes from Webpack code that comes fromwatchpack/DirectoryWatcher/etc spawning a new watcher for a chunk/source that might not have gotten garbage collected, so it stays open… or something like that).What I find particularly intriguing though is that - for Open Collective at least - this memory leak issue is not present in prod, only on the Next.js dev server, which is where I’ve focused my investigations. Maybe I’ll try to condense all the notes I’ve taken and post them and hopefully someone with more knowledge about bundler internals can use that as a springboard.
I find this interesting because I felt like the times I was having the server crash the most in development is when I was working on CSS (OC doesn’t use Tailwind, but does use the
styled-packages) rather than other types of frontend work. Perhaps something to do with the way Webpack hashes CSS and how that interacts with the Next.js updates? 🤔I’ve been getting this issue consistently after 1-2 hot reloads after upgrading to 9.5. It makes the dev experience slow (constantly restarting) and barely usable. The OOM occurred on 9.4 but much less frequently (once an hour) Has anyone been able to improve / resolve this issue?
@timneutkens it looks like we have a few repros, I can also throw mine into the mix but would need to be under an NDA.
@thereactguys deleted your spam.
I’ve noticed this issue happen 4 -5 times in the past month. It has happened when switching branches, but it also just happened when I updated my JSX to have an inline style. I tend to run
yarn devfor multiple days on end.We are using Next with Material-UI CSS-in-JS.
Next 10.0.7
Circular dependency detected: node_modules/next/dist/client/with-router.js -> node_modules/next/dist/client/router.js -> node_modules/next/dist/client/with-router.jsThis issue is to do with memory use while hot-reloading is enabled. If neither of those are relevant to your situation (I assume you aren’t load testing with hot-reloading enabled…?), maybe you should create a new issue. Have I misunderstood what you’re saying?
@sambecker Interesting - we honestly started seeing this after upgrading from Node 10 to 13. Upgrading to v14 is on the todo list and I’m really crossing my fingers that the issue just goes away when we do.
FWIW @timneutkens I appreciate your patience with this and the fact that you’re not closings this outright. Getting bug reports like this must be frustrating, and I’m sure I’m not the only one when I say I’d love to share our source code with you but are legally unable to. I’m circling up with my team about potentially reaching out for enterprise support, but even that is less than ideal as the red tape to get that NDA signed will probably take us months.
My team is doing a lot of investigation into this right now and I’ll be sure to share anything we find. The only smoking gun we’ve found is that a lot of the out of memory crash stacktraces point back to the
etagheader generation code that hashes the response before sending it. I don’t think the actual issue is that code, but it seems to be indicative of Node poorly managing memory. If anything, this is quite possibly a Node bug.I don’t know if this is useful, but I found that some React components were loading the
Linkelement with bad urls. The errors weren’t really exposed until I removed most of my pages from generation. There were some other issues but this was a big one. It might be good to have Next.js report more of it’s lifecycle data so it’s easier to pinpoint where things are happening, or some way to die on errors so they can be located.Anyways, seems one thing I learned was to cut down your generated pages to as few as possible and add them back in until you see errors so you see them.
Update 1: We have a big json file that’s a mapping of zip codes to coordinates. This was
required in one of our API files. I changed it to a.jsfile and imported itimport { zipMap } from 'lib/zipToCoordinates.js'and that resolved one more OOM error that had zero context.I have improved this situation a lot (as in I need to test more to consider it fixed, but it stopped crashing) by replacing tailwindcss with @tailwindcss/jit, I guess it is leaking memory in CSS processing and Tailwind does pull in a huge amount of CSS. If that’s the case, it’s probably not a Next.js problem but in some dependency.
May not be a global solution since people here didn’t report using Tailwind
We experienced the same problem: started after upgrading to 9.5 and never stopped.
We don’t use a custom server (we have no API code in the Next project, it’s just frontend) and we also make no use of Tailwind, so whatever the problem is Tailwind is not the cause.
Our library stack is Typescript, Material-UI and heavy CSS module usage. However, the problem started before we adopted typescript.
It got to the point where the server restarts every few minutes and regularly uses 4GB of RAM.
Maybe Material-UI is also affected? Unfortunately I can’t open up the codebase for this project.
@mmmeff Upgraded to Node 14 last week and we’re still experiencing the same problem so I wouldn’t count on that making a difference, unfortunately.
In our case it started happening when upgrading from Next 9.4.2 to 9.5.2 (without touching the Node version) so I’m not sure if it makes sense that it’s a Node bug either.
I seem to get this crash after I change some CSS in the same file as tailwind imports, I am also using typescript.
I am currently trying to create a test repo.Repo based on blog-starter-typescript but slightly modified https://github.com/lclc98/NextOOMI’ve been having the same issue for about 1-2 weeks as well. After starting up my local dev environment, I work for a bit and then end up having to restart due to a crash. I also get these
report.*.jsonfiles put into my root directory. This has happened most days.System information
(I can provide the full Node-generated report JSON if needed).
A coworker and I also noticed we were getting this error too:
When you say this do you mean a link to a PR with a live app on Vercel? We might be able to provide since Open Collective is open source and all our repos are open. (Edited to add: here’s the PR I’ve been working on where I experienced the latest crashes, you can pull the code from our repo and see the Vercel app link in the PR.)
I think this is a critical issue that strongly hinder adoption of Next framework in production environments.
I am a big fan of the framework, so please have a DEEP look into it, after almost 1 year without a fix this deserve a focused investigation and a definitive fix,
It is probably worth checking if you have fast refresh working properly.
https://nextjs.org/docs/basic-features/fast-refresh
We had a huge index file for our
@ecologi/react-componentspackageAfter changing the package.json
mainfield in our react components package to the following…Which was needed to get Next Transpile Modules to pick up changes in each component folder properly.
And also updating all the imports to have imports for each component.
We now have fast refresh for all the components which makes it possible to develop on our apps using
yarn devHowever there is still issues navigating around the site. After compiling more than 3 or 4 different pages in dev mode it becomes unbearably slow and will crash or need restarting.
The issue seems occur when there is any refresh of the browser window or navigation to another NextJS page in
yarn devmode.If you don’t have fast refresh enabled it will reload the browser window for most changes and this will cause it to crash all the time.
it is getting pretty annoying
react@17.0.1 next@10.0.9 emotion@11.1.5
ant.design@4.15.0 framer-motion@4.0.0 reduxjs/toolkit@1.5.0 prisma@2.20.1
I am still encountering this error in
10.1.x.Log:
My mistake. Nonetheless, the issue here isn’t in of itself NextJS. Alone, the framework is completely functional.
@ranisalt I experience the memory leak with the tw v2 only, works just fine with the v1.
We are also having this problem.
Next version:
next@9.5.5Reading this issue has been somewhat frustrating. I understand that it is difficult to diagnose these issues without a reproduction, but consider that most of us simply won’t be able to hand over our closed source repos. If it was easy for us to create a minimal reproduction, I’m sure someone would have done so, but this also is not easy.
I also understand that since this is likely due to some third party, it must be frustrating being lumped with the problem. I’m not really sure what else we can do in this case except work together to find the culprit.
Other than requests for reproductions, it hasn’t been stated what we can do to help you! I’m sure many of us would gladly give more information if we knew what you might find useful.
Here are some suggestions:
package.json- this would help narrow down possible dependency culpritsOne piece of information I haven’t seen here (I might have missed it) is that looking at heap snapshots makes me think that it is related to this problem in some way: https://nextjs.org/blog/webpack-memory
While the changes noted in that post seem to have fixed the memory problems when building, I see similar symptoms: huge amounts of memory attributed to
CachedSource. Now, I’m a complete novice at reading heap snapshots, so take that with some scepticism!I tried setting
output.futureEmitAssetsin our config, but it had no effect. I don’t understand what this does enough to predict what should happen though.I’m going to echo what I think I’ve seen others say, which is that I’m happy to talk privately and show you what’s happening with our setup. I understand this might not be ideal for you, but it’s probably the best compromise I can offer.
EDIT: I can also offer some small bits of analysis about memory and file sizes.
Looking at the heap snapshot, there seems to be 47MB of memory taken up by
CachedSource(from thewebpack-sourcesmodule) forstatic/chunks/pages/_app.jsalone. 35MB forstatic/chunks/pages/[region]/places/[id].js, and decreasing amounts for other pages. Again, my interpretation of the snapshot information might be off, but that’s how it appears to me.The size of the files on disk in the built
server/pagesdirectory seems to be around 900kB each, with_app.jsbeing 1.7MB.I don’t know what numbers are expected here or even if they’re helpful, but there they are!
@TheReactGuys please stop spamming. I’ve reported the account to GitHub.
@malimccalla I also have a similar setup to you, and have been experiencing the same Out of Memory errors as have been previously described. I’ll see if removing these from my app mitigates the errors in any way. Thanks for the lead!
Tried with next
11.1.2and node16.9.0and have exactly the same problem:This issue is still actual!
I did, but i was worried that if i would let some script do the works I would end up with many errors and loose my head around. By doing it myself I was sure every import is correct, of course human error is a biggie here. But thankfully I am kinda good at this “copy-paste-robot like” things 😆. Anyway, there must be for sure a way to script this process quite easily.
Also like others said, thanks to this there are improvements in build size, build speed and jest runtimes.
Ok so for me the issue was it seem this:
I have folder named components, where i have a lot of small components that app re-uses. In that folder i had index.js file, which exported all those components, so i could import them in pages/views like this:
import {ComponentA, ComponentB, ComponentC} from './components'I removed index.js and changed all the imports, to direct imports e.g. :
import ComponentA from './components/ComponentA'After i did this in whole project (maaan was this a painfull job for many many hours 🤮 😭 ) it seems that the problem with memory leak no longer occurs (haven’t got a single error in a month).
Hope that someone has the problem as I did and it will help him 😃. Happy coding!
I exactly had the same problems, after fixing all of them my projects works just fine again, many thanks!
Those tracebacks don’t help to solve the issue. Vercel guys have said pretty clearly that don’t help: https://github.com/vercel/next.js/issues/15855#issuecomment-760187734 and that they need reproducible examples: https://github.com/vercel/next.js/issues/15855#issuecomment-668478456
Why is it spamming? He provided a stack trace (different than the others, mind you) and the list of packages and versions involved. If you don’t want to get notified, ignore the issue or fix it.
We are also having this issue with 10+ Next and using styled components.
I have the same problem. The project has 14 static pages + x dynamic pages + roughly a total of 100 components (small to large scale).
Base FE stacks
The log:
I have been facing this issue since last year and still see it. I tried other suggestions above; like ensuring no circular importing and play with
NODE_OPTIONS. but no luck. I believed it was because the WebGL component is very heavy.It stops happening when removing the
.nextbuild directory and restart the local server.I am also afraid of sharing the project application or the repository due to the project is private and confidential. I understand that it is just really tricky to give insight into this problem without looking at the actual project application.
Does anyone have any “feeling” or assumption of why refreshing the
.nextbuild directory folder remedies this problem immediately?CSS processing does seem to be a common theme. Thanks for the information.
@timneutkens I had also spotted/reported this circular dep in my attempts to debug this. Is this of any relevance to the out of memory dev server bug?
it’s reproducible by adding https://www.npmjs.com/package/next-circular-dependency to a NextJS app
fyi, this is still an issue as of
next 10.0.7, I’ve experienced it with two projects with very different setups(one is using TS + apollo client, the other is plain JS + Redux). Both have been having severe issues with their dev servers, making working with nextJS much more difficult than it should be.Obviously, i can’t post reproductions on this, since these are private projects.
@mmmeff, can you give me some insights on how you narrowed it down to
graphql-tagfor your app?Really interesting! Certainly looks like a likely culprit. Both for the errors I saw, and the introduction of these memory issues. Similarly, I know little about Webpack/bundlers, so investigating is hard. 😞
We’re using
styled-componentsso that fits.Does anyone have a good feel for how much work it would be to side-by-side compare projects with/without some CSS-heavy library?
I’m really unfamiliar with Next and the associated technologies, so I find the idea a bit daunting 😅
I just tried building Next.js locally and linking into our app as described here: https://github.com/vercel/next.js/blob/canary/contributing.md
I wanted to see if there was any truth to the idea that Next.js is using more memory in 9.5 vs 9.4, as per the OP:
However, I just get this error, and the server never seems to start:
Since that error doesn’t reference any of our own code, I don’t know where to start debugging, so I’m giving up for now.
Looking at memory use in htop though, I don’t see much difference, so I’m not convinced there’s been a big change in Next.js here. It’s possible that those of us who have started seeing this issue more often have simply increased the size of our apps: more dependencies, more pages, etc.
I have added Next Circular Dependency to my
next.config.jshttps://www.npmjs.com/package/next-circular-dependencyThis library uses https://www.npmjs.com/package/circular-dependency-plugin under the hood
I fixed any circular dependencies in my application code. The following remaining few circular dependencies highlighted in both NextJS and Shopify Polaris. Our team is working on replacing Shopify Polaris. I would highly recommend the Circular Dependency plugin to help determine if you have any memory leaks in your code.
My
next.config.jsis the following…I’ve tried to minimise the amount of webpack customisation in dev mode as shown in the above example, Disabling the headers and redirects features. However I am still seeing this issue with the dev server crashing frequently. It does seem to be related to navigation and switching pages in dev mode and also for example switch from one branch to another. It seems like these are both things which put a strain on NextJS and webpack.
These are the main libraries and webpack customisations I’m using in dev mode as follows…
I also have a custom babel config as follows
I’ve tried NextJS version 10.0.6 but the dev server also crashes there. I’ve also tried Node LTS which didn’t fix the issue. I’m currently using
v12.18.4Adding the circular dependency plugin and fixing those issues seemed to help a little. Is there any insight on the circular dependency reported in Next JS?
I can’t share the source code publicly however I would be happy to share it privately if there was any potential for some advice on fixing this issue. We are paying for the a Vercel hosting plan for all our websites and this dev server problem is really holding us back.
It would be great if there was first class support for TypeScript mono repos as this is the main custom webpack I am relying on in dev mode. I would love to remove all my webpack customisation, however it is not practical at this point.
Any advice on this would be greatly appreciated.
I had a massive memory leak and performance issue, which occurred with both the dev server and the production build. Turned out the reason was that I had set
target: 'serverless'in mynext.config.js, even though I was using a standard containerized “serverful” deployment.I still have some memory leak issues when running the the dev server, but it doesn’t ever crash anymore since I’ve set 4GB max heap size for Node. With the
'serverless'target the dev server would crash even with 8GB of heap memory after loading around 10-20 pages.I get this virtually every time after less than 10 reloads on Next 10
FWIW haven’t seen this happen since upgrading my system
Node.jsfrom13to14. Will keep an eye on it.I’m also getting this issue and I have customised my directory structure to look something like the following… Thought i’d share incase this may be a common theme amongst occurrences.
Each file under the
pagesdirectory exports the index from the correspondingviewsdirectory. For example,pages/a.tsxjust contains.I use this structure as I like to locate all related code under one directory (also see https://github.com/vercel/next.js/issues/4789#issuecomment-476758132). I’m not sure if this is even related to the OOM issue but felt it like it might be.
i tried now setting
NODE_OPTIONS='--max_old_space_size=4096'and it seems to be better at first glanceUnlikely we’ll be able to track it down based on just a next.config.js.
As said before if the only concern is that the code is sensitive we can look at enterprise support which would include a NDA.