next.js: next js 13 server error " TypeError: fetch failed "

A reproduction that works consistently for me:

Repo feature/steer-integration

pnpm i
export NEXT_PUBLIC_POOLS_API_V0_BASE_URL=https://pools-git-feature-steer-integration.sushi.com
pnpm exec turbo run dev --filter=evm

go to http://localhost:3000/pool/137:0x21988c9cfd08db3b5793c2c6782271dc94749251/smart

open the file sushiswap/apps/evm/ui/pool/Steer/SteerLiquidityDistributionWidget/SteerLiquidityInRangeChip.tsx

change Range on line 47 to Rangea and back to Range a couple times, it happens faster one time and slower the other

Might have something to do with a client component under a server component? Wild guess.

Noticed this line in the log a couple seconds before it started erroring out (both times I tried this repro): evm:dev: - warn The server is running out of memory, restarting to free up memory. I’ve got 16GB of memory.

Finally, the error itself:

evm:dev: TypeError: fetch failed
evm:dev:     at Object.fetch (node:internal/deps/undici/undici:11576:11)
evm:dev:     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
evm:dev:     at async invokeRequest (/home/lufy/sushi/sushiswap/node_modules/.pnpm/next@13.4.19_biqbaboplfbrettd7655fr4n2y/node_modules/next/dist/server/lib/server-ipc/invoke-request.js:17:12)
evm:dev:     at async invokeRender (/home/lufy/sushi/sushiswap/node_modules/.pnpm/next@13.4.19_biqbaboplfbrettd7655fr4n2y/node_modules/next/dist/server/lib/router-server.js:254:29)
evm:dev:     at async handleRequest (/home/lufy/sushi/sushiswap/node_modules/.pnpm/next@13.4.19_biqbaboplfbrettd7655fr4n2y/node_modules/next/dist/server/lib/router-server.js:447:24)
evm:dev:     at async requestHandler (/home/lufy/sushi/sushiswap/node_modules/.pnpm/next@13.4.19_biqbaboplfbrettd7655fr4n2y/node_modules/next/dist/server/lib/router-server.js:464:13)
evm:dev:     at async Server.<anonymous> (/home/lufy/sushi/sushiswap/node_modules/.pnpm/next@13.4.19_biqbaboplfbrettd7655fr4n2y/node_modules/next/dist/server/lib/start-server.js:117:13) {
evm:dev:   cause: Error: connect ECONNREFUSED 127.0.0.1:41375
evm:dev:       at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1495:16) {
evm:dev:     errno: -111,
evm:dev:     code: 'ECONNREFUSED',
evm:dev:     syscall: 'connect',
evm:dev:     address: '127.0.0.1',
evm:dev:     port: 41375
evm:dev:   }
evm:dev: }

For anyone who is getting this issue, the workaround is to use next@13.4.12 . Another workaround and why this is happening is because fetch url protocol converted to https on production with localhost. You can reproduce this in dev env using cloudflared:

• Run pnpm dev one tab
• Run `cloudflared tunnel --url “<localhost-url>”
• Add log for req.nextUrl.origin somewhere in middleware.ts file.
• Browse cloudflared tunnel url and check the log, you’ll see https://localhost:... which is the cause of this error if you’re using req.nextUrl anywhere in server fetch. So to solve this either next@13.4.12 or use http://${req.nextUrl.host} as temporary workaround.

Edit: With vercel hosting, it’s working fine though.

@timneutkens You’re evaluation on bullet point 3 I think is just wrong. There are numerous comments here and in the previous thread describing this issue, and it isn’t coming from our code. I repeat. We never see these fetch failed while we hit our own pages… they appear in the logs later, often after the process has been up for hours. In our specific example I have tried load testing the application for an hour handling hundreds of thousands of requests and there are 0 fetch failed. Yet, our production logs are absolutely full of them, when I’m hitting the same exact URL.

There is some sort of internal mechanic going on which is triggering the fetch failed. It aligns with what many of the commentators here are bringing up, and unfortunately I have zero confidence the Next team even thinks this is an issue.

Adding a link to a Git Project which can replicate the same issue, I tested it by using Node 16 and 18, Git Project, It may require some test supabase credentials to work but does the job to replicate the issue. Error:

"TypeError: fetch failed\n    at Object.fetch (node:internal/deps/undici/undici:11576:11)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"

And another medium link where a similar issue has been explained and “tried” to fix https://medium.com/@kaloyan_17221/fix-vercel-next-js-fetch-failed-from-undici-polyfill-8c66346c9c2f

Just read through every post. They’re all about different issues and most are not caused by Next.js but by application code.

As a first step upgrade to the latest stable version of Next.js, for example what https://github.com/vercel/next.js/issues/54961#issuecomment-1709991281 and other shared has already been fixed. Though unable to reproduce we were able to just remove that code path altogether so that specifically can’t happen anymore.

TLDR: Next.js no longer uses fetch in the hot path while rendering, whereas in 13.4 it did, so this error can’t happen in the normal rendering path.

For others I’m seeing a bunch of different issues being confused:

• next build prerender errors out with fetch failed, usually this means:
  • You’re trying to fetch an endpoint that doesn’t exist at build time (e.g. pages/api or route handlers), for this case call the code directly instead, it’s all in the same Next.js application.
  • Your build system doesn’t have access to the API (i.e. there’s an IP block or something similar)
  • Your build system doesn’t have the right API Keys set up (i.e. you forgot to add an environment variable)
• Fetch failed at runtime during rendering
  • I.e. when you fetch() in a server component you need to ensure that the url you’re fetching is available
  • Your runtime system doesn’t have access to the API (i.e. there’s an IP block or something similar)
  • Your runtime system doesn’t have the right API Keys set up (i.e. you forgot to add an environment variable)
• Fetch failed in Next.js
  • There is a case where Next.js internals fetch(), in particular what was mentioned above wrt the image optimizer, however the error shown above is caused by an image url being provided that fails to fetch, e.g. because you’re offline, it’s unreachable and such.

Hope that provides enough context!

This is not a new issue. This has have been happening since roughly 13.2.x However it seems to have gotten worse since 13.4.19. For me on windows, i’m getting now 1 crash per day on my development environment. In the past the next server would restart which was nice, but now i have to manually restart. Also it seems to be less of a problem on mac. If you search there might me some tips in similar topics. In general development on windows with Next 13 is far from optimal

Updated from 13.4.10 to 13.4.19 and now get this error

@timneutkens In the end, if all it takes is to downgrade Next.js to 13.4 (down from 13.5), you can’t reasonably assume the problem is in application code. In such case, it doesn’t matter if the url fetch is trying to access, is blocked or otherwise unavailable. It’s demonstrably Next.js just spewing out an error that it didn’t before, and is not useful.

Isn’t not application code or something getting blocked or whatever else, it is purely Next.js that has put extra unneccesary checks in place for the certificate. And frankly, on a development environment, I don’t give 2 tosses about the correctness of certificates. If I could work without https, I would, but when I can’t, I just don’t care about certificates. I’m on a development machine, and I just need to do my job instead of messing around with certificates that have always worked perfectly fine. I don’t think that’s all that unreasonable, that a developer just wants to get his/her work done, and not have to deal with certificate bollocks. Therefor it’s super unproductive to restrict certificate requirements further than they already were.

That is of course, for my version of this problem. Other versions may enjoy different argumentation.

@GoudekettingRM Downgrade to 13.5.2 => It should be fine. 13.5.3 introduced a new bug with fetch - I think.

Same prob with v13.5.1, going back to 13.4.12 😦

Hi,

Just wanted to chime in to say we’re experiencing the same symptoms in our application. We recently upgraded next.js from 13.4.7 to 13.4.19. Node version 18.7.1.

We see steady increase in memory usage culminating in a The server is running out of memory, restarting to free up memory. log message. After that the app starts returning an internal server error and logging out the same TypeError: fetch failed stack trace as above over and over. Requires a restart to get the app running again.

I’m going to try rolling back next.js and let it run for a day or two.

Confirming that this has been driving me nuts lately.

really frustrating building in NextJs lately.

This problem happens with next start on the production. Ubuntu 20.20 Node.js v18.17.1 It happens randomly when our server is busy, once it happens next.js server can’t continue to work => We have to restart the Next.js server.

Fastify servers work fine, but it just happens with Next.js servers.

P/S Downgrade to "next": "13.4.12" works fine for me.

I updated next.js to 13.4.20-canary.33, and it looks like the problem was fixed.

@karlhorky I’d say that the fix for that issue and some PRs after that screwed up Next.js’ IPv6 handling even further, so yeah. Got fixed real good after that though, should work properly starting from 13.5 😃 I fixed that IPv6 handling, but there were some issues after I got that in (13.4.16). They got rid of the render workers some time after that, which fixed those issues as well (13.4.20-canary.24).

@timneutkens hello. Thank you for your reply. I don’t know other users’ codes or when the error occurs, but the above error occurs irregularly for us( Occurs when the server is running for more than a day). This happens just by accessing the K8s liveness probe at a time when no one is accessing the site.

Also, this error occurs starting from a certain version. Some people say that the error does not occur from version v13.4.9 or lower, but most people say it does not occur version v13.4.12 or lower (the error occurs starting with v13.4.13).

If it occurred starting from a specific version of the same code, it would be difficult to say for sure that it was a problem with the users’ application code.

When looking at the differences between versions as shown in the link below, it appears that undici has been applied starting from v13.4.13. (Issue link https://github.com/vercel/next.js/compare/v13.4.12...v13.4.13#diff-abd1f3c858290361038c2f039d2ebdd031900f46daec6c4a613450dd8fa21dc3)

Also, it appears that a similar issue has been open in undici for quite a long time ago. (issue link https://github.com/nodejs/undici/issues/1602)

From the user’s perspective, there is no way to prepare for the sudden occurrence of an error when the patch version is updated from the semantic version. If backward compatibility is not properly supported, it appears that at least a change is needed starting from the minor version.

We are sorry that we have difficulty providing you with our product code. Probably, it would be difficult for most companies to provide service codes.

Thanks for the expansive answer @timneutkens.

Not quite sure about the wrt optimizer though. If I skip it, the images load without error - i.e. the images exist and are reachable. As soon as I use the default optimizer it errors with ENETUNREACH/UND_ERR_CONNECT_TIMEOUT. I understand that that is a message that indicates the optimizer cannot get the image, but seeing that the image is reachable and present, I’m confused as to why not and I’m not convinced it’s purely application code error. I’ve tried a few times and I see that sometimes - after like 5 refreshes - it does load, but not always. If I have some time later today I’ll see if I can replicate it in a new project.

Same problem here.

It’s a production build, running on windows using a corporate proxy.

To setup the proxy we were setting up HTTPS_PROXY env var

As it was not enough, ignore in some internal requests, we use node --require to preload some code.

In this case globalAgent.

It worked ok, until undici got into the scene and needed to use the same preloaded code to setup the proxyagent on the global undici object.

And now, since 13.4.13, we get the ECONN_REFUSED on the image optimizer requests. Just downgrading after 13.4.12, everything works ok.

So it looks like NextJs, has problems with corporate proxies as we always have to look for workarounds

PS: all custom code requests works, just internal requests done by NextJS, fail. So probably our scenario is not on the list by @timneutkens

For all the people who have been reporting that it happens with Next.js v13.4.19, please upgrade to the latest canary on the Releases page (currently 13.4.20-canary.23) and try again - this will be helpful to see if the problem still occurs

(I am also experiencing this issue on the latest canary personally, but would be good to get some other data points)

@balazsorban44 I wish I could provide more details, I basically found it difficult to work in dev mode. I had upgraded the next version from 13.4.10 to 13.4.19, and would consistently get the above error, and my dev environment became unbearably slow.

I downgraded to 13.4.13 and it’s been stable since.

I might test out the canary version once I roll my current implementation out.

hopefully, someone else may have provided more context before then.

@balazsorban44 I don’t have a codebase to prove that it is indeed “getting worse”. It’s just my guess.

I’m working on a project with NextJs v13.2.4. We were trying to upgrade it to v13.4.x. And then in the production environment, the “TypeError: fetch failed” showed up so often (~1 per minute) that we had to roll it back.

Then after a bit searching, I found these issues: #51605 #53353 , thinking that these issues are somehow related to our problem.

In our case, the error codes differ (“ECONNREFUSED” and “ECONNRESET”). They take place after calling on our NextJs API route.

I’m not sure if it’s a (upstream) bug or not. It has something to do with load. If I call API route at high rate (with curl for example), the “TypeError: fetch failed” logs also appear with NextJs v13.2.4. But in production we haven’t seen it so far as often as with v13.4.x.

That’s why I’m also curious about the performance of NextJs API routes. Are there guidelines hereof? Because of these errors, we aren’t able to upgrade NextJs and leverage the app dir.

I can’t reproduce this issue with the latest Next.js canary

Nice, removing the hostname configuration wasn’t working back when https://github.com/vercel/next.js/issues/49677 was resolved, but I can confirm that removing hostname or configuring hostname: 'localhost' is now working with our apps (on next@13.5.4) 🙌

I’ll keep an eye out for any intermittent fetch crashes in the next days too (related to this current issue), just in case.

I noticed the same issue as @pealmeid , I have something in place on my system that sets NODE_EXTRA_CA_CERTS. I need it to be able to use some local https service.

The cause of this issue is this PR: https://github.com/vercel/next.js/pull/55775. There you can see that process.env.NODE_EXTRA_CA_CERTS gets set. I think that should not be done.

A simple reproduction is to put an export in your login shell that sets NODE_EXTRA_CA_CERTS. If you use zsh you could add to your ~/.zhsrc a line like:

export NODE_EXTRA_CA_CERTS=/path/to/a/ca/you/want/to/load

If you put a console.log in next.config.js you can see process.env.process.env.NODE_EXTRA_CA_CERTS is empty. If you hack into your node_modules and disable the line that sets process.env.NODE_EXTRA_CA_CERTS it works again as expected.

I don’t think it helps or solves the original problem of this issue, perhaps this should be a separate issue.

*edit

There is apparently an experimental way to do https with nextjs locally (--experimental-https), but I suppose that should not override process.env.NODE_EXTRA_CA_CERTS if nothing is passed to fill it with. Especially if you are not using the experimental flag (which I am not).

Still the same on 13.5.3.

Revert to 13.4.x works around it, but it had a severe bug pertaining to locales that was fixed only a few days ago, iirc.

So, it’s a matter of choose the lesser evil, at least for the moment 😟

I updated next.js to 13.4.20-canary.33, and it looks like the problem was fixed.

I upgraded to next@13.4.20-canary.35 and we have also not experienced the fetch errors / crashing since then… 👀 🤔

I will keep an eye on this, but maybe this has been fixed

Edit: 2 weeks later, and haven’t seen any fetch-related crashes

Note we saw this on 13.4.19 (when we upgraded from 13.3.4). Downgrading to 13.4.13 has the same issue for us actually. Currently trying 13.4.10.

Seems like it takes a few hours before it begins to spam that fetch error in the logs and return internal service…

• Edit - 13.4.10 also has same issue… Strange

@Elvincth that seems unrelated to this fetch error that this issue is about, but in your stack trace it’s showing Node.js v18.16.1 - there’s a memory leak before Node.js v18.17.1, so you should upgrade 😃

For all the people who have been reporting that it happens with Next.js v13.4.19, please upgrade to the latest canary on the Releases page (currently 13.4.20-canary.23) and try again - this will be helpful to see if the problem still occurs (I am also experiencing this issue on the latest canary personally, but would be good to get some other data points)

Hello @karlhorky , I have been facing the same issue with Next.js v13.4.19, (Node.js v18.16.0). it is happening frequently. It restarts in less than 5 minutes of starting the server. I am new to Next.js and How do I upgrade to the canary release?

See here for updating to canary release. Or you can directly replace the version of next e.g. "next": "13.4.20-canary.24", in your package.json file then rerun the command npm i to update.

I recently switched to version 13.4.20-canary.23 and encountered an unexpected type error that read “X TypeError [ERR_INVALID_STATE]: Invalid state: ReadableStream is already closed”. I’m unsure of what caused this error, as there is no backend portion to my next.js project.

This issue continues happened with the latest ^13.4.19 version with my m1 machine in development mode, it happens randomly.

This problem happens with next start on the production. Ubuntu 20.20 Node.js v18.17.1 It happens randomly when our server is busy, once it happens next.js server can’t continue to work => We have to restart the Next.js server.

Fastify servers work fine, but it just happens with Next.js servers.

P/S Downgrade to "next": "13.4.12" works fine for me.

Anyone saying “same issue”, or not posting helpful comments to debug, please note that adding a reproduction link helps us verify/investigate more than anything. Stating your Next.js version in itself or an out-of-context code snippet is not very helpful. 🙏 Please comment your reproduction repository instead, we would like to investigate!

I added the label please verify canary, because the linked reproduction is on 13.3.1 which closes in on being half a year old version. We’ve introduced many improvements since then. The reproduction is also pretty complex and is not clear from the reproduction steps how to recreate the issue. @AhmedShehata98 do you have some specific steps you could share?

@cosieLq undici is the de facto fetch implementation that powers Node.js 18+ versions. In those cases, we don’t polyfill at all, just rely on the underlying runtime, so in that case this would indicate an upstream bug. Do you have a codebase we could look at to verify what’s the issue that you are seeing “getting worse”?

For anyone, upgrading Node.js might be a good start to verify if it’s related or not. 👍