next.js: App router deduping not working as expected

Isn’t this a massive issue? I’m kinda confused how everyone is experiencing this and nobody seems to be talking about the fact that deduping doesn’t really work…

Yes, I think the docs are very clear, it’s not meant to de-dup across different URLs.

The problem is that it doesn’t actually work. Posting another repo with 14.1.4, the problem is still there: https://codesandbox.io/p/devbox/next-double-fetch-mcfppg

This is still happening on 13.4.16 btw, and is causing large performance and costs for people not aware it is happening.

@zwarunek We got around this issue by manually wrapping every fetch with the cache function documentation.

The docs say requests are memoized only when the URL and options are identical.

I cannot reproduce this issue anymore with next 14.1.0 and my example project. Would be great if someone else wants to try, otherwise i will close this in the future. 😃

@OlegLustenko

Next.js extends the native fetch Web API to allow you to configure the caching and revalidating behavior for each fetch request on the server. React extends fetch to automatically memoize fetch requests while rendering a React component tree.

Docks👆

Docs says that dedupe works only if you use native fetch which Next.js extends. So yeah I’m personally experiencing things with pure fetch and didn’t expect the same from libraries. And if you want to get this with third-party libraries it is recommended to use cache function.

Having the same issue. Did anyone found workaround or something? From what I’m seeing wrapping fetch in cache may be the possible workaround… After a lot testing I no longer understand when it breaks and how it works. Somehow my project works fine on Next.js 13.5.4 and below, and after uploading to Vercel it stays the same, but all versions above just don’t dedupe fetch requests. Most frustrating thing that when I do all the same on codesandbox it just didn’t work whatever version I choose request are not getting deduped, is so unpredicted and buggy. Because of that I can’t upgrade to Next 14.

I’m surprised how little people care about it, there is only few issues and they don’t have any responses. How people even use Next then. It completely breaks app router paradigm with many small server component where you can fetch your data and don’t worry about duping.

In my case I have strict rate limit which I can’t break and with this bug it just break all completely. It should be fixed ASAP.

I reproduces this issue on CodeSandbox for anybody who don’t want to clone git repository to test this issue.

Link to the code that reproduces this issue or a replay of the bug

https://codesandbox.io/p/sandbox/gallant-tree-4yxvwz

To Reproduce

1. Refresh the page
2. In console you will see two timestamp with different time

Expected Behavior

1. Refresh the page
2. In console you will see two timestamp with the same time because second request was deduped and they share only one response.

You can wrap the API requests with React.cache instead.

https://react.dev/reference/react/cache

const getData = cache(async () => {
  const res = await fetch("https://worldtimeapi.org/api/timezone/UTC", {
    credentials: "include",
  });
  return res.json();
});

I tried this on your codesandbox and it returns the same timestamp twice, and only sends the API request once, as you expected.

https://github.com/vercel/next.js/blob/218d0709eb97017e5bbd55f603bce033206bf71e/packages/next/src/server/lib/patch-fetch.ts#L501-L504

Hi @ztanner ! (author of incremental-cache at #53030) I’m trying to solve this issue with @rjsdnql123 . We have question about lock() on incremental-cache!

On patch-fetch.ts#L502, request try to lock incremental-cache first. When incremental-cache is empty and multiple requests with same url(cacheKey) comes in, these requests can get lock incremental-cache at the same time?

https://github.com/vercel/next.js/blob/218d0709eb97017e5bbd55f603bce033206bf71e/packages/next/src/server/lib/incremental-cache/index.ts#L241-L252

On above code, I think createLockIfAbsent logic is not synchronized over multiple requests so maybe multiple requests can create lock at the same time? 🤔

Has this been resolved? I tried to reproduce the issue, but it appears that the server request is only occurring once in reality.