next.js: `clientRouterFilter` causes random links to fail due to duplicate `basePath`

Verify canary release

I verified that the issue exists in the latest Next.js canary release
I verified that the issue exists in the Next.js 13.3 release

Provide environment information

Operating System:
      Platform: darwin
      Arch: arm64
      Version: Darwin Kernel Version 22.3.0: Mon Jan 30 20:39:35 PST 2023; root:xnu-8792.81.3~2/RELEASE_ARM64_T8103
    Binaries:
      Node: 18.15.0
      npm: 9.5.0
      Yarn: N/A
      pnpm: N/A
    Relevant packages:
      next: 13.2.5-canary.16
      eslint-config-next: 13.2.4
      react: 18.2.0
      react-dom: 18.2.0

Which area(s) of Next.js are affected? (leave empty if unsure)

App directory (appDir: true), Routing (next/router, next/navigation, next/link)

Link to the code that reproduces this issue

None

To Reproduce

I have not found a reliable way to reproduce the issue. It seems to affect random links. It is reproducible in our actual production app. I am happy to test potential fixes, etc.

activate appDir in an existing application
have a basePath configured for that application
observe random links to fail loading

Describe the Bug

We have an existing application which was build with the pages router. Once we added appDir: true to the configuration we noticed that a single link in our application (/plan/configuration/activities) where we have the basePath configured to /plan/configuration) stopped working. Instead of navigating to the correct page, it would instead do a hard reload to the page /plan/configuration/plan/configuration/activities.

Through debugging, we were able to trace the issue to the clientRouterFilter flag. Setting that flag to false fixes the issue. Were noticed that https://github.com/vercel/next.js/blob/canary/packages/next/src/shared/lib/router/router.ts#L1098 was causing a hard navigation, due to matchesBflStatic being true. This part of the code also added the unnecessary additional basePath. It seems like the internal link is incorrectly classified as requiring hard navigation.

We tested using the Link component as well as using the router.push method directly. In both cases, the given URL was /activities.

Note that the link to /skills works fine. Only /activities is broken.

Expected Behavior

Activating appDir: true in an existing application does not cause issues with existing links.

Which browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

next dev

About this issue

Original URL
State: closed
Created a year ago
Reactions: 17
Comments: 25 (11 by maintainers)

Commits related to this issue

Update default error rate for client filter (#60542) This updates our default error rate to be much more precise and reduce false positives by increasing the default size of the client filter we ge... — committed to vercel/next.js by ijjk 6 months ago
Ensure client filter with basePath is correct (#60580) Follow-up to https://github.com/vercel/next.js/pull/60542 this adds regression tests for `basePath` with the client router filter to ensure we... — committed to vercel/next.js by ijjk 6 months ago
Add documentation for client router filter (#60585) This adds documentation around the client router filter we leverage in `pages` to allow incremental migration from `pages` to `app`. Also adds me... — committed to vercel/next.js by ijjk 6 months ago

Most upvoted comments

We’re experiencing the same issue in our application.

We’ve set basePath: /app in our frontend application. However, occasionally the app redirects to localhost:3003/app/app when using the router.push function.

I’ve delved into the Next.js source code and arrived at the same conclusion as @myxoh.

In our application, the bloom filter is set as:

numItems: 1, errorRate: 0.01, numBits: 10, numHashes: 7, bitArray: [0, 0, 1, 0, 0, 1, 1, 1, 1, 1],

Due to these settings, this condition evaluates to true, leading to a handleHardNavigation with the variable asNoSlashLocale, which has a duplicated /app in the URL.

I attempted to reproduce the behavior in a new application and couldn’t replicate the issue. I believe this is because the bloom filter settings in the new app are different:

numItems: 1, errorRate: 0.01, numBits: 20, numHashes: 7, bitArray: [0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1],

How are the bloom filter parameters determined? While setting clientRouterFilter: false does work, what potential drawbacks does this approach have?

You can reproduce the issue using different bloom filter configurations at this link: https://codesandbox.io/s/javascript-forked-f4zl66?file=/index.js

NicolasGiaco on Sep 13, 2023

Disable clientRouterFilter in next config can work around this problem.

 experimental: {
    clientRouterFilter: false,
  },

JunlinPan-Crypto on Jun 6, 2023

This configuration is also tied to this issue (causing full page navigations rather than client navigations): https://github.com/vercel/next.js/issues/54231

It seems to me the BloomFilter is leading to collisions - maybe the hash has too little entropy? (Just throwing some ideas but I’m not too familiar with this branch of the code)

myxoh on Aug 21, 2023

One month has passed, but this BUG still persists. A simple way to reproduce: goto https://nextjs.org/docs/advanced-features/measuring-performance and click Multi Zones.

https://user-images.githubusercontent.com/21997724/230564789-eb209ac4-2b52-4fb6-9177-d4d403715bf6.mp4

powerfulyang on Apr 7, 2023

Hey yeah we can look at getting this behavior/flags documented so it’s more obvious how this should work and how it can be customized more.

ijjk on Jan 12, 2024

What logic is applied here that makes this work seemengly completely randomly

This probabilistic data structure and we originally had a lower filter size which increased the likely hood of false positives, after further investigation we can increase the filter size which reduces the likelihood of false positives quite a bit (< 0.01%) as the filter compresses very well.

x-ref: https://github.com/vercel/next.js/pull/60542

Thanks @crisvergara for providing that regression test!

ijjk on Jan 12, 2024

We have the same issue with a dynamic url, it’s very hard to debug: for a specific dynamic route pattern, one page for which the bllomfilter “contains” function return true and for another one it returns false. it creates a random experience for users, where some links will trigger a hard navigation to an unexisting url, and some will not. The only solution for now seems to not use next/link at all.

rflament on Nov 15, 2023

Same issue with duplicate basePath and hard navigation for a few routes.

It seems like the matchesBflStatic would be true in certain cases, which causes the above problem.

https://github.com/vercel/next.js/blob/canary/packages/next/src/shared/lib/router/router.ts#L1083

JunlinPan-Crypto on Jun 6, 2023