next.js: SSR-prefetches with middleware break application between deployments

Link to the code that reproduces this issue

https://github.com/thoaltmann/next-middleware-prefetch

To Reproduce

Build and serve the application the first time
Visit the index page
While keeping the page open, rebuild and serve the application without making any changes
Click on a link on the index page

Current vs. Expected behavior

Current Behavior:

When a middleware is present, hovering over links to pages with getServerSideProps, the client sends _next/data requests (which is the intended behavior and that’s okay). After the first build, the response also includes Headers to signal the client that the request was skipped/bailed (X-Middleware-Skip=1) so the actual request is then made when the user clicks on the link.

However, after the app has been rebuilt/served/deployed, this behavior changes. The _next/data request then also responds with an empty object, but the X-Middleware-Skip header is missing. Instead, a resolved path with the previous build-id is in the X-Nextjs-Matched-Path Header (e.g. /_next/data/5NLSryPAF39BoEpy7KklK/test/1.json). With this response, the client application doesn’t know that this was a bailed SSR-request and uses the empty object as the actual data for the page, resulting in an error. When using a cache, this problem persists as long as the cache does, as the rendered HTML contains an old build-id in the __NEXT_DATA__-script

Expected Behavior:

After the second build, the _next/data-response should include the necessary headers for the client, so it doesn’t use the empty object as actual data. After clicking a link and making the actual request to the data, the server will then send a 404, which leads to a hard navigation, similarly to how it behaves without a middleware.

next-middleware-prefetch

Verify canary release

I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:01 PDT 2021; root:xnu-8019.41.5~1/RELEASE_ARM64_T6000
Binaries:
  Node: 18.17.1
  npm: 9.6.7
  Yarn: 1.22.19
  pnpm: 8.10.4
Relevant Packages:
  next: 14.0.5-canary.12
  eslint-config-next: N/A
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.1.3
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

Data fetching (gS(S)P, getInitialProps), Middleware / Edge (API routes, runtime), Routing (next/router, next/navigation, next/link)

Additional context

No response

About this issue

Original URL
State: closed
Created 7 months ago
Reactions: 21
Comments: 17 (3 by maintainers)

Most upvoted comments

Closing as this should be resolved by https://github.com/vercel/next.js/pull/60968, please upgrade to the latest canary of Next.js and give it a try!

ijjk on Feb 6, 2024

@thoaltmann FYI. I think I have found a workaround for this issue. That this workaround works would also explain why there are not a gazillion issues created about this.

Workaround: Adding an app-folder with:

// app/layout.tsx - A default layout
export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>{children}</body>
    </html>
  );
}

// app/whatever/page.tsx - Dummy app page
export default function Page() {
  return null;
}

Seems to resolve the issue. On hover you get a 404 and then on click you get a hard navigation resulting in a 200 with the document.

vilindberg on Jan 9, 2024

I found this solution to work in our case: This requires in next.config.js:

the DefinePlugin to replace the BUILD_ID variable with Next.js’ buildId
skipMiddlewareUrlNormalize set to true

This helps the client to eventually force refresh its JavaScript and aligns the version again for smooth experience and client-server communication

export const middleware = (req: NextRequest): NextResponse | void => {
  if (req.nextUrl.buildId && req.nextUrl.buildId !== process.env.BUILD_ID) {

    return NextResponse.json(
      { message: 'client version outdated, propagate exception to the client to force refresh' },
      { status: 500 }
    )
  }
 // ...
}

Since this is a version skew problem, there is a few options among:

Backward compatibility, which is isn’t supported by the default build system
Force the client to update/refresh which seems to work with the snippet above (let’s improve it ?)

Lezzio on Jan 21, 2024

The commit that seems to have created the issue (1398de9) unfortunately involves a significant rework/restructure of the routing logic which makes it a bit hard for a newcomer of the repo to understand the full scope of the changes.

However, I suspect the cause of the issue is this if-statement:

https://github.com/vercel/next.js/blob/a5ae1a6653d0906f2e9bb7f443e76161a405a453/packages/next/src/server/lib/router-server.ts#L216

Removing this if-statement seems to resolve the issue, reverting the behavior to its previous state. After this modification, a 404 error on the _next/data/the-build-id/page.json-request is followed by a 200 on the /page-request. Additionally, running the test suite post-modification shows no issues.

Sorry to bother you @ijjk , but I was wondering if you might have any insights into the rationale behind this if-statement’s inclusion. Understanding that its removal might impact other functionalities, I’d greatly appreciate any guidance on potential alternative solutions or areas to investigate further.

Maybe we do not need to remove entire if statement, but just replace status code with 404 instead of 200, for me it fixed a problem during debugging.

https://github.com/vercel/next.js/blob/d08e891ea6c04cffdb775b5dad44b7b563e1b2b3/packages/next/src/server/lib/router-server.ts#L223

ydubinskyi on Jan 10, 2024

The commit that seems to have created the issue (https://github.com/vercel/next.js/commit/1398de9977b89c9c4717d26e213b52bc63ccbe7e) unfortunately involves a significant rework/restructure of the routing logic which makes it a bit hard for a newcomer of the repo to understand the full scope of the changes.

However, I suspect the cause of the issue is this if-statement: https://github.com/vercel/next.js/blob/a5ae1a6653d0906f2e9bb7f443e76161a405a453/packages/next/src/server/lib/router-server.ts#L216

Removing this if-statement seems to resolve the issue, reverting the behavior to its previous state. After this modification, a 404 error on the _next/data/the-build-id/page.json-request is followed by a 200 on the /page-request. Additionally, running the test suite post-modification shows no issues.

Sorry to bother you @ijjk , but I was wondering if you might have any insights into the rationale behind this if-statement’s inclusion. Understanding that its removal might impact other functionalities, I’d greatly appreciate any guidance on potential alternative solutions or areas to investigate further.

vilindberg on Jan 7, 2024

I was able to fix our issue, either by removing our custom middleware.ts (not really an option), or to change it a bit. Cause I did not wanted to add app routing to our app, this would also increase our bundle size, it’s not required and above all was unwanted with regard to our 404 pages (see my comment above).

src/middleware.ts

const buildIdRegex = /\/_next\/data\/([^\/]+)/;

export const middleware = (request: NextRequest): NextResponse => {
    const host = request.headers.get('host');

    const response = NextResponse.next();
    if (request.nextUrl.pathname.startsWith('/_next/data/')) {
        const matches = request.nextUrl.pathname.match(buildIdRegex);
        if (matches?.length > 1 && matches[1] !== process.env.NEXT_BUILD_ID) {
            return NextResponse.redirect(new URL('/404', request.url));
        }
    }

   // .... we dome very secret custom stuff here ;-)
  return response;
 }

And for this to work and be able to grab the buildId from the _next/data/<buildId> xhr requests, I needed to add this to my next.config.js.

As well, make the Next BUILD_ID available at runtime, so that I can actually perform the check in above snippet.

next.config.js

skipMiddlewareUrlNormalize: true,
webpack: (config, { buildId, isServer, webpack }) => {
    config.plugins.push(
      new DefinePlugin({
          'process.env.NEXT_BUILD_ID': JSON.stringify(buildId),

See: Advanced Middleware Flags for the skipMiddlewareUrlNormalize config option.

karaoak on Jan 19, 2024

Unfortunately also not fixed with the release of Next 14.1.0 😢

karaoak on Jan 19, 2024

Hi @vilindberg, unfortunately we didn’t find a solution and had to create our own custom middleware as a cloudfront viewer request lambda.

thoaltmann on Jan 8, 2024