next.js: Client cannot recover from version skew
Link to the code that reproduces this issue
https://github.com/knpwrs/nextjs-skew-recovery-bug
To Reproduce
- Run
npm run build - Run
npm start - Open
http://localhost:3000and make sure the browser development tools are open. - Press the
Server Actionbutton. - Observe logs on the server indicating the function was called.
- The network response has a
200response code indicating no errors and atext/x-componentmime type. - Shutdown the server, leave the app running in the web browser.
- Rename the
logServerfunction inactions.tsand update the import and usage incomponents.tsxto match (for instance,logServercan be renamed tologServer2). - Run
npm run build - Run
npm start - Go to the already running app
- Press the
Server Actionbutton. - Observe an error on the server:
[Error: Failed to find Server Action "006c3c7b08402d18959b82a9692db1011f32bcc8fd". This request might be from an older or newer deployment. Original error: Cannot read properties of undefined (reading 'workers')] - There are no errors on the client. Error boundaries do not trigger. There are no uncaught errors in the console. There is no way for the client to know that the function call failed and no way for the client to recover.
- The network response has a
200response code indicating no errors and atext/htmlmime type. - Press the
Throw Errorbutton. Observe an uncaught error in the console.
Note that I couldn’t get error.tsx or global-error.tsx to work for either the failed function call or the thrown client-side error.
Current vs. Expected behavior
Currently the client is not able to recover from version skew when a server action cannot be called. Everything appears normal to the client.
I would expect the error boundary to catch an error so the client can refresh and recover.
Provide environment information
Operating System:
Platform: darwin
Arch: arm64
Version: Darwin Kernel Version 24.2.0: Fri Dec 6 19:01:59 PST 2024; root:xnu-11215.61.5~2/RELEASE_ARM64_T6000
Available memory (MB): 32768
Available CPU cores: 10
Binaries:
Node: 23.6.0
npm: 10.9.2
Yarn: 1.22.19
pnpm: 9.12.2
Relevant Packages:
next: 15.2.0-canary.33 // Latest available version is detected (15.2.0-canary.33).
eslint-config-next: N/A
react: 19.0.0
react-dom: 19.0.0
typescript: 5.7.3
Next.js Config:
output: N/A
Which area(s) are affected? (Select all that apply)
Server Actions, Error Handling
Which stage(s) are affected? (Select all that apply)
next start (local), Other (Deployed), Vercel (Deployed)
Additional context
This is particularly problematic given the following quote from this blog post:
Secure action IDs: Next.js now creates unguessable, non-deterministic IDs to allow the client to reference and call the Server Action. These IDs are periodically recalculated between builds for enhanced security.
I couldn’t find any documentation about this. It appears that action IDs can change at any time and clients which haven’t refreshed yet won’t have any way to deal with this.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 10
- Comments: 51 (8 by maintainers)
Commits related to this issue
- Respond with `404` for unknown server actions (#80613) When a server action is sent from an older deployment and skew protection is not enabled or implemented, we intended to return a `404` response.... — committed to ZainCheung/next.js by unstubbable a year ago
- Respond with `404` for unknown server actions (#80613) When a server action is sent from an older deployment and skew protection is not enabled or implemented, we intended to return a `404` response.... — committed to vercel/next.js by unstubbable a year ago
@jonschmidt If you’re using vercel you can probably turn on Skew Protection and not worry about it, right? https://vercel.com/blog/version-skew-protection
I have tried it. While it does not prevent the generation of new server action IDs, they are more stable.
I believe what I’ve observed is the following:
Without a consistent NEXT_SERVER_ACTIONS_ENCRYPTION_KEY encryption key:
With a consistent encryption key:
So I think this is a valid workaround and what I will probably try for our deployments:
Hopefully that will work.
Anyway this is not good to have to opt into using this environment variable during build steps in order for server actions to be (only semi-) backward-compatible with clients. It would be great if there was some way to have explicit control over what each server action’s name is. Or otherwise some other config option that allows us to make sure they do not change from build to build.
I really agree here. Actions should be treated like any API endpoint with respect to backwards compatibility for clients in the wild. Obfuscating them randomly isn’t really security and only hurts logging and observability.
I think very often features that might make sense for deploying in Vercel are leaking into Next.js as a framework without as much consideration for those who are deploying and supporting Next.js on their own.
I still think there’s a lot of missing information about
NEXT_SERVER_ACTIONS_ENCRYPTION_KEY:In my personal (and I suppose professional) opinion, the encryption key for action ids does not provide increased security whatsoever. Especially since there is a mapping of action ids sent down to the client in the server-rendered HTML. I’ve never had a need to obscure endpoints in my last 13 years as a professional web developer. It would be far better to encourage good security practices for developers and avoid all this encryption rigmarole altogether.
I can certainly see the argument for using an encryption key to send sensitive session state down to the client, but not for obscuring the action ids. It certainly seems to create more problems than it solves.
In any case, this issue is about the client not being able to recover from version skew. The client has no way of knowing that an action call has failed.
I tried to open an issue to get clarity around this in docs and it was immediately closed by a bot. https://github.com/vercel/next.js/issues/75448
Have you all tried the NEXT_SERVER_ACTIONS_ENCRYPTION_KEY? I still see this error from time to time, not as much as before but still. 🤔
Edit: Maybe I’m to eager, gonna let this be in production for a while. It could be due to customers having tabs open from before fix.
But a question, if I change the serverAction code it will also result in a new Id correct?
I think this comment deserves a gold medal, sir. Thanks for such detailed explanation 💪
You mean a normal endpoint huh?
Like @J4v4Scr1pt asked;
Can we get some clarification on this? Does it change if I edit the specific action? Or other actions in the same file?
I agree we need more details on how this variable is used. I’ve tested locally and it seems we only need this during build time to retain consistent action ids. Can someone confirm this?
I extracted this from the source code, this is how keys are currently generated
I agree with this sentiment unless we implement our own skew protection, rotating this key would cause the same issue. It seems like we have to make a tough trade-off between user experience vs security.
Thank you so so much for your reply and I understand it now. But I am thinking if this is a little bit of paradox? For the safety considering, the server actions will be “changed” after every build by default, and this can cause version skew after every build; To avoid the version skew, we can keep the encryption key be the same for every build, which actually make every server action, or we can say the endpoint of our server action API routes, always be the same for every build.
This is a little bit of complex but seems like the current conclusion is we accept the version skew for better security, or we solve the version skew and give up for better security.
I don’t know if I also missed some info (and don’t know why I just read the encryption part again to understand this rather than the first time), does next have a way to detect the version skew on the server side, or will we have a plan for the future? I think it would be a better solution if we can detect version skew, set up a business-based solution for version skew (like refresh or something), then we could solve the two issues I have mentioned above.
Thanks again! And just for a reminder for others:
@J4v4Scr1pt To reiterate what was mentioned above, you can essentially opt-out of this behavior when self-hosting → https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations#overwriting-encryption-keys-advanced
Have you tried this? https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations#overwriting-encryption-keys-advanced
@opeologist Please create a new issue if https://nextjs.org/docs/app/guides/data-security#overwriting-encryption-keys-advanced does not work as expected. I marked this issue as fixed by #80613 because it’s titled “Client cannot recover from version skew” and the “Expected behavior” section says “I would expect the error boundary to catch an error so the client can refresh and recover.” This is fixed with the PR. We will also follow this up to improve the DX of handling the error in #78933.
we’re using the environment variable created via below for a standalone next app:
we’re passing that into our Dockerfile as an arg, applying it as an ENV before running the build script. locally all seems correct, as when i use a key in my .env file, the ID for a server action is the same across multiple runs of the app, and different when i don’t have the key.
what i don’t understand is how some boxes in production are randomly saying they failed to find a server action ID, but i can correlate the host that logged the error and traces from that host that has 200s from that action id… how can a host report a 200 for a server action ID and about an hour later have a log saying it failed to find that same server action ID 🤔
Thanks for the thorough suggestion! We actually have an external api that we use, and its all in the same VPN. There is very little cost to us switching to client side mutations and refetching using SWR for now, and waiting to Vercel comes up with a better solution (or maybe we migrate to a provider that handles this better).
Just adding cacophony of voices. We have implemented our first dozen or so server actions that we were ready to ship to production until we discovered this problem in QA. We are deploying on AWS.
As far as I can tell, there is no fix for this. Setting the encryption key NEXT_SERVER_ACTIONS_ENCRYPTION_KEY in both our build and runtime environments does not result in stable IDs. Unless someone has better info on how to achieve stable IDs, we have to delay our new launch and rewrite everything using client side fetching.
I just remembered, yes. But seems like this not actually helps a lot especially attackers could just observe the business actions to know where the ids points to. 😂
Maybe based on this key to encrypt and decrypt the request and response body will be more completely.
Yes! So it’ll be up to your digression on how you want to rotate the key.
Thank you so much for your response! Sry that I missed this information. After reading it, just to make sure I understand correctly. Does it mean if I set the NEXT_SERVER_ACTIONS_ENCRYPTION_KEY that the action Id will essentially always be the same after deployments?
I bet you have 100 other things to do so thx again for your help 🙏.
What a great work you have done investigating @LikeDreamwalker 🙌 You workaround seems solid, but should not be needed imo.
This issue should be quite broad for people using Next 15 and standalone right?
Good to address security in v.15 but this is now a serious issue for us in production affecting several customers and very hard to track, understand and fix. Is there any way to turn it off?
We are considering doing a re-work and remove all server-actions in favor of route-handlers instead. But as you can imagen this would be a to cumbersome and a fix would be much more preferred.
@leerob I think this issue should not be overlooked because this is something that could potentially cause users a lot of issues. You(or someone else) help to find a solution for this would be very appreciated by many I think 🙏.
Does the action-id only change if you change the server-action code somehow? Or does it get a new Id on every build?
Edit: Got the the answer
I guess a workaround would be to just not use serveractions when self-hosting and just use route handlers… 😕
I think I have a rough solution based on @knpwrs 's repo: LikeDreamwalker/nextjs-skew-recovery-bug
To get started, we need to be clear with some concepts:
So based on this issue’s original reproduce, we can know that this issue happens because client side is trying to request to server with an out of date action id, which causes the version skew issue. My idea is we try to do something in middleware, if we can know that the incoming server action (or any other) requests is out of date, we can use some way to notice this client to ask it to reload the page.
For the deployment part, I choose deploymentId to keep the client and server in sync. This feature seems very much like the normal package verison, but it will be exposed to every request’s header if we set it manually.
For the checking part, I use middleware to achieve this:
This is a version for test, and if you want to use as a workaround solution, please update the specific parts. There are many ways to notice the client with something, but if you want client to be noticed ASAP, by control both the server action and normal request would be better and redirect to a special route is also acceptable.
And here is how you could test with my version:
staticId.tsfile, like “version-skew-v20”window.location.hrefafter 3 secondsAnd this solution is a kind of rough because:
router.refresh()can work better thanlocation.href(). But for version skew I think it can be understandable to force reload rather than refresh. You can watch this video to understand:https://github.com/user-attachments/assets/e014af89-8216-48b4-aead-1c80ccecb851
Solve #76149
@mbranch For that it’s working as expected because we need a GitHub repo link (you provided a link to an issue instead)—the bot should not run when you run the Documentation issue template.
@mbranch @knpwrs Looks like there’s an issue with the GitHub actions closing these Documentation template issues, taking a look 👁
After some more experimentation the following code also does not catch any client-side errors, even though the server output indicates that the action cannot be found: