zod: Slow parsing performance

This morning I decided to check in on zod and see if there’s been any changes (it’s been a few weeks since last I checked), and WOW! HUGE improvements since v3.1! Here is the results of my latest benchmark on our internal dataset adding Zod 3.5.1:

library	parse time (ms)
zod 3.5.1	76.74
zod 3.1.0	2453.06
zod 3.0.0	2080.56
zod 1.11.17	723.87
myzod 1.11.17	26.72

So just a massive improvement. This is so awesome. Thank you so much to @milankinen, @colinhacks, and everyone who has been contributing - I really appreciate you all. This is really amazing and definitely makes zod a usable option for me again - I’ll definitely be looking into switching back.

Side note: I did run into some minor tweaks I had to make to my schema where I was using passthrough with intersection which is causing an error in 3.5.1, but that was pretty straightforward to clear up.

I did a very fast experimental refactoring last weekend based on the descriptions in my previous comments (diff here, very WIPWIP) and got approximately 50% performance improvement in “real world” and “object” suites and 100% in primitive suites. I’d be nice to compare this with the no-clone approach. 🙂

I’ve implemented #1023, #1021, and #1022 to explore different vectors at making zod faster. I think updating the target is the highest payoff for the least amount of change, but the other PRs also have some notable perf improvements. I’ll keep looking - I think it’s possible to make zod as fast as the alternatives.

I’ve focused mostly on the primitive types because they’ve been pretty straightforward to tweak & get some little wins. Should probably take a break to work on my product 😆 but in the meantime, some thoughts:

• A lot of real-world schemas look like they include .optional().nullable() on a lot of types. That has come with a performance penalty, which #1028 reduces, but could be reduced more by making optional & nullable more like ‘modifiers’ (like maxLength) than their own types. But that would make the codebase less elegant. Anyway, this is all at the micro-level with values and probably isn’t the biggest performance drag.
• I bet a bunch of realworld performance is heavily influenced by ZodObject, which is already pretty well-written but does require an array of objects be created and then merged back into each object. That seems like a worthwhile thing to optimize in the future.

Fwiw, if folks want to benchmark their things, what I’ve been relying on is the npm run benchmark script in this repo combined with

node --prof bench.js
node --prof-process --preprocess -j isolate*.log > profile.v8log.json

And then dumping the results into speedscope.

---

I ran the moltar benchmarks on the latest zod release and it’s 3.6x faster than the previous benchmarked release (3.11.x).

I’ve skimmed a few competing modules and I think the main remaining difference is in Zod’s approach to objects and arrays: zod truly parses, doesn’t validate, and it creates copies of input datastructures. Probably if that approach changed - or we optionally allowed for a non-copying implementation - we’d be able to close more of the gap.

@kibertoad If adding a no-clone option is something that the maintainers would want and if it brings significant performance improvements, then I would love to contribute.

My team was using zod to validate JSON messages coming from IoT devices via a queue. We received hundreds of messages per minute (so not exactly high volume). Validating 150 messages with zod takes more than 6 seconds. Using ajv we can validate the same amount of messages in 120 ms with a fraction of the CPU usage.

We liked the idea of zod, but we have to move away from it until we can get a decent performance. 😦

I’m definitely open to this! Slightly concerned about bundle-size bloat, and I’m not sure how things like transforms will work (I suppose they would just throw). But ultimately it’s super worth it if perf is a blocker on adoption. @InfiniteRain don’t hesistate to get in touch about any implementational questions - Twitter is probably best 👍👍

My first instinct is that this should be a separate method, probably .validate{Async}. (Yeah, yeah, I know.) I like the idea of having the return type as a type predicate (input is T) since there’s no need to re-return the value you just passed in. This API also makes it structurally apparent that the method isn’t returning a clone of the input (since only a boolean gets returned.)

const mySchema = z.object({ name: z.string() });
const data = { ... };
const isValid = mySchema.validate(data);
if(isValid){
  data; // {name: string}
}else{
  data; // unknown
}

The downside is that a “safe” version of this method isn’t really possible, because type predicates can’t be nested inside objects.

I gotta say @tmcw, your work is seriously appreciated. I migrated away from Zod because its real-world performance was too slow but this focus on performance makes it viable again!

I think a fresh issue about performance should probably be more specific than this one, otherwise it’s not really something that could ever be closed. Performance could always be improved, it’s a theme not a task.

My guess in terms of further performance tweaks is that zod would need an option to turn off cloning, which would also need to disable methods like transform or require a new fancy implementation of those methods. Cloning is basically the main overhead, afaict, and the modules that do well in the moltar benchmarks are the ones that do not clone.

This is forcing me to abandon zod for the time being, unfortunately. I’m seeing it take seconds to parse even small chunks of data. I was hoping to see an improvement with v3 but it’s actually worse for me. It’s really unfortunate because I’m having a hard time finding a decent replacement. I was hoping to do a drop-in replacement with myzod, but that doesn’t work quite the same and has some really unusual behaviors. Every other library I’ve tried has severe limitations as well, it seems. And none of them match the level of documentation found here. It’s a really excellent library otherwise, and I appreciate it very much – I hope I can use it again soon.

@dgg I maintain myzod, which is basically just a lightweight reimplementation of zod. It doesn’t have a big community like zod does, but it is much much faster, and has an extremely similar api.

I strongly agree with the idea about “parse, don’t validate” semantics which is (imho) one of the biggest reasons to choose zod over those high performance alternatives. We need to keep in mind that those libraries have been developed from the scratch focusing explicitly on the performance, whereas zod has had more focus on developer friendliness and fail-proof semantics. Because of this, the “legacy” of the former design and implementation choices (e.g. decision to support asynchronous parsing) will have a moderate effect on the performance even though the API and semantics are changed. As a result, zod loses one of it’s main advantages (developer experience) and gets a little bit better performance but is still not able to compete with the top ones in terms of performance.

Luckily there still seems to be some room for performance improvements in zod’s internals. 🙂 I quickly skimmed through the codebase and at least following parts could give some nice speed improvements without changing the existing API and semantics:

Parse context management

The current implementation seems to be using contexts with “lazy” parsing path where path is concatenated upon request. In theory, this is a nice idea in theory, but if we take look at the usage, path is read for each object property. That’s O(k*n) complexity where k = length(path) and n = length(objectKeys). In addition, the whole parse context get “cloned” for each property because the path is different for each key. Same applies to arrays. This is a huge amount of internal cloning, object allocations and GC pressure that must be done regardless of the public API semantics.

I blame myself of this design choice. 😞 When I did my performance improvement PR, I left the path management in such state that the path is constructed prematurely which was definitely a mistake. If we think about where the path is really needed, the only places are issues that are returned to the user. By “returned”, I mean issues that are visible to the user of zod APIs - as long as the parsing is in progress, the parse path isn’t relevant. In practice this means that the path must be generated only if the (sub)-parser returns any issues. In pseudo:

ZodObject._parse = (ctx, object) => 
  const issues = []
  const parsed = {}
  shapeKeys.forEachKey key =>
    const keyResult = keyParser._parse(ctx, object[key])
    if (isInvalid(keyResult))
      keyResult.issues.forEach issue => issue.path.unshift(key)
      issues.push(...keyResult.issues)
    else
      parsed[key] = keyResult.value

  return issues.length > 0 ? INVALID(issues) : OK(parsed)

Even the ctx can be passed as it is because there are no dynamic parts in it anymore. The only required “cloning” is when pushing key issues to the object’s issues. In fact, this could also be eliminated by using linked list (linking two lists is an O(1) operation). Compared to the current implementation, the difference should be significant.

Extra object keys parsing

The current implementation has roughly O(n^2) complexity. And in addition those extra keys are not even needed in the default mode. Fortunately the fix for the default case (strip) should be trivial, just use shapeKeys and don’t even try to find extras. For passthrough, strict and catchall, the most efficient implementation that comes to my mind is to merge both shape and data object into single work-in-progress clone and use it:

// same error handling as in default "strip" case
const issues = []
const wip = Object.assign({}, shapeObject, dataObject)
for const key of keys(wip)
  if key in shapeObject 
    wip[key] = shapeObject[key]._parse(ctx, dataObject[key])
  else
    wip[key] = handleExtraKey(ctx, key, dataObject[key])

return issues.length > 0 ? INVALID(issues) : OK(wip)

Other improvements

There are also some other smaller improvements that could be implemented, e.g using function composition for number/string refinements and simplifying the ok/aborted/invalid result handling but I’m not sure whether they’re worth changing or not.

Just went through versions on our end too. Seems like there’s a huge performance regression from 3.9 -> 3.10. One of our functions went from 192ms to 660ms between the two.

3.11: 3.9:

Typeform.com is also considering using zod for form validations and API request payloads in comparison to yup 👍

Sure, so - a no-clone option, as far as I can see it, would mean:

• No transforming would be possible, no coerce, objects will only support .strict or .passthrough, not .strip
• Objects or arrays with prototype additions would leak through instead of being recreated as plain-old objects.

This is, fwiw, the same tradeoff you’ll see in any of the faster modules on moltar’s benchmarks. I would reckon that people are generally okay with these tradeoffs because those modules are heavily used, and if zod doesn’t support a no-cloning mode, then it’ll never get close to their performance.

Anyway, cloning would be a choice, and you could use the cloning (default) mode and get the guarantee of fresh object prototypes, or use the no-cloning mode and accept that you might get a modified object prototype. At least for my application, I think no-cloning would be very useful and would probably be a lot faster, and modified object prototypes - in my experience - are not especially concerning.

@tmcw I’m not sure if I understand the current caching correctly. Are you implying that the cache is permanent? This would explain why parsing a 150 MiB JSON with a complex schema uses up over 4 GiB memory (not sure how much over, just that it’s less than 8 as I had to manually bump node’s memory limit) on one of my side-projects.

Do you know if the current cache gets freed over time? Also, do you know how I could try out this change to see what the performance difference is with my real-world data? The schema is insanely complex as the data structure it has to parse is ancient and organically grew. There’s lots of .optional()s etc… So, would be really interesting to see the impact of your PRs.

Also, thanks a lot for starting to work on zod performance. I’m really looking forward to this as it currently takes >4 GiB of memory and around 2 minutes to parse the data. So, let me know if there’s a way to try this out.

I originally commented this on the PR, but I think let’s keep the PR’s comments for the code itself

---

Update 1:

I managed to test-out the es2018 version (#1022), and this change alone reduces my real-world 150 MiB complex JSON schema parsing time from 100s to 67s. Nice 33% reduction, just from adjusting a single line.

Update 2:

Trying out the cache-related changes from #1023, I only see an improvement from 100s to 96s. Not bad, but also not life-changing haha

Update 3:

Both PRs combined via cherry-picking locally reduce the total time to 64s. 36% better performance is a very very nice start. I hope there’s still some more aspects that can be optimized, but I’d gladly take even a 0.5% improvement at this point 😄

Update 4:

With both PRs combined, the minimum memory required to succeed is between 3.75-3.875 GiB. Down from the (now tested 4.4-4.5 GiB) which is a nice 15% reduction

I am here because I am actually punched by the slow performance of zod. I use zod to validate http api response. It takes seconds to parse 2000+ rows of data, sad 😦

Interesting things are there are so many zod-like libraries here with tens of stars just because of the performance. If zod could improve the performance of itself, the world would be cleaner since zod is absolutely with the most features and supports.

I’ve implemented the suggested tweak to the default object parsing - not collecting extraKeys, in #1393. It’s a very modest performance bump, seems like around 5% in the realworld testcase. Unfortunately I think that’s what we’re looking at with the cloning strategy - there’s maybe another 10-20% optimization left, but systematically zod will be magnitudes slower than non-cloning modules.

Yes, path indeed is an important property when processing the received parsing/validation issues and the issue contract should not be changed. 👍 But I’m not talking about issue paths. I’m talking about refinement context path property that is not officially document (or at least I didn’t find any references to it) but still accessible through superRefine, e.g.

import { z } from "zod";

const notTsers = z.string().superRefine((s, ctx) => {
  console.log("ctx.path", ctx.path);     // <<< this 
  if (s === "tsers") {
    ctx.addIssue({
      code: z.ZodIssueCode.custom,
      message: "can't be 'tsers'"
    });
  }
});

const person = z.object({
  name: z.object({
    first: notTsers,
    last: notTsers
  })
});

console.log(
  "result",
  person.safeParse({ name: { first: "matti", last: "tsers" } })
);

If you run the snippet, you can see that the ctx.path prints ['name', 'first'] and ['name', 'last'] to the console. However, the parse issue (from safeParse) includes the same path although I don’t explicitly set it to the added issue.

My question is: how important the read-only access to the ctx.path during the superRefine callback execution is, and could that be removed? That is the only functionality in the public API contract that forces the path to be created prematurely and preventing the performance optimizations I presented earlier. Removing the ctx.path doesn’t affect the issues returned from safeParse, they’ll continue to include the path like before.

Recently migrated from ajv to zod then to myzod, so there is no baseline for me to compare between zod versions. Just compared between zod & myzod and zod overall making my task to take around 50% to 80% more time to complete. As the task itself has lot’s mini validations individually it won’t look much but they add up by the time task completes. I also suspect there could be some GC pressure too.

Oh wow a toggle for no clone mode would be incredible! Literally best of both worlds depending on scenario…

the only places are issues that are returned to the user

Now that I looked the codebase again, it seems that the path is actually needed in the refinement context and that might be the reason why I left the implementation as it is… 😬

The .path property is not documented in the README or in the error handling documents at all, so how big deal it would be to remove it? Technically it’s a breaking change but not to the official documented feature… 😇

Not stale. Re-opening due to importance of performance.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@tmcw that is extremely cool!

@tmcw

Incredible work and thanks so much for digging in! I’ll spend some time tomorrow and this weekend trying to think through any implications that need to be addressed.

The point about switching our target is an important one: we have mostly treated compatibility as a best-effort thing, but this would be a bit of a line in the sand. I am not involved in enough open source projects of this size to have a good feel for the general approach the community usually takes here, but I feel like that might warrant a major bump?