arktype: Surprisingly slow performance vs Zod

Report a bug

I’ve been trialling Arktype as a potential replacement for Zod in a project, and have been running benchmarks against both. Surprisingly to me, arktype is benching roughly twice as slowly as the equivalent Zod schema. Running a CPU profile, it seems like arktype is spending a lot of time inside mergeConfigs and errorFromNodeContext. Is this expected? Is the schema incorrect in some way?

I don’t know if this is a bug per se, but this seemed like the most appropriate issue template.

Link to profile: https://profiler.firefox.com/public/kw0z7ke5rjmsz7rmfhfm9djtyvwh0et87cwynjg/flame-graph/?globalTrackOrder=0&thread=0&timelineType=category&v=10

Screenshot (viewed in SpeedScope rather than Firefox profiler as I think it’s clearer):

Image

🔎 Search Terms

zod performance mergeConfigs errorFromNodeContext

🧩 Context

  • ArkType version: 2.0.0
  • TypeScript version (5.1+): 5.7.3
  • Other context you think may be relevant (JS flavor, OS, etc.): Node, MacOS

🧑‍💻 Repro

import { describe, bench } from "vitest";
import { type } from "arktype";


 const userSchemaArktype = type({
  id: "string",
  email: "string",
  profile: {
    firstName: "string",
    lastName: "string",
    age: "number",
    preferences: [{
      key: "string",
      value: "string|number|boolean"
    }]
  },
  metadata: "Record<string,unknown>",
  createdAt: "Date",
  "updatedAt?": "Date"
});

const validUserData = {
  id: "123e4567-e89b-12d3-a456-426614174000",
  email: "test@example.com",
  profile: {
    firstName: "John",
    lastName: "Doe",
    age: 30,
    preferences: [
      { key: "theme", value: "dark" },
      { key: "notifications", value: true },
    ],
  },
  metadata: {
    lastLogin: "2024-01-17T00:00:00.000Z",
  },
  createdAt: new Date(),
};

describe("Arktype Validation Benchmark", () => {
  bench("Arktype Validation", () => {
    userSchemaArktype(validUserData);
  });
});

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 22 (13 by maintainers)

Most upvoted comments

@drivasperez Well in at least one case it is if you look at the switch statement in union32Apply.

The union is automatically discriminated so we can jump to the correct data type in constant time. However, once we’re within the "boolean" case and have true | false remaining, it’s an important part of the internal representation that "boolean" is actually represented as === true || === false.

It gives more accurate type comparisons and better error messages in the general case. So yes, you can always add edge case optimizations for validation, but in this case the current behavior is intended and will give descriptive error messages for "boolean" the same way it would for 0 | 1. If you mess around with TypeScript conditional types, you’ll notice that it represents boolean internally as true | false for the same reason. So in this case, it wouldn’t be an optimization we’d want, but luckily we can bypass all of this anyways for valid data!

If you look at the allows logic you’ll see it’s much more streamlined if you don’t have to keep track of the current path and build descriptive errors.

@drivasperez Set-based types are quite the rabbit hole, but really a beautiful subject once things start to click. All of ArkType’s type comparison and validation logic is essentially derived set-based intersections and reductions. I’m always looking for people to nerd out about that stuff with so feel free to send questions any time 🤓

Whoops, found the mistake 😬

Image

I recently added defaultable tuple elements, which are a form of morph, but undefined should have been 0 here, so all arrays were being treated as morphs- very inefficient!

Will publish a fix in 2.0.3 tonight or tomorrow.

Ahh, I see. So the reason you’re seeing that call is because of the union "string|number|boolean". Even though in the end we don’t end up needing the error because one of the branches ends up being valid, in the general case, there isn’t a way to know we won’t need it.

That said, looking back at this, because your data doesn’t include any context-dependent types like morphs, for valid data we should actually be bypassing error creation altogether (this would not be the case if you added e.g. string.numeric.parse somewhere, at which point we have to track context even for valid data).

So time for me to eat crow- you’re right, this is inefficient! Not necessarily compared to Zod or similar, which don’t have separate allows and apply implementations in the first place, but wasteful nonetheless.

For some reason it appears includesMorph ends up returning true once preferences is converted to an array. I’ll address that for next release so your time type will be running truly optimally 😅