ajv: JSONSchemaType incorrectly requires optional properties to be nullable

Hey future traveler 👋

I just discovering this post and was about to give up, when I saw nullable: true and it worked!

Here’s a clear example:

export const schema: JSONSchemaType<{ value: string, embed?: string, page?: string}> = {
    type: "object",
    properties: {
        value: {
            type: 'string',
        },
        embed: {
            type: 'string',
            nullable: true
        },
        page: {
            type: 'string',
            nullable: true
        },
    },
    required: ['value'],
    additionalProperties: false
}

@erikbrinkman i think the correct way to improve it in the next major version is to only require nullable: true if the property has null as possible value in typescript, independently of whether it’s required or optional. That will be a breaking change.

This is a deliberate decision to require that optional properties are allowed to be null values, as it is how APIs are commonly designed…

Whether it was a correct choice is a good question but it’ll definitely stay like that until the next major version…

Hey future traveler 👋

I just discovering this post and was about to give up, when I saw nullable: true and it worked!

The problem with annotating a property with nullable: true is that allows for runtime values where the key exists and the value is null. This is different than the key being missing entirely or the value being undefined.

An example object that would pass validation of the schema you provided that I would not want to be valid:

{
  "value": "foo",
  "embed": null,
  "page": undefined
}

To say this another way, I want to be able to define a schema that says “if the embed key exists, the value must be a string”. This type of rule is important when the object in question goes through some mapping process where Object.entries(([k, v]) => {}) is used.

Using your schema, the signature of the callback is:

([k, v]: [string, string | null | undefined]) => void

whereas I’d like to write a callback with a signature of

([k, v]: [string, string]) => void

It’s not that optional properties are “allowed” to be null values, it’s that they must be marked as nullable in the schema. This means that it is impossible to use JSONSchemaType with any schema that has a non-nullable optional property. This seems like a pretty common use case…

This is breaking because it change the existing typechecks in a way that causes type errors where there wouldn’t be before.

I think there’s an argument for gradual migration, but I don’t think it’s very compelling. Gradual migration for types like this is really complicated. It’s not about a flag in the validator, it’d be in constructing the type itself. The migration path would be more like creating two JSONSchemaTypes, and having validate work for either schema type. However this leaves a number of other problems:

1. The types already push the limit of what typescript wants to do, and in refactoring these types, it was not uncommon to run into errors related to type complexity
2. Since the migration path involves adding a new type, it extends the fix another generation, because you’d have to introduce the fix type in this version, then make them alias to the same type in 9, and then remove the fix type in 10, which is a lot of work for something that has been known about for two years #1375

However, there already exist a number of fixes for people who really care, and make the migration tradeoffs themselves:

1. use the wrapper types above to fool aspects of typescript to arrive at the correct behavior.
2. use the fixed type manually, and specify your own interfaces for validate. This should make migration for you to v9 (wrt to this problem) trivial, but puts the burden of migration on the libraries that care, rather than everyone else.

If none of this is compelling, you’re always free to submit a PR with a proposed fix so we can further discuss the benefits versus costs of gradual migration.

Yes, this hasn’t been landed yet because it’s breaking, and hence why it’s in the v9 branch. I can’t speak for when v9 should be released.

I found https://github.com/ajv-validator/ajv/commit/b4b806fd03a9906e9126ad86cef233fa405c9a3e which is itself quite old at this point. Is this change not included because it is seen as a breaking change to the current typescript API?

You can either wait until the next major version of ajv where it’s fixed

Is there something we can follow to see when that happens? Would this be v9?

@kilahm See my post previously. ajv does validate appropriately if you leave out the nullable, it’s just the the type converter doesn’t handle it. You can either wait until the next major version of ajv where it’s fixed, or use my casting trick above to make it think the types work out, while still getting the validation you want.

I should be clear that I think what needs to be redone from scratch is just SomeJSONSchema e.g. just a type for unchecked JSON Schemas. This should be independent of the actual implementation of JSONSchemaType which is why I think it’s an easier change. It still won’t be backwards compatible thought.

As a side note, it may fix: #1652 as I think that also stems from misalignment of SomeJSONSchma. When / if implementing this, it probably makes sense to add a test for that too, but I’m ultimately not sure if this will fix that.