dbt-core: [CT-3316] [Bug] Invalid JSON schema for manifest v11

@mars-lan thank you for surfacing this and thank you to Metaphor Data for sponsoring Coalesce! I hope the conference went well for you all. Wanted to update you and let you know that engineers are looking into this and will update shortly.

What doesn’t work is putting it in the left hand box of https://www.jsonschemavalidator.net/ to show that it can be used as a custom schema to validate json.

Just wanted to break in and say that https://www.jsonschemavalidator.net/ doesn’t support JSON Schema 2020-12 and seems not to be the best validator. The v11.json schema looks good in the eyes of these validators:

• https://jschon.dev/schema
• https://json-everything.net/json-schema/
• https://json-schema.hyperjump.io

All of them support 2022-12 dialect and are listed on the official list of the online JSON Schema validators: https://json-schema.org/implementations#web-(online).

Also I’d like to note that using “$ref” and “$defs” keywords together is valid, as can be seen from this example:

• https://github.com/json-schema-org/json-schema-spec/issues/523#issue-284635788

@mars-lan Just to confirm, are you saying that v11 manifest currently hosted is working for you or that you pulled the v12 manifest from the PR and that is working for you?

The latest published v11 is working. I haven’t tested v12 yet, and doesn’t seem like there’s one published either: https://schemas.getdbt.com/dbt/manifest/v12/index.html

Confirmed that the latest JSON schema is passing. Feel free to close this now.

I synced with @dbeatty10. Although the v11.json manifest artifact can be used to produce models with the datamodel-code-generator. What is currently broken is using the artifact as custom schema to validate input json (which it used to do). In my demo, I put the schema in the right hand box on https://www.jsonschemavalidator.net/ which verified that it was valid jsonschema. What doesn’t work is putting it in the left hand box of https://www.jsonschemavalidator.net/ to show that it can be used as a custom schema to validate json.

@QMalcolm I was able to reproduce with catalog/v1.json. The original poster reported issues with the schema for manifest.json, but I had a small catalog.json handy, and it exhibits the same underlying issue with https://www.jsonschemavalidator.net/.

Reprex

1. Copy the contents of https://schemas.getdbt.com/dbt/catalog/v1.json (or here) into the Select schema text area on the left side of https://www.jsonschemavalidator.net/
   • Select “Custom” from the drop-down
2. Copy the following contents into the Input JSON text area on the right side

   {
     "metadata": {
       "dbt_schema_version": "https://schemas.getdbt.com/dbt/catalog/v1.json",
       "dbt_version": "1.7.0b2",
       "generated_at": "2023-10-07T13:34:04.627535Z",
       "invocation_id": "c09b4da4-35c5-418d-8a4b-447d2925f075",
       "env": {}
     },
     "nodes": {},
     "sources": {},
     "errors": null
   }
   

3. You should see an error like the following:

But if you replace the Select schema with the contents from here, it should validate:

@mars-lan I am currently working on this

@mars-lan Thanks for opening this issue. We have reproduced the behavior you discovered using https://www.jsonschemavalidator.net/, and we find it concerning, but we are curious whether you have encountered problems with the schema with any other tools, or other parts of your workflow. In other validators we have tried, and with the libraries we use internally for testing, the schema performed as we expected.

We’ll follow this up regardless, but knowing the above would help us better understand the scope of the issue.

We used https://github.com/koxudaxi/datamodel-code-generator to generate Python classes from the JSON schema and it also had issues parsing the schema.