json_schema: Upgrade from v 0.19 -> v 0.20 results in infinite loop for `lookup_schema`

Hi! We recently upgraded our version of json_schema from v 0.19 -> v 0.20 and a related prmd command no longer works with the very helpful error message of “stack level too deep”

I was able to pinpoint the error to this method call: https://github.com/interagent/prmd/blob/d4908cba5153a76cced9979e6ec17bd217b96865/lib/prmd/commands/verify.rb#L68

Which is referencing this json_schema method: https://github.com/brandur/json_schema/blob/9c4656774a7c7d22a6f466932b34fd47d67c88dc/lib/json_schema/document_store.rb#L25

I dropped in a debugger to follow the method calls and it gets to here

json_schema/reference_expander.rb @ line 332 JsonSchema::ReferenceExpander#traverse_schema:

    329: def traverse_schema(schema)
    330:   add_reference(schema)
    331:
 => 332:   schema_children(schema) do |subschema|
    333:     if subschema.reference && !subschema.expanded?
    334:       dereference(subschema, [])
    335:     end
    336:
    337:     if !subschema.reference
    338:       traverse_schema(subschema)
    339:     end
    340:   end
    341:
    342:   # after finishing a schema traversal, find all clones and re-hydrate them
    343:   if schema.original?
    344:     schema.clones.each do |clone_schema|
    345:       parent = clone_schema.parent
    346:       clone_schema.copy_from(schema)
    347:       clone_schema.parent = parent
    348:     end
    349:   end
    350: end

and then it goes back to lookup_schema

I noticed that the schema_children method was refactored between versions 0.19 and 0.20 so my guess is that the source of the issue is somewhere in there but haven’t been able to pinpoint just yet.

Let me know if I can provide any more info.

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 19 (11 by maintainers)

Most upvoted comments

I think that should do it 😃

kytrinyx on May 3, 2019

@jessieay Yepp, sorry, that’s what my jumbled (but victorious) comment was trying to convey 😃

I’m at the point where I think I might have a clue to what’s going on. I think we hit this section:

https://github.com/brandur/json_schema/blob/9c4656774a7c7d22a6f466932b34fd47d67c88dc/lib/json_schema/reference_expander.rb#L128-L133

and the schema is already expanded, but we’re dereferencing again anyway.

Here’s the improved test, btw. It defines the fixtures within the test (so the scaffold doesn’t have one-off fixtures for just that test), and also removes any data (title, description) that is unnecessary, and only expands schema 2 (it should recursively expand schema 1, which it is trying to do, but then keeps going around in circles).

it "expands a schema with a reference to an external schema in a oneOf array" do
  sample1 = {
    "$schema" => "http://json-schema.org/draft-04/schema#",
    "id" => "http://json-schema.org/draft-04/schema#",
    "definitions" => {
      "schemaArray" => {
        "type" => "array",
        "minItems" => 1,
        "items" => { "$ref" => "#" }
      }
    }
  }
  schema1 = JsonSchema::Parser.new.parse!(sample1)

  sample2 = {
    "$schema" => "http://json-schema.org/draft-04/hyper-schema#",
    "id" => "http://json-schema.org/draft-04/hyper-schema#",
    "allOf" => [
      {
        "$ref" => "http://json-schema.org/draft-04/schema#"
      }
    ]
  }
  schema2 = JsonSchema::Parser.new.parse!(sample2)

  store = JsonSchema::DocumentStore.new
  expander = JsonSchema::ReferenceExpander.new

  store.add_schema(schema1)
  store.add_schema(schema2)

  expander.expand!(schema2, store: store)

  assert schema1.expanded?
  assert schema2.expanded?
end

kytrinyx on May 1, 2019

Thanks @kytrinyx ! I will let you know if I am able to narrow it down any further.

For now we are happy that we have everything working with the version locked at 0.20.3 so there is no rush on our side but I look forward to hearing what you find when you are able to look into this.

jessieay on Apr 4, 2019

@jessieay Thanks, narrowing this down to two files is a tremendous help!

I’m about to get on a plane, but will dig into this as soon as I can (probably within the next 4-5 days).

kytrinyx on Apr 4, 2019

Hi, yeah this part of JSON schema is really complex!

Would you happen to be able to provide an example of a schema that fails? (E.g. a small subset of your schema that gets the error when you load it?)

I can take a look at this if there’s a sample schema to debug with, but without it I can’t even imagine where to begin. Here’s an example of another issue where we chased down a recursive problem: https://github.com/brandur/json_schema/issues/100

kytrinyx on Apr 3, 2019