graphql-php: Ability to defer field resolution (to solve N+1 problem)

There were several discussions about ways to solve N+1 problem in GraphQL - #42, #60.

There are two classes of solutions for this problem:

Look-ahead soltion when we analyze the query and pre-fetch required data upfront (see #65)
Defer field resolution (and thus enable batching queries using techniques like DataLoader)

Deferring syntax:

'resolve' => function($obj) {
    $this->dataLoader->buffer($obj->someId);

    return new GraphQL\Deferred(function() use ($obj) {
       // This will internally load all buffered ids (once) and return the one requested here:
        return $this->dataLoader->fetch($obj->someId);
    });
}

Other possible syntax:

'buffer' => function($obj) {
    $this->dataLoader->buffer($obj->someId);
},
'resolve' => function($obj) {
    return $this->dataLoader->fetch($obj->someId);
}

Technically it will require Executor algorith change from Depth-First to mix of Breath-First and Depth-First (requires researching and prototyping).

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 1
Comments: 16 (8 by maintainers)

Most upvoted comments

For anyone interested - we added ability to defer field execution in master. Still experimental phase, but all tests pass and it’s time to play with it.

The syntax for standard sync mode is the one I described previously, namely:

'resolve' => function($obj) {
    $this->dataLoader->buffer($obj->someId);

    return new GraphQL\Deferred(function() use ($obj) {
       // This will internally load all buffered ids (once) and return the one requested here:
        return $this->dataLoader->fetch($obj->someId);
    });
}

It is fully backwards compatible and works in standard PHP versions (no need for ReactPHP or other async platforms to leverage this feature).

Technically it simply alters the order of field execution. Deferreds are skipped for resolution until all non-defered fields are resolved. It enables quite effective batching of queries.

Under the hood it uses Promises as main concept for deferring. Promises do help even in sync environment when combined with queues.

We could implement it without promises (by using field queue instead), but with promises we can leverage any async platform which supports them (like ReactPHP, HHVM, icicle.io, php threads, etc). Obviously the syntax will be different in this case comparing to sync mode (you will return promises of the platform of interest vs GraphQL\Deferred), but conceptually it will work the same way.

Thanks a lot to @mcg-web for contributing Promises support and tests!

vladar on Dec 2, 2016

Docs entry: http://webonyx.github.io/graphql-php/data-fetching/#solving-n1-problem

vladar on Dec 19, 2016

After reviewing #67 and overblog/dataloader-php I am actually in favor of the promise approach. For me, async data loading that solves the N+1 problem is the significant blocker to adopting GraphQL for real world applications.

shadowhand on Nov 16, 2016