strawberry: Support for query complexity

Hey folks, I while ago I wrote down some ideas on how we can add support for query complexity. For this we’ll need two things:

  1. An extensions that calculates the costs based on an operation
  2. Support for customising the costs based on fields/arguments

For customising costs on fields and arguments I was thinking about something like this:

import strawberry

@strawberry.type
class Query:
    @strawberry.field(cost=2)
    def expensive_list(self, limit: Annotated[int, strawberry.argument(cost_multiplier=1)]) -> list[str]:
        return ["Item"] * limit

Here we are saying that each individual item in the expensive_list field costs 2, and the total is a multiplication between the field cost and the limit, so the result would be:

field_cost × limit × 1

We can also have defaults like this:

class StrawberryConfig:
    # Existing attributes and methods ...
    
    default_argument_cost_multiplier: Dict[str, Union[int, float]] = {
        "limit": 1,
        "first": 1, # probably not needed, but worth showing as an example
        "last": 1
    }

default_argument_cost_multiplier is a map between argument name and their multiplier, though I don’t like the name that much.

What do you think?

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 19 (16 by maintainers)

Most upvoted comments

@patrick91

I’ve been looking at this quite a bit as well.

Here are some resources I’ve found and things I like about them that we could maybe take advantage of

  • https://graphql-ruby.org/queries/complexity_and_depth
    • Some way to log/see the complexity of a request (maybe only in graphiql? or configurable to decide when to allow clients to be able to see it?)
    • Default values for common types, e.g. simple types default to 1, connection types multiply
    • Complexity override functions, not just integers
  • https://github.com/pa-bru/graphql-cost-analysis
    • Map of costs (as you’ve posted above)
    • Clear documentation on how unions and interfaces are handled
    • I don’t love setting the cost as a directive, I feel like complexity limiting is a security thing, not something that clients should really care about, assuming they’re good actors
  • https://github.com/slicknode/graphql-query-complexity
    • Estimators that can be defined, extended, and stacked, i.e. ordered list of functions that calculate complexity of a field, returning the first one that “matches”
  • https://shopify.engineering/rate-limiting-graphql-apis-calculating-query-complexity
    • Documents defaults and why
    • Outputs calculated cost in the extensions object at root response level, with varying levels of detail
    • Requested vs actual cost (e.g. asking for 1000 things but only getting 2, estimated cost is much higher than actual cost)

I do like the Annotated with field/argument idea, because it makes it very clear what the cost is. A potential tradeoff is that the cost of fields is scattered around the codebase, rather than being defined explicitly in one place, like the map idea.

I think being able to define your own Estimators and pass those into the Extension could be a great way to provide defaults and to give a lot of flexibility for clients to do things themselves

So my overall thoughts on what I’d like to be able to do/have:

  • Easily be able to enable complexity limiting
  • Set a max complexity on the schema
    • It could be useful to be able to set a max complexity override on specific operations, though exact use cases of this are unclear atm
  • Have some default cost values so if I add the extension and do nothing else, I do get some value out of it
    • e.g. types: 1, scalars: 0, lists/connections: multipliers, metafields: 0 (e.g. cursor, pageInfo)
  • Set my own cost values, potentially with more configurability than just a number
    • I’d like to be able to define costs and/or multipliers on types or on “categories of types”
  • A way to see/log/debug the complexity of a request
  • Would be nice to be able to work in a supergraph setting, where not all fields in the request are resolved by my schema
    • I assume you’d lose some features here, but presumably for federated fields, the schema could add some extra complexity or a multiplier or something

Something like

  • QueryComplexityLimiter extension
  • Has a default max complexity limit
  • Has a default list of estimators that determine how to calculate cost on a field
  • Has a default map of costs/cost functions
  • Clients can extend/override the estimators and map
  • Clients can define costs in the map or at the field/input level
    • Inheritance would be nice here, e.g. a Connection type that has a cost/multiplier built in that would apply to all child implementations
  • Complexity is calculated with cost integers and multipliers
  • Potentially calculate cost at validation time, and actual cost at resolve time

@serramatutu the rust parser is almost ready for beta use 😉 Even then though, we should support pure-python for now.

+1 on separating cost calculation logic vs limitation logic, this would enable easy abstraction of both components