graphql-ruby: [1.8] Field level testing broken in class based API

I feel bad opening so many issues so I’ll waver a little off topic here instead.

I am curious as to why “Don’t test the schema” is considered a best practice. I am surprised that others haven’t run into the bug i documented here, so perhaps I’m doing something wrong, but I don’t feel like its wrong. I want to write tests that:

1. As a whole, exercise as fully as possible my GraphQL API
2. Avoid excessive boilerplate
3. As individual tests, have a narrow focus

I don’t see a way to balance these factors without field testing. Using a decorator pattern or similar to separate resolution from the API layer, and testing just that, means we are no longer testing for errors in the schema, instrumentation, or resolvers. Executing full queries means I have to traverse from my root node down to the fields I wish to test, which is expensive in aggregate. This encourages writing broad tests that query a lot of fields at once, which I believe is bad practice.

Whereas field tests allow me to write tests that test a single thing (one field of one type), do it with little boilerplate (no GraphQL in tests), while avoiding schema level breakage from say, a bad instrumenter.

I would go as far as saying I am surprised that the docs do not encourage field testing, and provide test helpers for it. I have a built a little something for RSpec that needs work but is working well for us so far:

RSpec.describe "LocalResource" do
  subject { DirectorySchema.types["LocalResource"] }
  let(:resource) { create(:local_resource) }

  describe "name" do
    let(:field) { subject.fields["name"] }
    it "returns name" do
     # def resolve_field(object = nil, arguments = {}, context = {})
      expect(resolve_field(resource)).to eq resource.name
    end
  end
end

I’m sorry, I really don’t have the bandwidth to maintain low-level APIs to schema primitives. My best suggestions are:

• If you GraphQL layer contains a lot of logic, extract it to POROs and unit test them
• And, run queries against your schema to test the GraphQL layer

I’ve heard of folks building systems that generate GraphQL queries against a schema (eg https://github.com/pocke/graphql-autotest), and I think that would be a really good way forward, if anyone wants to take it on.

Imagine a testing library that, based on a type name and an object, generates a few queries and runs them, and makes assertions on the results. It would be a pretty thin abstraction over writing query strings by hand, but it would dry it up a lot.

A big advantage to a “full-stack” approach is that it engages all the various instrumenters, tracers, authorization hooks, analyzers, etc – so that the test result really reflects the GraphQL layer.

Another way to consider it is to compare a GraphQL object type to a Rails controller. Rails controllers are usually (always?!) tested with HTTP-like setups, provided by Rails test helpers. Personally, I’ve never manually initialized a controller and called methods on it. And, I don’t think I’d want to: part of my integration tests are routing, authorization, before_actions, etc, so I want the test harness to run those things. If my controller action contains a lot of business logic, that should be extracted so it can be unit tested.

Anyways, sorry I can’t help more here. If anyone takes a shot at a GraphQL testing library, please do share it!

Any progress here? This is the one thing holding us back from the greatness that is 1.8! I don’t mind contributing, but I will need a pointer as to where to begin.

We have a similar wrapper to make field testing easier (which we do extensively). I’m pretty happy with how it’s going, but I do wish there were some way to opt-in to instrumentation, type-conversion, etc. (I’ve had a few bugs relating to the graphql arguments object not behaving like the hash I’m passing in in my tests).

Yeah, the new objects are pretty different. There’s a way to get the old-style field:

field = MySchema.find("Property.testMe")
field.resolve(...)

I hope that helps!