gatsby: Querying `allMarkdownRemark` with no markdown nodes fails

So this is interesting. I’m a bit surprised this is causing confusion. Let me try to explain my mental model for these parts of Gatsby.

At a high level this is how I see Gatsby

   +-----------+   +------------+   +----------------+
   |           |   |            |   |                |
   |  Source   +---> Transform  +--->  Infer GraphQL |
   |  Data     |   | Data       |   |  Schema        |
   |           |   |            |   |                |
   +-----------+   +------------+   +---------^------+
                                              |
                                              |
                  +-------------------+       |
                  |   GraphQL Queries+--------+
                  |                   |
                  |                   |
                  | React Web App     |
                  +-------------------+
                           |
                           v
                    Optimized Production
                    Build
                           +
                           |
                           v
                      Static Files Output

(I hope you enjoy my 5 minute ascii drawing 😄)

Source & Transformer plugins don’t (generally) have anything to do with the GraphQL schema directly. They just “source” data from remote sources or “transform” data that source plugins pull in. So they don’t create a schema, they create data (nodes in Gatsby-speak) which after they’ve finished their work, another Gatsby system comes along and infers a GraphQL schema from the nodes.

So if there’s no markdown files, there’s no File nodes with a mediaType of text/markdown which means gatsby-transformer-remark doesn’t create an nodes of type MarkdownRemark which means there’s no GraphQL types allMarkdownRemark or MarkdownRemark.

Transformer plugins don’t need to know the type of nodes they’re creating as often the type is only known at runtime e.g. “gatsby-source-contentful” creates nodes whose types are taken from the types of content created through the Contentful UI.

I get this is confusing as there’s multiple data transformations going on and many layers of types going on as well.

I suppose a solution could be if a) a transformer plugin only creates “static” types (e.g. type names we know at code-writing time e.g. MarkdownRemark) then b) it could register with Gatsby this type name with any fields in knows for sure would be in the eventual type (generally not much).

Perhaps we just need to do a better job in docs explaining the setup so it’s less surprising what’s going on?

Other ideas?

(To cut down drawing time: http://asciiflow.com/ 😉 Edit: Took another look and seems you used it?! 😄 Nevermind, sorry for the noise ppl!)

Right, changing that behavior is what this issue is about 😄 . It was immensely confusing to install the transformer, have it in the config, yet have my queries marked as invalid. I spent a while debugging until I realized it was due to not having an existing .md file yet so there were no nodes. This also means if one were to transition off sources/transformers, a perfectly valid config can suddenly start throwing errors (like, for example if you deleted your last .md file).

While I don’t think this is a huge issue, I would expect sources/transformers to register their GraphQL type and the “all” queries on source/transformer activation regardless of existing nodes.

This expectation is from the way GraphQL generally works…if you query a field that doesn’t have data, you get null back rather than an error. There is a dividing line between the schema and the data in the schema, and that separation is broken when it comes to Gatsby and its plugins here.

@tfaieta i’ve cloned your repo, installed the dependencies and issued gatsb build && gatsby serve. And i’m presented with this:

Doing a bit of digging it looks like you have installed gatsby-plugin-feed and activated it in the gatsby-config.js. Disabling it will allow a sucessful build as you can see here:

If you need that plugin, a little more of a configuration might be needed.

More on the plugin usage here

Update on this: near completion - just need to update the tests and add a few new cases. I don’t have any time tonight so I would expect the PR more likely on Friday/over the weekend. Thanks Kyle!

Yeah, it’d be new. Lemme add it and move gatsby-plugin-typography over.

transformers were made to report/log on the runs when they found nothing to transform

Ooo! That’s a great solution. But for helping people build good mental models of Gatsby and avoiding confusion about why things aren’t showing up in GraphiQL but also, yeah having lots of unused transformers would slow things down somewhat (though not a ton most likely as most plugins just do a simple if check on nodes which is very cheap).

This one caught me too; I agree it’s not the largest of issues but I think it would be a move that improved the developer experience.

The core issue for me was that my first thought when it occurred was not “oh, I don’t have any markdown files in that folder yet”, but rather “why hasn’t the plugin registered the type? There must be an issue with how I installed and configured the plugin…”. And that led me on a bit of a wild goose chase.

the MarkdownRemark node type will only be created when there is a markdown node, or there will be no MarkdownRemark node type at all, so you can’t query allMarkdownRemark

Hi all, really loving Gatsby, but I’m running into the same issue when I’m trying to build my site. Running gatsby develop works fine but I get this error when I run gatsby build:

 Error: Cannot query field "allMarkdownRemark" on type "Query".
 GraphQL request (3:9)
 2:       {
 3:         allMarkdownRemark(
            ^
 4:           limit: 1000,

The problems started after I added gatsby-mdx and reconfigured my gatsby-config and gatsby-node files. I currently have no .MD files and only have one .MDX file.

Here’s a link to the repo if you all can help me out I’d appreciate it: https://github.com/tfaieta/tfaieta.com

@djm This is great, it will help a lot

@KyleAMathews

Had a crack at this last night and got pretty far but I need some guidance…

Example internal output from a test project:

# Plugins that implement sourceNodes or onCreateNode.
[ 'internal-data-bridge',
  'gatsby-source-filesystem',
  'gatsby-plugin-typography',
  'gatsby-transformer-remark',
  'gatsby-transformer-sharp',
  'default-site-plugin' ]

# state.nodes are owned by:
[ 'internal-data-bridge',
  'gatsby-source-filesystem',
  'gatsby-transformer-remark' ]

# Leaving the "should warn" crowd. These use the APIs but don't own nodes.

[ 'gatsby-plugin-typography',
  'gatsby-transformer-sharp',
  'default-site-plugin' ]

Out of that final list, I believe the only one we actually want to politely warn about is gatsby-transformer-sharp, as my project had no images to find and therefore it wasn’t actually used.

However, the other two pose questions:

1. gatsby-plugin-typography implements sourceNodes but doesn’t source anything.
2. default-site-plugin - sounds like it can be ignored?

What shall we do about these two? and do you know of any others that might be edge cases?

Thanks!

@KyleAMathews What you said makes full sense! The confusion on my part (I can’t speak for LegNeato) came from making the assumption that the transformers have any say whatsoever in the graphql schema generation.

I think you’re right, a documentation solution is probably enough – just telling users that transformers return just data (nodes) in the same sense source plugins do.

Edit: Perhaps if the confusion continues another solution could be that the transformers were made to report/log on the runs when they found nothing to transform; this may also help with people not realising they can remove certain transformers as they don’t use them (for build perf). I can’t think of a reason why you’d leave a transformer in if it wasn’t transforming anything.

It looks like the magic is for every node type registered (in this case MarkdownRemark) a field for all[Typename] (in this case allMarkdownRemark) is added to RootQueryType:

https://github.com/gatsbyjs/gatsby/blob/524ff171cd890b79ce9535d6dc10550c70365d33/packages/gatsby/src/schema/index.js#L13

This only happens if there are nodes, hence this issue.