gatsby: [gatsby-transformer-remark] query on markdownRemark no longer returns added fields

@neonguru sorry for the wait, when i was about to write this comment i was called away to take care of some errands.

Below is detailed the steps i took to make this work:

• Created a new Gatsby website using the starter mentioned.
• Ran npm audit fix, which for me bumped some dependencies.
• Issued gatsby develop with no changes whatsoever, i can confirm that the error pops up.
• Took a look at the code inside gatsby-node.js, especially to the following functions, onCreateNode, setFieldsOnGraphQLNodeType and addSiblingNodes. Starting from onCreateNode, it’s working as it should, with a little extra work, but it does the trick, it extends the node associated with markdown with the corresponding fields. Namely date and slug, and then it pushes the node in question to a array called postNodes.
• Moved onto the function setFieldsOnGraphQLNodeType as i’m somewhat familiar with Gatsby, but not an expert in any way shape or form, i went to the documentation page for that specific api call and from my read on it, it’s being used rather incorrectly. The reasoning why it’s used in the starter and how it’s used is beyond me, I can only speculate that probably at some time you could implement this approach.
• Moved onto addSiblingNodes function. Now it seems that’s the root of the issue, reading the code:

function addSiblingNodes(createNodeField) {

  postNodes.sort(
    ({ frontmatter: { date: date1 } }, { frontmatter: { date: date2 } }) => {
      const dateA = moment(date1, siteConfig.dateFromFormat);
      const dateB = moment(date2, siteConfig.dateFromFormat);

      if (dateA.isBefore(dateB)) return 1;

      if (dateB.isBefore(dateA)) return -1;

      return 0;
    }
  );

  for (let i = 0; i < postNodes.length; i += 1) {
    const nextID = i + 1 < postNodes.length ? i + 1 : 0;
    const prevID = i - 1 >= 0 ? i - 1 : postNodes.length - 1;
    const currNode = postNodes[i];
    const nextNode = postNodes[nextID];
    const prevNode = postNodes[prevID];

    createNodeField({
      /*  */
      node: currNode,
      name: "nextTitle",
      value: nextNode.frontmatter.title
    });
    createNodeField({
      /* node: currNode, */
      node: currNode,
      name: "nextSlug",
      value: nextNode.fields.slug
    });
    createNodeField({
      /* node: currNode, */
      node: currNode,
      name: "prevTitle",
      value: prevNode.frontmatter.title
    });
    createNodeField({
      /* node: currNode, */
      node: currNode,
      name: "prevSlug",
      value: prevNode.fields.slug
    });
  }
  fs.writeFile('parseddata.json',JSON.stringify(postNodes,null,2),'utf-8',err=>{
    if (err){
      throw err
    }
  })
}

It would seem that it iterates over the items that were injected while the onCreateNode api call is processing and then extend the node, by adding the fields. Now if you notice i intentionally left the following piece of code there:

fs.writeFile('parseddata.json',JSON.stringify(postNodes,null,2),'utf-8',err=>{
    if (err){
      throw err
    }
  })

After it runs the file generated will contain the following, i’m only showing one item, to keep it short:

{
    "id": "6c4242d8-e9a3-5122-9423-aff6106d7efc",
    "children": [],
    "parent": "5072bbba-6b48-528b-aea7-ddb2be8076db",
    "internal": {
      "content": "\r\nAround a year ago, this project was one of the first ones ported to the `v1` version of GatsbyJS.\r\n\r\n<blockquote class=\"twitter-tweet\" data-lang=\"en\"><p lang=\"en\" dir=\"ltr\">Great news! Gatsby Material starter is now based on stable V1 of <a href=\"https://twitter.com/gatsbyjs\">@gatsbyjs</a>!<a href=\"https://t.co/P2JqNbcRmk\">https://t.co/P2JqNbcRmk</a></p>&mdash; Ruben Harutyunyan (@Vagr9K) <a href=\"https://twitter.com/Vagr9K/status/883073085963993089\">July 6, 2017</a></blockquote>\r\n\r\nAnd now, it's super exciting for me to announce the `v2` version of `Gatsby Material Starter` with the support of the newest Gatsby v2 features!\r\n\r\nYou can also visit [my personal blog](https://vagr9k.me) if you want to see a fully implemented blog based on this starter.\r\n\r\nNew features are:\r\n\r\n- NetlifyCMS support\r\n- Complete optimization of all image assets via `gatsby-image` for faster loading times\r\n- New date formatting options\r\n- Netlify build configuration to make deploys easier\r\n- Less restrictive URL options\r\n- Bug fixes related to dates and sorting\r\n\r\nFull feature list:\r\n\r\n- Blazing fast loading times thanks to pre-rendered HTML and automatic chunk loading of JS files\r\n- [React-MD](https://github.com/mlaursen/react-md) for Material design\r\n  - Integrated FontAwesome support\r\n  - Integrated Material Icons support\r\n- SASS/SCSS styling\r\n- [NetlifyCMS](https://www.netlifycms.org/docs/intro/) support for creating/editing posts via admin panel\r\n- Separate components for everything\r\n- High configurability:\r\n  - User information\r\n  - User social profiles\r\n  - Copyright information\r\n  - More!\r\n- Author segment\r\n  - Name\r\n  - Location\r\n  - Description\r\n  - Links\r\n  - Follow Me button\r\n- Posts in Markdown\r\n  - Code syntax highlighting\r\n  - Embedded YouTube videos\r\n  - Embedded Tweets\r\n- Tags\r\n  - Seprate page for posts under each tag\r\n- Categories\r\n  - Separate page for posts under each category\r\n- Suggested posts segment\r\n- Disqus support\r\n  - Notifications about new disqus comments\r\n- Responsive design\r\n  - On mobile, Disqus is loaded only after expanding comments for better performance\r\n- Social features\r\n  - Twitter tweet button\r\n  - Facebook share/share count\r\n  - Reddit share/share count\r\n  - Google+ share button\r\n  - LinkedIn share button\r\n  - Telegram share button\r\n- SEO\r\n  - Sitemap generation\r\n  - robots.txt\r\n  - General description tags\r\n  - Schema.org JSONLD (Google Rich Snippets)\r\n  - OpenGraph Tags (Facebook/Google+/Pinterest)\r\n  - Twitter Tags (Twitter Cards)\r\n- RSS feeds\r\n- Loading progress for slow networks\r\n- Offline support\r\n- Web App Manifest support\r\n- Automatic image transformation and size optimization\r\n- Netlify deploy configuration\r\n- Google Analytics support\r\n- Development tools\r\n  - ESLint for linting\r\n  - Prettier for code style\r\n  - Remark-Lint for linting Markdown\r\n  - write-good for linting English prose\r\n  - gh-pages for deploying to GitHub pages\r\n  - CodeClimate configuration file and badge\r\n\r\nNOTE: Take a look at [gatsby-advanced-starter](https://github.com/Vagr9K/gatsby-advanced-starter) if you prefer building UI from scratch and/or only interested in fundamental features.\r\n\r\nMore information is available on [GitHub repository page](https://github.com/Vagr9K/gatsby-material-starter).\r\n",
      "type": "MarkdownRemark",
      "contentDigest": "4d4fb19bd9b3c15b45312c3c8dff293b",
      "owner": "gatsby-transformer-remark",
      "fieldOwners": {
        "date": "default-site-plugin",
        "slug": "default-site-plugin",
        "nextTitle": "default-site-plugin",
        "nextSlug": "default-site-plugin",
        "prevTitle": "default-site-plugin",
        "prevSlug": "default-site-plugin"
      }
    },
    "frontmatter": {
      "title": "Introducing Gatsby Material Starter",
      "cover": "starter-logo-1024.png",
      "category": "gatsby",
      "tags": [
        "gatsby",
        "material",
        "starter"
      ],
      "date": "2018-07-02"
    },
    "excerpt": "",
    "rawMarkdownBody": "\r\nAround a year ago, this project was one of the first ones ported to the `v1` version of GatsbyJS.\r\n\r\n<blockquote class=\"twitter-tweet\" data-lang=\"en\"><p lang=\"en\" dir=\"ltr\">Great news! Gatsby Material starter is now based on stable V1 of <a href=\"https://twitter.com/gatsbyjs\">@gatsbyjs</a>!<a href=\"https://t.co/P2JqNbcRmk\">https://t.co/P2JqNbcRmk</a></p>&mdash; Ruben Harutyunyan (@Vagr9K) <a href=\"https://twitter.com/Vagr9K/status/883073085963993089\">July 6, 2017</a></blockquote>\r\n\r\nAnd now, it's super exciting for me to announce the `v2` version of `Gatsby Material Starter` with the support of the newest Gatsby v2 features!\r\n\r\nYou can also visit [my personal blog](https://vagr9k.me) if you want to see a fully implemented blog based on this starter.\r\n\r\nNew features are:\r\n\r\n- NetlifyCMS support\r\n- Complete optimization of all image assets via `gatsby-image` for faster loading times\r\n- New date formatting options\r\n- Netlify build configuration to make deploys easier\r\n- Less restrictive URL options\r\n- Bug fixes related to dates and sorting\r\n\r\nFull feature list:\r\n\r\n- Blazing fast loading times thanks to pre-rendered HTML and automatic chunk loading of JS files\r\n- [React-MD](https://github.com/mlaursen/react-md) for Material design\r\n  - Integrated FontAwesome support\r\n  - Integrated Material Icons support\r\n- SASS/SCSS styling\r\n- [NetlifyCMS](https://www.netlifycms.org/docs/intro/) support for creating/editing posts via admin panel\r\n- Separate components for everything\r\n- High configurability:\r\n  - User information\r\n  - User social profiles\r\n  - Copyright information\r\n  - More!\r\n- Author segment\r\n  - Name\r\n  - Location\r\n  - Description\r\n  - Links\r\n  - Follow Me button\r\n- Posts in Markdown\r\n  - Code syntax highlighting\r\n  - Embedded YouTube videos\r\n  - Embedded Tweets\r\n- Tags\r\n  - Seprate page for posts under each tag\r\n- Categories\r\n  - Separate page for posts under each category\r\n- Suggested posts segment\r\n- Disqus support\r\n  - Notifications about new disqus comments\r\n- Responsive design\r\n  - On mobile, Disqus is loaded only after expanding comments for better performance\r\n- Social features\r\n  - Twitter tweet button\r\n  - Facebook share/share count\r\n  - Reddit share/share count\r\n  - Google+ share button\r\n  - LinkedIn share button\r\n  - Telegram share button\r\n- SEO\r\n  - Sitemap generation\r\n  - robots.txt\r\n  - General description tags\r\n  - Schema.org JSONLD (Google Rich Snippets)\r\n  - OpenGraph Tags (Facebook/Google+/Pinterest)\r\n  - Twitter Tags (Twitter Cards)\r\n- RSS feeds\r\n- Loading progress for slow networks\r\n- Offline support\r\n- Web App Manifest support\r\n- Automatic image transformation and size optimization\r\n- Netlify deploy configuration\r\n- Google Analytics support\r\n- Development tools\r\n  - ESLint for linting\r\n  - Prettier for code style\r\n  - Remark-Lint for linting Markdown\r\n  - write-good for linting English prose\r\n  - gh-pages for deploying to GitHub pages\r\n  - CodeClimate configuration file and badge\r\n\r\nNOTE: Take a look at [gatsby-advanced-starter](https://github.com/Vagr9K/gatsby-advanced-starter) if you prefer building UI from scratch and/or only interested in fundamental features.\r\n\r\nMore information is available on [GitHub repository page](https://github.com/Vagr9K/gatsby-material-starter).\r\n",
    "fileAbsolutePath": "..../test_custom_fields/content/Introducing-Gatsby-Material-Starter.md",
    "fields": {
      "date": "2018-07-01T23:00:00.000Z",
      "slug": "/introducing-gatsby-material-starter",
      "nextTitle": "Big Test",
      "nextSlug": "/big-sample-test",
      "prevTitle": "Bold Mage",
      "prevSlug": "/bold-mage"
    }
  },
]

As you can see so it seems, that the fields are generated, but they are not, what’s being updated is the node inside the array, not the actual node needed. Probably at some point in time this would work, but as of now, it will not, i’ll leave to more experienced people with Gatsby, namely @sidharthachatterjee to elaborate on this. I tested out some permutations and they all yelded the same error.

• After this i thought, is this absolutely needed? when the same result could be achieved with less code, so i went ahead and made some changes to gatsby-node.js resulting in the following, the requires were left out to keep it a bit shorter:

exports.onCreateNode = ({ node, actions, getNode }) => {
  const { createNodeField } = actions;
  let slug;
  if (node.internal.type === "MarkdownRemark") {
    const fileNode = getNode(node.parent);
    const parsedFilePath = path.parse(fileNode.relativePath);
    if (
      Object.prototype.hasOwnProperty.call(node, "frontmatter") &&
      Object.prototype.hasOwnProperty.call(node.frontmatter, "title")
    ) {
      slug = `/${_.kebabCase(node.frontmatter.title)}`;
    } else if (parsedFilePath.name !== "index" && parsedFilePath.dir !== "") {
      slug = `/${parsedFilePath.dir}/${parsedFilePath.name}/`;
    } else if (parsedFilePath.dir === "") {
      slug = `/${parsedFilePath.name}/`;
    } else {
      slug = `/${parsedFilePath.dir}/`;
    }

    if (Object.prototype.hasOwnProperty.call(node, "frontmatter")) {
      if (Object.prototype.hasOwnProperty.call(node.frontmatter, "slug"))
        slug = `/${_.kebabCase(node.frontmatter.slug)}`;
      if (Object.prototype.hasOwnProperty.call(node.frontmatter, "date")) {
        const date = moment(node.frontmatter.date, siteConfig.dateFromFormat);
        if (!date.isValid)
          console.warn(`WARNING: Invalid date.`, node.frontmatter);

        createNodeField({
          node,
          name: "date",
          value: date.toISOString()
        });
      }
    }
    createNodeField({ node, name: "slug", value: slug });
  }
};

exports.createPages = ({ graphql, actions }) => {
  const { createPage } = actions;
  const postPage = path.resolve("src/templates/post.jsx");
  const tagPage = path.resolve("src/templates/tag.jsx");
  const categoryPage = path.resolve("src/templates/category.jsx");
  return graphql(
    `
    {
      allMarkdownRemark(sort: { fields: [frontmatter___date], order: DESC }) {
        edges {
          node {
            fields {
              slug
            }
            frontmatter {
              title
              tags
              category
            }
          }
        }
      }
    }
    `
  ).then(result => {
    if (result.errors){
      console.log(result.errors);
      throw result.errors
    }
    const tagSet = new Set();
    const categorySet = new Set();


   const posts= result.data.allMarkdownRemark.edges;

    posts.forEach((edge,index) => {

      if (edge.node.frontmatter.tags) {
        edge.node.frontmatter.tags.forEach(tag => {
          tagSet.add(tag);
        });
      }

      if (edge.node.frontmatter.category) {
        categorySet.add(edge.node.frontmatter.category);
      }
     
      const previous = index === posts.length - 1 ? posts[ index - 1].node : posts[index + 1].node
      const next = index === 0 ? posts[posts.length - 1].node : posts[index - 1].node

      createPage({
        path: edge.node.fields.slug,
        component: postPage,
        context: {
          slug: edge.node.fields.slug,
          nexttitle:next.frontmatter.title,
          nextslug:next.fields.slug,
          prevtitle:previous.frontmatter.title,
          prevslug:previous.fields.slug
        }
      });
    });

    tagSet.forEach(item=>{
      createPage({
        path: `/tags/${_.kebabCase(item)}/`,
        component: tagPage,
        context: {
          tag: item
        }
      });
    })
    categorySet.forEach(item=>{
      createPage({
        path: `/categories/${_.kebabCase(item)}/`,
        component: categoryPage,
        context: {
          category:item
        }
      });
    })
  });
  
};

Key thing to take from this, pagination that was done through graphql field, is now made possible through Gatsby’s special prop context. It’s a trade off, but one of little consequence and which yelds the exact same result.

• With the adjustment done here, some changes had to be made to the template, namely ./src/templates/post.jsx and one extra component, that being ./src/components/PostSuggestions/index.jsx.

Starting with the template ./src/templates/post.jsx, i tried to maintain the strucure as much as possible to prevent confusion, also the code is commented out, so that when you’re reading it know what is happening at every step, leaving out the imports to keep the code shorter.

export default class PostTemplate extends React.Component {

  constructor(props) {
    super(props);
    this.state = {
      mobile: true
    };
  }
  /**
    * it's best apply the check on both lifecycle methods or some errors
    * will pop up, as all the pages/routes are generated in node in gatsby, so the access to those apis don't exist like window for instance
    */
  componentDidMount(){
    
    if (typeof window!=='undefined'){
      this.handleResize();
      window.addEventListener("resize", this.handleResize);
    }
  }
  componentWillUnmount(){
    if (typeof window!=='undefined'){
      window.removeEventListener('resize',this.handleResize)
    }
  }
  
  handleResize=()=>{
    if (window.innerWidth >= 640) {
      this.setState({ mobile: false });
    } else {
      this.setState({ mobile: true });
    }
  }

  render(){
    // the variable extracted through destructuring from the state
    const {mobile}= this.state
    const expanded = !mobile;
    //
    const postOverlapClass = mobile ? "post-overlap-mobile" : "post-overlap";
    /**
     * all of the data passed in gatsby.node inside context will be retrieved through the prop pageContext
     * then retrived what is needed
     *  location is a internal Gatby routing prop
     */
    const {pageContext,location}= this.props
    const {nexttitle,nextslug,prevtitle,prevslug,slug}=pageContext
    //

    // data retrieved from query
    const postNode = this.props.data.markdownRemark;
    const post = postNode.frontmatter;
    if (!post.id) {
      post.id = slug;
    }
    if (!post.category_id) {
      post.category_id = config.postDefaultCategoryID;
    }
    //
    const coverHeight = mobile ? 180 : 350;
    return (
      <Layout location={location}>
        <div className="post-page md-grid md-grid--no-spacing">
          <Helmet>
            <title>{`${post.title} | ${config.siteTitle}`}</title>
            <link rel="canonical" href={`${config.siteUrl}${post.id}`} />
          </Helmet>
          <SEO postPath={slug} postNode={postNode} postSEO />
          <PostCover
            postNode={postNode}
            coverHeight={coverHeight}
            coverClassName="md-grid md-cell--9 post-cover"
          />
          <div
            className={`md-grid md-cell--9 post-page-contents mobile-fix ${postOverlapClass}`}
          >
            <Card className="md-grid md-cell md-cell--12 post">
              <CardText className="post-body">
                <h1 className="md-display-2 post-header">{post.title}</h1>
                <PostInfo postNode={postNode} />
                <div dangerouslySetInnerHTML={{ __html: postNode.html }} />
              </CardText>
              <div className="post-meta">
                <PostTags tags={post.tags} />
                <SocialLinks
                  postPath={slug}
                  postNode={postNode}
                  mobile={this.state.mobile}
                />
              </div>
            </Card>
            <UserInfo
              className="md-grid md-cell md-cell--12"
              config={config}
              expanded={expanded}
            />
            <Disqus postNode={postNode} expanded={expanded} />
          </div>
          {/* old component <PostSuggestions postNode={postNode} /> */}
          <PostSuggestions postNode={{next:{title:nexttitle,slug:nextslug},previous:{title:prevtitle,slug:prevslug}}} />
        </div>
      </Layout>
    );
  }
  
}


export const pageQuery = graphql`
  query BlogPostBySlug($slug: String!) {
    markdownRemark(fields: { slug: { eq: $slug } }) {
      html
      timeToRead
      excerpt
      frontmatter {
        title
        cover
        date
        category
        tags
      }
      fields {
        slug
        date
      }
    }
  }
`;

• Finally ./src/components/PostSuggestions/index.jsx. Same as above is applied here:

export default class PostSuggestions extends Component {
  render() {
    // destructuring(extraction of the component prop and elements inside)
    const { postNode } = this.props;
    const {previous,next}= postNode
    // 

    return (
      <div className="post-suggestions md-grid md-cell--12">
        {/* <Link to={postFields.prevSlug} className="post-suggestion"> */}
        <Link to={previous.slug} className="post-suggestion">
          <FontIcon
            forceFontSize
            forceSize={64}
            className="secondary-color arrow-nav"
          > 
            arrow_back
          </FontIcon>
          <div className="headline-container hide-on-mobile">
            <h2 className="md-body-2 secondary-color">Previous</h2>
            <h6 className="md-headline secondary-color">
              {/* {postFields.prevTitle} */}
              {previous.title}
            </h6>
          </div>
        </Link>
        {/* <Link to={postFields.nextSlug} className="post-suggestion"> */}
        <Link to={next.slug} className="post-suggestion">
          <div className="headline-container">
            <h2 className="md-body-2 secondary-color">Next</h2>
            <h6 className="md-headline secondary-color">
              {/* {postFields.nextTitle} */}
              {next.title}
            </h6>
          </div>
          <FontIcon
            forceFontSize
            forceSize={64}
            className="secondary-color arrow-nav"
          >
            arrow_forward
          </FontIcon>
        </Link>
      </div>
    );
  }
}

• Issued gatsby develop and opened http://localhost:8000 and i’m presented with:

• Clicked on a random item yelded the following:

Feel free to provide feedback, so that we can close this issue, or continue to work on it till we find a solution. Also if you need i’m more than happy to push the code to repository so that you can take your time looking at it. And before i forget, sorry for the extra long comment.

@Vagr9K no need to thank, i’m glad that i was able to help out in solving this issue.

@neonguru i’m currently finishing up a reproduction for you. I’ll post back soon…Just doing some tweaks. Mind waiting just a little longer?