gatsby: Requests for page-data.json return 404 in client side routes

Sorry for the late notice, I have been on vacation for the past week and a half. I can bring you the good news that we will be fixing this natively in gatsby. We’ll put urls inside match-path.json that match a dynamic route.

I’ll be creating a PR to fix this issue. I’ll let you know when it’s up and running to get some feedback.

Unfortunately there’s not much we can do about errors cluttering console. There is closed ticket on chromium bug tracker that ask for a way to not show those errors - https://bugs.chromium.org/p/chromium/issues/detail?id=124534 (this is about application code) - it started in 2012 and last response was 2 days ago, so it’s closed but still kind of active heh.

You can locally opt into ignoring those errors (at least in chrome - dunno about other browsers) by using “Hide network” in dev tools console options:

There are also a lot of concerns about this on #15398 as well.

While @pieh and @wardpeet i understand the gravity of returning to the page manifest, and i dont think anyone here is recommending you roll back a huge performance boost by doing so, i think discarding this as a non-issue is ill informed, and the idea that if it cant be done there, it can’t be done anywhere, feels a little like you’re saying it’s not worth the time to try.

Having some kind of solution, whether it’s purely in the client side routing plugin, or having a situation where Gatsby’s Link has a way to prevent the lookup (similar to what someone here said) feels like at least a more thoughtful approach to helping people out rather than nothing at all.

This is clearly impacting people and really any amount of 404’s for client side applications is NOT a normal side effect of an application. If Gatsby is coming at this as a “Won’t Fix”, you’re basically confirming client side routing is NOT a 1st class citizen of the Gatsby framework, which is not how it appears your team is trying to position it as, at which point, why wouldn’t someone just use Create React App or something of the like?

1 thing that IS a 1st class citizen for Gatsby is the developer and their experience. Hiding the 404s via the terminal filters isn’t really a solution, as then you’re masking logs that may actually be a problem. People with large applications with large amounts of dynamically generated client side routes (such as myself and a few others that prodded here and the other thread) are left to be swamped reviewing tons of 404s in the console to determine if there’s an issue or to find the real ones amongst the swarm of 404s.

We all do appreciate you taking the time to think this through, but I would encourage you to not disregard this as a non-issue. As someone who advocates a lot for Gatsby, it’s kind of disappointing seeing how this is being pushed off. Thanks for your time

@kirbydcool I think this issue should be reopened, the 404 is an issue. For applications with large amounts of links per page (https://github.com/gatsbyjs/gatsby/issues/16097#issuecomment-515488361) it can cause 404s to go through the roof, if we know that the route will result in a 404 the application should not be making it in the first place, instead it should route to a 200.

My temp solution to this issue is to add 301 redirects for page-data.json

page-data/vehicle/([0-9]+)/page-data.json
--->
%{scheme}://%{host}/page-data/vehicle/*/page-data.json

This does result in the correct page-data for the dynamic route to be displayed. However a 301 still results in a network request for a new document each time.

e.g. page-data/vehicle/123456789/page-data.json

Gatsby should identify this route as being dynamic and the loader should fetch the */page-data.json.

The result would be:-

1. Browsers would intelligently cache the page-data.json.
2. Bursty requests would not fire for pages with n Link items.
3. CDN logs would not be polluted with 100,000s of 404s

Is there someone on the gatsby core team who can prioritise this issue?

I’ve created a draft PR, just need to write a test and I’ll publish a canary build so you can test it. To make sure it works for all your use cases.

PR got published in latest gatsby. I couldn’t mimic prefetch behaviors as mentioned above. If you still have it, please open a new issue.

Hello @wardpeet, why is this closed? There are only workarounds here, the issue still happens regularly on our website. We’re on gatsby 4.1, the fix that closed this issue was on 2.x.

FWIW, I still get this when using Gatsby’s <Link /> to pages in my static folder. The workaround is to… not do that. 😃

I’m also getting 404 for prefetch request /page-data//page-data.json . Not sure where is this double slash coming from, possibly something is missing between these 2 slashes…

FWIW, you can also see this in the client-only-paths example. Screenshot here:

I too think that this should be reopened. FWIW we’re experiencing it as well, and the rationale behind closing it was not “This is not an issue”, but rather, “We can’t fix this.” I think I’ve got a fix, related entirely to the comment brought up by @eknowles . I’ll have it up shortly.

You can currently use npm install --save-dev gatsby@page-data-404 as a workaround. I’ll be finishing up this PR today.

Would it be possible to extend the <Link> component with a clientSide or noPrefetch boolean prop that, prevents it loading the page-data.json? I guess it would still try to load it once the page is openend, but if we can can at least prevent the prefetch that would help.

I’ve published a canary build gatsby@page-data-404. Please install it with npm install --save-dev gatsby@page-data-404 or yarn add --dev gatsby@page-data-404

I’m happy to get any problems

Made a video of my issue, slack-files.com/T03A23LJR-FN12WFD1S-5f3218378f

@josephkandi 😱 that’s weird, so it’s waiting for the page-data to complete?

@wardpeet Am still getting the 404 page shown but the pages load faster now than before.

I’ve published a canary build gatsby@page-data-404. Please install it with npm install --save-dev gatsby@page-data-404 or yarn add --dev gatsby@page-data-404

I’m happy to get any problems

Made a video of my issue, slack-files.com/T03A23LJR-FN12WFD1S-5f3218378f

@josephkandi 😱 that’s weird, so it’s waiting for the page-data to complete?

+1 for the fix, having the same issue as well for an an app we are planning to go to production in a few weeks time. Was planning to abandon Gatbsy and try use CRA instead.

How about something that creates an empty page-data.json file at build for client-side routes? You would need to define what client-side pages you have somewhere which adds some boilerplate however…

Client only routes can contain a dynamic element that is not known at build time, like a product page that references an item id (ie. /widgets/12345/details/).

I’ll reopen since it seems like there’s more discussion here.

Does this cause issues other than just showing errors in console?

This seems to be breaking direct Links to images using the public URL. Steps to reproduce:

• gatsby new test-site

• Edit the /src/components/image.js component to

  1. import { Link, useStaticQuery, graphql } from "gatsby"
  2. Add publicURL to the placeholderImage query.
  3. Wrap the <Img ... /> in a <Link to={data.placeholderImage.publicURL}>

• Build & Serve the site

When you click the image to follow the link you get the public URL (i.e. http://localhost:9000/static/gatsby-astronaut-6d91c86c0fde632ba4cd01062fd9ccfa.png) but it results in the Not Found page and the console shows the page-data.json is returning the 404.

Reloading the error page results in the image (though you are no longer in React).

This same behavior happens if I import the Link from @reach/router.

Is this something withPrefix can address?

@pieh, it just needs to know the dynamic routes, not every route right?

this would be the contents of ./cache/match-paths.json

e.g.

[
    {
        "path": "/product/*",
        "matchPath": "/product/*"
    },
    {
        "path": "/admin",
        "matchPath": "/admin/*"
    }
]

if we know that the route will result in a 404 the application should not be making it in the first place

Well, this is exactly the problem. For Gatsby to know that it would need to have centralized pages manifest - and this doesn’t scale - see https://matthewphillips.info/programming/gatsby-pages-manifest.html as example of issues that centralized pages manifest cause