gatsby: Refreshing then hitting the back button causes 404 [Update: Development-mode only]

Update

From comments below: If you use gatsby build / gatsby serve, the bug doesn’t happen, apparently. So, it affects development only.

Description

If you refresh a page, then click on the back button, you get 404 instead of getting to the page you were on.

Steps to reproduce

Links

• Reproduction on Codesandbox: https://codesandbox.io/s/github/hossameldeen/Gatsby-site
• Source code link: https://github.com/hossameldeen/Gatsby-site

Steps

• Clone the repo, run gatsby develop, and open http://localhost:8000 (or just open codesandbox link above)
• You should see a link click to go to /another-page
• Click on the link.
• You should now see a page with some text.
• Click on refresh button in the browser.
• Click on the back button in the browser.

Expected result

Go back to the homepage http://localhost:8000 and show its contents (the link)

Actual result

Getting 404

Environment

  System:
    OS: Windows 10 10.0.19041
    CPU: (12) x64 Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz
  Binaries:
    Node: 12.19.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.5 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
    npm: 6.14.8 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Spartan (44.19041.423.0), Chromium (88.0.705.63)
  npmPackages:
    gatsby: ^2.32.3 => 2.32.3 

More info

• Same issue when using navigate instead of Link.
• Seems to be the same issue as #7261
• Credits of finding the issue and providing a reproduction to @mostafa-zaied