puppeteer: page.goto returns null periodically

Here’s a bit of a workaround until this is fixed:

let fullResponse = await chromePage.goto("https://wherever.com");
if (fullResponse === null) {
  console.log("Got null, trying wait.");
  fullResponse = await chromePage.waitForResponse(() => true);
}

if (response === null) {
    response = await page.waitForResponse(() => true);
}

This will return the first response you’re getting once interception starts. This will NOT return the intial request you’re actually looking. So if you actually need that request, you’re out of luck unless you use an incognito tab.

Yep, I can confirm this works on 1.8.0!

Actually, regarding my above example of reproducing it. If you comment out the first request (or second, doesn’t matter) so it looks like this:

const puppeteer = require("puppeteer");
(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  // const Url = "https://www.peterbe.com/#anything";  // FAILS
  const Url = "https://www.peterbe.com#anything";  // ALSO FAILS
  // const Url = "https://www.peterbe.com";  // WORKS
  const r1 = await page.goto(Url, { waitUntil: "domcontentloaded" });
  if (r1 === null) console.log("Response is null!");
  else console.log("Response OK?", r1.ok());

  // const r2 = await page.goto(Url, { waitUntil: "domcontentloaded" });
  // if (r2 === null) console.log("Response is null!");
  // else console.log("Response OK?", r2.ok());

  await browser.close();
})();

Then it works.

So, pretty sure the response becomes null if the URL has been requested before and contains an anchor part.

FWIW, If you could have an issue due to service workers, over at playwright they have the ability to block the service worker registration (but maybe the real issue is that page.goto still returns the wrong ‘frame’)

https://github.com/microsoft/playwright/pull/14321/files#diff-5e6d61431da6c4bb6fda4276ba5781ae64e98537d9f68c9c51a465e1fcb4c6c2R125

ah yes… I can confirm it’s working thanks! 😄

It seems this problem has something todo with caching. When I create a createIncognitoBrowserContext for every request. This never happens. Since I removed that extra context. This Problem happened also to me. So my guess is that now the second request is fullfilled via cache and so there is no response. This is also why the chromePage.waitForResponse(() => true); works. Since it waits for the cache refresh request.

@jtara1 that sounds unlikely, for me it happens everytime that a page has a redirection initiated by javascript after the event domcontentloaded is triggered

@aslushnikov Hi, sorry, it was my fault, the problem was the ssl certificates and chromium is killing the request.

@bluepeter that workaround works for me, thanks a lot