puppeteer: page. setRequestInterception causes network requests to hang/timeout

Facing the issue in 2021 as well

Enabling await page.setRequestInterception(true); stops futher execution, and removing await page.setRequestInterception(true); will run smoothly. There is issue with await page.setRequestInterception(true);

Just in case you make the same mistake as me

Make sure you set up -

page.on("request", (request) => ...);

Before -

await page.goto(...)

The docs state -

“Once request interception is enabled, every request will stall unless it’s continued, responded or aborted.”

So page.on("request", (request) => request.continue()) is mandatory for every request you want to continue.

We are using mapbox as map provider The map is not loaded if using interceptor, similar to @joelgriffith’s issue described above Map requests get the warning message

this is my interceptor code:

    await page.setRequestInterception(true);

    page.on('request', (interceptedRequest) => {
        if (DOMAINS_BLACKLIST.includes(new URL(interceptedRequest.url()).host)) {
            logger.info(`Blocked url:${interceptedRequest.url()}`);
            requestsBlocked.push(interceptedRequest.url());
            interceptedRequest.abort();
        } else {
            logger.silly(`Intercepted request continue:${interceptedRequest.url()}`);
            interceptedRequest.continue();
        }
        interceptedRequest.continue();
    });

I tried with Puppeteer 1.19.0 and Puppeteer 2.1.1 No difference between those versions

Facing the issue still.

Hey y’all, I found the solution to this. The download script for Chrome was pulling the latest nightly version, which is clearly full of bugs. I downloaded a stable binary of Chrome and symlinked it to the downloaded binary location in the ~/.local folder, and everything went smoothly.

Tested with v2.1.1. Now the problem seems a bit different. Blob URLs are already loaded and there’s no “CAUTION: request is not finished yet!” in Timing panel.

However, workers are still empty.

Seems to be an issue with blob: protocol url’s

Are y’all still experiencing this with puppeteer 2.0.0 or 2.1.0 ?

Hello, any ideas here? having same issue. Thx

Check that you’re not using Google Fonts on the page. Google Fonts (and, I’m assuming, other Google properties?) have an implicit rate limit. I’ve just debugged it to removing Google Fonts from my tested pages and the network timeout disappeared. They don’t reject your request, they time out (at least at the time of this writing).

Same problem with Google + puppeteer 11.0.0, despite using request.continue() Rolling back to 7.1.0 does not fix the issue. Other websites such as https://httpbin.org/ip work well. I’m still investigating this issue, will report progress.

Reproduction code

  const browser = await puppeteer.launch({
    executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
    headless: false
  });

  const page = await browser.newPage();
  await page.setRequestInterception(true);
  page.on('request', async (req) => {
    console.log('received a requst!!!');
    await req.continue();
    console.log('told the request to continue ;)');
  });
  await page.goto('https://www.google.com');

Edit: hacky solution

const puppeteer = require('puppeteer');

async function main() {
  const browser = await puppeteer.launch({
    executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
    headless: false
  });

  const page = await browser.newPage();
  page.on('request', async (req) => { await req.continue(); });
  await page.setRequestInterception(true);
  await page.goto('https://google.com/' + Date.now())
  await page.click(`a[href*=google]`)
}

void main();