puppeteer: Page.Content freezes without error after running 101 times

Upstream fix has landed as https://chromium-review.googlesource.com/c/chromium/src/+/1496759; this’ll be fixed after a roll.

Happening to me still with v2.0

Have a similar issue with page.waitForSelector. At the same time page.evaluate(() => document.querySelector(...)) works:

const puppeteer = require("puppeteer");

(async () => {
  const browser = await puppeteer.launch();
  const [page] = await browser.pages();

  for (let i = 0; i < 200; i++) {
    await page.goto("http://www.example.com/?page=" + i);
    console.count("i");
    try {
      await page.waitForSelector("body");
    } catch (error) {
      const exists = await page.evaluate(() =>
        Boolean(document.querySelector("body"))
      );
      console.log(error.name);
      console.log("exists: " + exists);
    }

    await page.waitFor(500);
  }

  await browser.close();
})();

Seems like a frame._secondaryWorld (which is used in page.content, page.waitForSelector and so on) stops to receive execution context at some point. I logged Runtime.executionContextCreated event here which normally happens two times per navigation, one for default and one for isolated context. But here after number of reloads event starts to happen just once (for default context).

i: 94
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type default
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type isolated
i: 95
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type default
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type isolated
i: 96
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type default
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type isolated
i: 97
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type default
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type isolated
i: 98
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type default
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type isolated
i: 99
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type default
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type isolated
i: 100
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type default
i: 101
TimeoutError
exists: true
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type default
i: 102
TimeoutError
exists: true
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type default
i: 103
TimeoutError
exists: true
executionContextCreated: frameId 8848BB97D31B30DE5B85C03C971953D6, type default
i: 104
TimeoutError
exists: true

Yes, it’s happening to me still with v2.0 too

@Mahanchello Try this:

await page.evaluate(selector =>
  Boolean(document.querySelector(selector)),
  sel.sub.confirmBtn
);

Can confirm - downgrading to 1.11.0 solves freezing. (in my case caused by waitForSelector() in v 1.13.0 with Chrome )

I can reproduce with this reduced code:

'use strict';

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const [page] = await browser.pages();

  for (let i = 0; i < 200; i++) {
    await page.goto('http://www.example.com/?page=' + i);
    console.log('Before Page.content', i);
    const html = await page.content();
    console.log('After Page.Content', i);
    await page.waitFor(500);
  }

  await browser.close();
})();

It stops at:

Before Page.content 100

Hello! Same here, puppeteer worked great for several iterations of going through a page, then after N loops it will hang at waitForSelector(). Running 10.2.0 as well.

I am still having this exact same issue as the original post. Running puppeteer 10.2.0? Any thoughts?

@aslushnikov Does the fix mentioned above rely on some lib that could be not getting installed in Docker? I am unable to reproduce this locally but when it gets deployed into a docker container, It happens more often than not. Could this be a possible reason we are still seeing this after a fix was deployed?

@djErock : I am also facing the similar issue, it works fine for me in local machine, but when we run it as docker image i see timeout issues for page.waitforselector. Can you please help, if you were able to find some solution for it. Thanks

We are also facing timeouts issues on jenkins, which is running on docker image. can you help us understand what worked for you?

@aslushnikov Does the fix mentioned above rely on some lib that could be not getting installed in Docker? I am unable to reproduce this locally but when it gets deployed into a docker container, It happens more often than not. Could this be a possible reason we are still seeing this after a fix was deployed?

@djErock : I am also facing the similar issue, it works fine for me in local machine, but when we run it as docker image i see timeout issues for page.waitforselector. Can you please help, if you were able to find some solution for it. Thanks

@pendo324 solution on puppeteer-core v2 headless works great. page.waitforselector or iframe works periodically and result in timeout, especially for nested iframe.

@vsemozhetbyt helped, thanks 😄

This still happens to me on HeadlessChrome/79.0.3945.79 with puppeteer-core 2.0.0.

If I replace my page.waitFors with

await page.evaluate(() =>
  Boolean(document.querySelector('...'))
);

my code works as expected.

Downgrading to 1.11.0 also works as expected.

puppeteer@1.12.2 same problem. page.content() freezes, some waitForSelector() fall by timeout. happens from time to time. going down to 1.11.0

Did 1.11.0 help solve the issue?

yeah, 1.11.0 works perfectly!