puppeteer: When would an error "Cannot find context with specified id undefined" happen?

Hi, encountered this problem when updating code that was using puppeteer < 1.0 to newest version. I was also able to find source of the problem.

First, my reproduction script

const puppeteer = require('puppeteer');
let page;

const sleep = (time) => new Promise(resolve => setTimeout(resolve, time));

function wait() {
  page
    .waitForSelector('body')
    .then(() => {
      wait();
    })
    .catch(e => {});
}

(async () => {
  const browser = await puppeteer.launch();
  page = await browser.newPage();
  wait();
  for(let i = 0; i < 10; i += 1) {
    await sleep(1000);
    await page.goto('http://example.com/');
  }
  await browser.close();
})().catch(e => {});

with error:

(node:47660) UnhandledPromiseRejectionWarning: Error: Protocol error (Runtime.callFunctionOn): Cannot find context with specified id undefined
    at Promise (/Users/jakubbogucki/Projects/GeneratorChisel/puppeteer-test/node_modules/puppeteer/lib/Connection.js:200:56)
    at new Promise (<anonymous>)
    at CDPSession.send (/Users/jakubbogucki/Projects/GeneratorChisel/puppeteer-test/node_modules/puppeteer/lib/Connection.js:199:12)
    at ExecutionContext.evaluateHandle (/Users/jakubbogucki/Projects/GeneratorChisel/puppeteer-test/node_modules/puppeteer/lib/ExecutionContext.js:79:75)
    at ExecutionContext.evaluate (/Users/jakubbogucki/Projects/GeneratorChisel/puppeteer-test/node_modules/puppeteer/lib/ExecutionContext.js:46:31)
    at Frame.evaluate (/Users/jakubbogucki/Projects/GeneratorChisel/puppeteer-test/node_modules/puppeteer/lib/FrameManager.js:326:20)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)

Please note that it doesn’t happen every time so I added the loop, but possibly 10 runs may be not enough.

End the source of the problem is that line: https://github.com/GoogleChrome/puppeteer/blob/808bf8e5582482a1d849ff22a51e52024810905c/lib/FrameManager.js#L875 You can replace evaluation with

this._frame.evaluate(s => !s, success).catch(e => {})

If you look at rerun function as a whole you will see that earlier it is wrapping evaluation in try/catch and handles errors but that evaluation is not protected in any way. Ironically later there is check for Cannot find context with specified id but it is never reached if second evaluation fails.

/cc @JoelEinbinder @aslushnikov author and reviewer of #1746 that introduced the problematic line

Then why not release 0.14.0? Like, now.

There is literally no reason whatsoever not to automate the releases. There is no reason whatsoever not to release a new version every time a PR has been merged into the master. If you merge something into the master, then it is ready for the release.

This needs more attention. It hasn’t even been acknowledged as a bug.

I have ceased using Puppeteer and ended up developing our own abstraction mostly because of this issue (and general project maintainer attitude towards this project).

Really a shame to see that what Google Chrome team are pushing as an “official” abstraction is being treated with such negligence.

As I am no longer using Puppeteer, I am closing this issue and unsubscribing. Project maintainers are welcome to re-open.

@gajus installing from master gives you the cadence you want…

I had the same issue that was happening when I was doing a page.click(‘selector’).

I reverted to puppeteer 0.12.0 and I had a much clearer error message (it was related to the element I wanted to click on that was not available yet). I fixed the issue by waiting for the element then updated again to 0.13.0 and it is now working fine.

FYI I seem to have worked around this temporarily with this:

wait-for-navigation-and-context.js

/*
  This resolves an issue where puppeteer cannot interact with the page yet because
  a context is not defined. This is usually related to navigation.
  See also: https://github.com/GoogleChrome/puppeteer/issues/1325
*/

const promiseRetry = require('promise-retry')
const timeout = 1000
const iv = 100

module.exports = (page, maxTimeout = 120000) => promiseRetry(async (retry, number) => {
  try {
    await page.evaluate(iv => {
      return new Promise((resolve, reject) => {
        checkReadyState()

        function checkReadyState() {
          if (document.readyState === 'complete') {
            resolve()
          }
          else {
            setTimeout(checkReadyState, iv)
          }
        }
      })
    }, iv)
  } catch (err) {
    if (err.message.indexOf('Cannot find context with specified id undefined') !== -1) {
      retry()
    } else {
      throw err
    }
  }
}, { retries: Math.ceil(maxTimeout / timeout), minTimeout: timeout, maxTimeout: timeout })

Usage:

await page.goto(parsedUrl.href, navigationOpts)
await waitForNavigationAndContext(page, 60000)

At the risk of spamming, I’m going to post this again, so we can stay on top of the actual issue:

The underlying problem is that there is a race condition between the event that clears execution contexts and the event that adds new execution contexts.

When you navigate to a new page, sometimes headless chrome will add the new contexts, before clearing the old ones. When it goes to clear the contexts, it blows away the recently added context, causing: Cannot find context with specified id undefined.

I think the fix should come from upstream, since I think it’s going to be quite difficult to handle correctly in puppeteer.

Just adding another voice on this one: if you hit this while trying to do a form submission or something using evaluate() that causes navigation to occur, try not returning anything from evaluate().

This throws the error:

let waitForSubmit = page.waitForNavigation();
await page.evaluate(() => $("#contentwrapper > form").submit());
await waitForSubmit;

This works correctly:

let waitForSubmit = page.waitForNavigation();
await page.evaluate(() => {
    $("#contentwrapper > form").submit();
});
await waitForSubmit;

I’ve installed from master but I’m still getting the same error thrown. Anyone else?

I found a way to get around this issue that works for me. I made a little utility function as follows:

const waitFor = function(timeToWait) {
  return new Promise(resolve => {
    setTimeout(() => {
      resolve(true);
    }, timeToWait);
  });
};

In my code I had the following calls next to each other. And the getProperty('href') would throw the exception. But once I put the waitFor(3000) in there things started working again.

  back.click();
  await page.waitFor(elementSelectors.backButton);
  await utils.waitFor(3000);  // This is where I had to add my custom waitFor()
  const elms = await page.$$('a');
  // ... some looping code
  const href = await elms[i].getProperty('href');  // This would throw if I took out the custom waitFor()

❗️ I’m getting the same error. I can’t share the entire codebase or provide a full repro since it’s closed source and has lots of moving parts, but here is the code that I’m pretty sure is causing the issue:

  console.log('Clicking Submit');
  try {
    console.log('before 123');
    await page.evaluateHandle(passwordInput => passwordInput.form.submit(), passwordInput);
    console.log('after 123')
  } catch (err) {
    console.log('Caught error!!!!', err)
  }

The output I get when I run it is:

Clicking Submit
before 123
after 123
Running AutoLogin: 389.826ms
(node:22339) UnhandledPromiseRejectionWarning: Error: Protocol error (Runtime.callFunctionOn): Cannot find context with specified id undefined
    at Promise (/Users/davidnagli/Desktop/coding/projects/Axex/server/node_modules/puppeteer/lib/Connection.js:200:56)
    at new Promise (<anonymous>)
    at CDPSession.send (/Users/davidnagli/Desktop/coding/projects/Axex/server/node_modules/puppeteer/lib/Connection.js:199:12)
    at ExecutionContext.evaluateHandle (/Users/davidnagli/Desktop/coding/projects/Axex/server/node_modules/puppeteer/lib/ExecutionContext.js:73:75)
    at ExecutionContext.evaluate (/Users/davidnagli/Desktop/coding/projects/Axex/server/node_modules/puppeteer/lib/ExecutionContext.js:46:31)
    at Frame.evaluate (/Users/davidnagli/Desktop/coding/projects/Axex/server/node_modules/puppeteer/lib/FrameManager.js:326:20)
    at <anonymous>
    at process._tickDomainCallback (internal/process/next_tick.js:228:7)
(node:22339) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:22339) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Notice that the error occurs after after 123 is already logged, and the error never get’s caught by the try-catch. 😟

Another important thing to note is that in my page.evaluateHandle call I’m submitting a form. My theory is that maybe since I’m navigating to another page before the evaluation completes that’s causing the issue 🤷‍♂️

I noticed that this issue has been getting a ton of comments and very little replies from contributors, so it would be amazing if one you guys can just quickly explain what’s the status of this bug and let us know if you need any more info you need to help reproduce it. If I would understand the issue better I’d be happy to submit a PR.

@gajus the library is still prerelease. Automating deployment is probably best when things have stabilized all around. Things are still in a good bit of flux, so imo it is better updates be batched in case multiple breaking changes occur at once. Then developers just look over their existing code once instead on multiple times for multiple updates.

Hey folks, are we sure that runtime.clearExecutionContexts is doing the right thing?

In the code it clears out all execution contexts in all frames. The reason I ask is that I’m also running into this in a different library. It doesn’t happen all the time, only occassionally.

Here’s a full feed of the actions (most of these logs correspond to puppeteer actions):

=== RUN   TestClick
DevTools listening on ws://127.0.0.1:53553/devtools/browser/224356af-06f2-4a9d-ba05-af67173d5d12
  INFO[0023] set main frame                 id=9612C9BCADCD1D4AAA6C9241958BDAD4 parent=<nil>
  INFO[0023] chrome/page/goto: navigating url=https://google.com
  INFO[0024] chrome/frame/waituntil: still waiting for events events=unload id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/setlifecycle: setting lifecycle event=init id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/setlifecycle: set navigating channel
  INFO[0024] chrome/frame/clearexecutioncontext: cleared all execution contexts frame=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chome/page: execution context destroyed: couldn't find the frame id execution=1
  INFO[0024] chrome/frame/addexecutioncontext: added an execution context frame=9612C9BCADCD1D4AAA6C9241958BDAD4 id=2 is_default=true
  INFO[0024] chrome/frame/setlifecycle: setting lifecycle event=firstPaint id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/setlifecycle: setting lifecycle event=firstContentfulPaint id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/setlifecycle: setting lifecycle event=firstTextPaint id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/setlifecycle: setting lifecycle event=firstImagePaint id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/setlifecycle: setting lifecycle event=firstMeaningfulPaintCandidate id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/setlifecycle: setting lifecycle event=firstMeaningfulPaintCandidate id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/setlifecycle: setting lifecycle event=DOMContentLoaded id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/waituntil: event found event=unload id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/waituntil: event found event=DOMContentLoaded id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/waituntil: still waiting for events events=load id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0024] chrome/frame/setlifecycle: setting lifecycle event=load id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0025] chrome/frame/waituntil: event found event=load id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0025] chrome/frame/waituntil: still waiting for events events=unload, DOMContentLoaded, load id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0025] chrome/frame/setlifecycle: setting lifecycle event=init id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0025] chrome/frame/setlifecycle: set navigating channel
  INFO[0025] chrome/frame/setlifecycle: setting lifecycle event=firstMeaningfulPaint id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0025] chrome/frame/setlifecycle: setting lifecycle event=networkAlmostIdle id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0025] chrome/frame/setlifecycle: setting lifecycle event=networkIdle id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0025] chrome/frame/waituntil: event found event=unload id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0025] chrome/frame/waituntil: still waiting for events events=DOMContentLoaded, load id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0025] chrome/frame/setlifecycle: setting lifecycle event=init id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0025] chrome/frame/setlifecycle: set navigating channel
  INFO[0026] chrome/frame/removeexecutioncontext: removing execution context id=2
  INFO[0026] chrome/frame/addexecutioncontext: added an execution context frame=9612C9BCADCD1D4AAA6C9241958BDAD4 id=3 is_default=true
  INFO[0026] chrome/frame/clearexecutioncontext: cleared all execution contexts frame=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0026] chrome/frame/setlifecycle: setting lifecycle event=firstPaint id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0026] chrome/frame/setlifecycle: setting lifecycle event=firstMeaningfulPaintCandidate id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0026] chrome/frame/setlifecycle: setting lifecycle event=firstContentfulPaint id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0026] chrome/frame/setlifecycle: setting lifecycle event=firstTextPaint id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0026] chrome/frame/setlifecycle: setting lifecycle event=DOMContentLoaded id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0026] chrome/frame/setlifecycle: setting lifecycle event=firstImagePaint id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0026] chrome/frame/setlifecycle: setting lifecycle event=load id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0026] chrome/frame/setlifecycle: setting lifecycle event=firstMeaningfulPaintCandidate id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0026] chrome/frame/setlifecycle: setting lifecycle event=firstMeaningfulPaintCandidate id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0027] chrome/frame/waituntil: event found event=DOMContentLoaded id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0027] chrome/frame/waituntil: event found event=load id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0027] chrome/frame/setlifecycle: setting lifecycle event=firstMeaningfulPaint id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0027] chrome/frame/setlifecycle: setting lifecycle event=networkAlmostIdle id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0027] chrome/frame/setlifecycle: setting lifecycle event=networkIdle id=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0027] chrome/page/title: getting title of main frame
  INFO[0027] chrome/frame/defaultexecutioncontext: getting the default execution context id=9612C9BCADCD1D4AAA6C9241958BDAD4

--- FAIL: TestClick (8.56s)
	chrome_test.go:106: chrome/page: unable to get the default execution context: headless/retrier: timed out: chrome/frame: couldn't find the default execution context

From the looks of it, it seems like the most important bit is here:

  INFO[0026] chrome/frame/removeexecutioncontext: removing execution context id=2
  INFO[0026] chrome/frame/addexecutioncontext: added an execution context frame=9612C9BCADCD1D4AAA6C9241958BDAD4 id=3 is_default=true
  INFO[0026] chrome/frame/clearexecutioncontext: cleared all execution contexts frame=9612C9BCADCD1D4AAA6C9241958BDAD4

Oh! You know I bet it’s a race condition between adding new execution contexts and clearing out previous execution contexts 😀

The order should probably be:

  INFO[0026] chrome/frame/removeexecutioncontext: removing execution context id=2
  INFO[0026] chrome/frame/clearexecutioncontext: cleared all execution contexts frame=9612C9BCADCD1D4AAA6C9241958BDAD4
  INFO[0026] chrome/frame/addexecutioncontext: added an execution context frame=9612C9BCADCD1D4AAA6C9241958BDAD4 id=3 is_default=true

Follow up, do we even need to clear the executions? Or can we just let runtime.executionContextDestroyed do it’s thing?

in the project you refer to, package.json has version 1.4.6, whereas the npm has 2.1.1. Is this supposed to be like this?

The version in the package.json is not relevant. The version information is pulled from NPM prior to making a release.

See this explanation, https://github.com/semantic-release/semantic-release#why-is-the-packagejsons-version-not-updated-in-my-repository.

how would this approach work with the documentation? It’s currently hosted on github for all released versions

Whats the current process to publish the documentation?

It would be simply a matter of automating publishing of the documentation after semantic-release script is done.

why installing from github doesn’t work for early adopters?

For the same reason we have semantic versioning in the first place.

Someone who is consuming Puppeteer does not want to be using head because that can introduce a breaking change. Equally, locking to a particular commit provides no way of checking if there is a more recent version.

When projects are setup to properly use semver, updating project dependencies is simply the matter of running a program such as https://www.npmjs.com/package/npm-check-updates. Imagine if every repository started to use your approach – maintaining project dependencies up to date would be a nightmare.

I am getting this error occasionally, and I have no idea where to catch this exception, so my browser hangs.

For now, I am disabling the load of websocket resources (because I don’t need them) to see if that helps.

What solved it for me was changing:

        await page.waitForNavigation({waitUntil: 'networkidle0'});

to

        await page.waitForNavigation();

… in one instance at least

I’ve done:

    page.on('request', (request) => {
        urlChanged++;
    });
    page.on('requestfailed', () => {
        if (urlChanged > 0) {
            urlChanged--;
        }
    });
    page.on('requestfinished', () => {
        if (urlChanged > 0) {
            urlChanged--;
        }
    });
...
async function waitForPageReady(waitUntil = ['networkidle2'], timeOut = 5000)
{
    const timeOutPromise = new Promise(async function (resolve) {
        await setTimeout(() => {
            urlChanged = 0;
            resolve('timedOut');
        }, timeOut);
    });
    const urlChangedPromise = new Promise(async function (resolve) {
        while (urlChanged > 0) {
            await new Promise((innerResolve) => {
                setTimeout(innerResolve, 1000)
            });
        }
        resolve('UrlChanged decreased');
    });
    const result = await Promise.race([timeOutPromise, urlChangedPromise]);
}

and just dotted await waitForPageReady(); around my code when I think the URL may change.

I had the problem where form buttons were being pressed and one of them triggered a page load (submit) but even though the mouse control was on an await, the script continued running and then I tried to screenshot the page but the page hadn’t loaded at that point. Using the above code worked around it.

A heads up for anyone hitting this issue - check through your code to make sure there is not a missing await on an earlier call. This can lead to a race condition and cause the error on a later, seemingly unrelated line.

I hit this issue and reading through the comments, I can see a few people that have a similar possible error in their code, so hopefully that helps someone.

I was digging into this the other day because it was causing issues for us pretty consistently for a site. What triggered it for us was a race condition between page navigation and trying to find an element. What it boiled down to was that a navigation event happened between these two lines. The document object was retrieved from one context, the navigation happened, and then the document object was used to try to find an element but it fails with the error discussed here because the context of the document was destroyed by the navigation.

Adding an explicit wait time before all of this obviously helps, but that’s fairly non-deterministic. Is there an easy way to check if a context has been destroyed without having to actually make a call via devtools protocol? For now, when encountering this error, we just retry what we were trying to do.

@dxinteractive yes, if the page.evaluate causes a navigation, than the execution context where the evaluation is happenning will self-destruct, and page.evaluate call will throw an error.

I you anticipate that your evaluation might trigger a navigation, than I’d recommend to gracefully handle this exception in your script.

@aslushnikov , I am getting Error: Protocol error (Runtime.callFunctionOn): Cannot find context with specified id. when I am selecting in dropdown, but it can able to select still throws error. ssample code is below page.waitForSelector(‘#selector’); page.select(‘#selector’, value);

🎉

I solved this by adding await page.waitForNavigation({waitUntil: ‘load’}); before typing in A couple of updates ocurred which I think made the error start, one being changing puppeteer from 1.2.0 to 1.3.0

As crazy as this might sound, I’ve found a quick workaround for this “Error: Protocol error (Runtime.evaluate): Cannot find context with specified id undefined’” issue.

I was trying to evaluate whatever contents within a page “on load” event handler, but I kept getting this error for one specific website (never happened with hundred of others).

I simply added an await page.waitFor("//html") line, and it fixed it…

initPageOnLoad() {
    this._page.on("load", async () => {

      if(this._onLoadInjectJS) {
        console.log(" [On.pageLoad] Injecting JS dependencies on page loading...");
        await this.injectFile(paths.utils);
      }
    });
}

async injectFile(filePath, page=null) {

    let currentPage = page===null ? this._page : page;

    let contents = await new Promise((resolve, reject) => {
      fs.readFile(filePath, 'utf8', (err, data) => {
        if (err) return reject(err);
        resolve(data);
      });
    });
    contents += `//# sourceURL=` + filePath.replace(/\n/g,'');
    
    // If we remove this line, we would have the undefined context error
    await currentPage.waitFor("//html");
    return currentPage.mainFrame().evaluate(contents);
}

@gajus installing from master gives you the cadence you want…

Might as well not use package manager then, because you are telling users to pull code from the repository.

If you are open to start using semantic-release, I am happy to raise a PR that integrates semantic-release. Alternatively, you can have a look into any of the projects that I maintain, http://github.com/gajus.

Example:

From https://github.com/gajus/isomorphic-webpack

• You need to add the following scripts to travis manifest, https://github.com/gajus/isomorphic-webpack/blob/c6af419769bda05aeacc4f523eaf67687ce93895/.travis.yml#L20.
• Install semantic-release as a dev dependency.
• Add NPM_TOKEN and GH_TOKEN secrets variables to Travis configuration.

@gajus installing from master gives you the cadence you want…

Might as well not use package manager then, because you are telling users to pull code from the repository.

Based on the info, the change probably happened in a Chromium roll between 0.12.0 and 0.13.0. There were a couple of updates in between those releases.

@aslushnikov @JoelEinbinder any weird changes to the protocol/chrome that would be related to this?

For now, I am disabling the load of websocket resources (because I don’t need them) to see if that helps.

Did that help?

@aslushnikov I will add debugging ASAP and report back

Me too, version 1.1.1 seems to still have this issue for me.

@dallashuggins thank you for sharing the details. In this case, the Cannot find context with specified id undefined error seems to be related to the page crashed failure. The page crashed error is a major problem; let’s debug it first.

I’ve created a designated issue for this here: https://github.com/GoogleChrome/puppeteer/issues/1785

@aslushnikov Can this be reopened? The bug still exists and affects the rest of us.

@ggobbe Thanks for your trouble shooting

I also had this error on 0.13.0 It errors on let text = await page.evaluate(() => document.body.textContent) Protocol error (Runtime.callFunctionOn): Cannot find context with specified id undefined at Session._onMessage (node_modules/puppeteer/lib/Connection.js:205:25)

Adding await page.waitFor(4000) gives PASS.

Is there a ‘better practice’ way?

Thanks

this error presented for me when changing passing timeout: 0 to page.launch options. Changing it to a high value (300k, though not sure magnitude matters) solved the problem.

I am in the same situation since I upgraded to v.0.13.0, this error appear before the page even start navigating…