puppeteer: ignoreHTTPSErrors is not working when request interception is on

I ran into the same problem, worked around it by doing the following:

const puppeteer = require('puppeteer');

async function visit() {
    const browser = await puppeteer.launch({
        args: [
            '--disable-setuid-sandbox',
            '--no-sandbox',
            '--ignore-certificate-errors',
        ],
        ignoreHTTPSErrors: true,
        headless: true,
    });

    // this initial visit is meant to bypass the certificate errors; don't intercept anything yet.
    var url = "SOME_URL_ON_THE_SAME_DOMAIN.com"
    const page = await browser.newPage();
    await page.setViewport({width: 1300, height: 1000});
    await page.goto(url); 

    // intercept requests; at this point we've already bypassed the certificate errors,
    // so the subsequent page visit will load.
    await page.setRequestInterception(true);
    await page.on('request', interceptedRequest => {
        console.log(interceptedRequest.url());
        interceptedRequest.continue();
    });

    // visit the actual page you want to do stuff on.
    var actualURL = "SOME_URL_ON_THE_SAME_DOMAIN.com/ACTUAL_PAGE"
    await page.goto(actualURL, {waitUntil: 'networkidle0'});
    await browser.close();
}

However, even with this work-around, I still think that the fix for this issue is super high priority. Thanks!

Is there any progress?

What the shit, after so many bloody years, still the same issue.

This is a major problemo. Hope it gets fixed soon.

@maZahaca: pptr 1.8.0 will be released on September, 6.

This bug will be fixed once Chromium completes migration onto Network Service.

As a workaround, network service could be enabled via a runtime flag --enable-feature=NetworkService.

Beware! NetworkService is not completed yet and doesn’t pass all pptr tests.

Still, the following works for me:

const puppeteer = require('puppeteer');

const URL = "https://halva.khady.info/";

(async() => {
  const browser = await puppeteer.launch({
    args: ['--enable-features=NetworkService'],
    headless: true,
    ignoreHTTPSErrors: true,
  });
  const page = await browser.newPage();
  await page.setRequestInterception(true);
  page.on("request", (request) => {
    if (request.resourceType === "Image")
      request.abort();
    else
      request.continue();
  });
  await page.goto(URL, { timeout: 8000, waitUntil: "load" });
  const html = await page.content();
  console.log(html);
  await page.close();
  await browser.close();
})();

Is there any progress?

When I use this args: --enable-feature=NetworkService then --proxy-server=xxx is not working!

seems like I use ‘–no-sandbox’，and proxy is not working, https://groups.google.com/a/chromium.org/forum/#!topic/network-service-dev/qr7yCQeoT4o

browser = await puppeteer.launch({
      ignoreHTTPSErrors: true,
      args: [
        '--no-sandbox',
        '--disable-dev-shm-usage',
        '--enable-features=NetworkService',
        '--proxy-server=' + proxyUrl
      ]
    });

as @bluepeter mentioned its still an issue. Its not working on google cloud which is Linux based Any workarounds?

#1441 same as yours

use this

args: [
    '--ignore-certificate-errors',
    '--ignore-certificate-errors-spki-list '
]

I am having same problem with https site, while running puppeteer in docker

Wow Wow Wow. The original issue from Chromium was just fixed! Yay!

Let’s get this stuff moving…

Why is it closed? Isn’t it gonna be resolved?

This work only when you launch puppeteer with headless: false. See https://github.com/GoogleChrome/puppeteer/issues/1159#issuecomment-343421349

BTW, the --ignore-certificate-errors-spki-list flag is not doing anything if you don’t use the --user-data-dir flag according to this documentation.

Just tried --enable-feature=NetworkService with Puppeteer 1.6.2 and accompanying Chromium, but it’s still not working.

I tested using the URL http://finishline.com which hangs when using with await page.setRequestInterception(true); and there was no difference using --enable-feature=NetworkService as without it. Puppeteer just hangs up until goto timeout ends,

browser launch: browser = await puppeteer.launch({ args: ['--enable-features=NetworkService'], headless: true, timeout: 30000, ignoreHTTPSErrors: true });

goto: await page.goto(url, { timeout: 60000, waitUntil: 'load' })

If you can use puppeteer without chrome headless, a solution seems to be the --ignore-certificate-errors flags. Unfortunately it doesn’t work with chrome headless.

Seems to still be an issue setRequestInterception(true) in docker as well when using the following setup:

  var executablePath = inDocker ?  : undefined;
  return launch({
    headless: true,
    ["--no-sandbox", "--disable-setuid-sandbox", "--disable-dev-shm-usage"] ,
    "/usr/bin/chromium-browser",
    defaultViewport: { width: 1000, height: 1600 },
    // dumpio: false,
    // ignoreHTTPSErrors: true
  });

@aslushnikov puppeteer-firefox has this issue still. Is there a separate thread for it?

Note: this is still an issue for those of us trying to run on Linux. See: https://bugs.chromium.org/p/chromium/issues/detail?id=877075

Great job!