puppeteer-extra: [Bug] CloudFlare protection (https://nowsecure.nl) bypass on local laptop (Windows 11, display), but not on remote server (Debian + xvfb)

Describe the bug

I’m trying to bypass CloudFlare protection, using for test https://nowsecure.nl/ website My simple code works well on my local laptop (Windows 11 is used, with real physycal display of course), but did not work on remote server (Debian + xvfb) Mode headless: "new" is used

* This is not a problem with blacklisted IP address of the remote server: I was trying to use paid proxies and get the same result on both environments

Code Snippet

const puppeteer = require('puppeteer-extra')
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
puppeteer.use(StealthPlugin())
const {executablePath} = require('puppeteer')

puppeteer.launch({
    executablePath: executablePath(),
    headless: "new",
}).then(async browser => {
    console.log('Running tests..')
    const page = await browser.newPage()

    await page.goto('https://nowsecure.nl')
    await page.waitForTimeout(10000)

    await page.screenshot({
        path: 'stealth_test__headless-false.png',
        fullPage: true
    })
    await browser.close()
    console.log(`All done, check the screenshot`)
})

Versions, results & console output

Local:

  System:
    OS: Windows 10 10.0.22621
    CPU: (32) x64 13th Gen Intel(R) Core(TM) i9-13900HX
    Memory: 12.84 GB / 31.75 GB
  Binaries:
    Node: 18.16.0 - C:\Program Files\nodejs\node.EXE
    npm: 6.14.18 - D:\OSPanel\domains\************\node_modules\.bin\npm.CMD
  npmPackages:
    puppeteer: ^19.5.2 => 19.11.1
    puppeteer-extra: ^3.3.4 => 3.3.6
    puppeteer-extra-plugin-anonymize-ua: ^2.4.4 => 2.4.6
    puppeteer-extra-plugin-stealth: ^2.11.1 => 2.11.2
Screenshot with result: protection is passed

stealth_test__headless-false

Debug info from console 
$ DEBUG=puppeteer-extra,puppeteer-extra-plugin:* node resources/puppeteer/puppeteer-extra-plugin-stealth.js

  puppeteer-extra-plugin:base:stealth Initialized. +0ms
  puppeteer-extra plugin registered stealth +0ms
  puppeteer-extra dependencies missing Set(16) {
  'stealth/evasions/chrome.app',
  'stealth/evasions/chrome.csi',
  'stealth/evasions/chrome.loadTimes',
  'stealth/evasions/chrome.runtime',
  'stealth/evasions/defaultArgs',
  'stealth/evasions/iframe.contentWindow',
  'stealth/evasions/media.codecs',
  'stealth/evasions/navigator.hardwareConcurrency',
  'stealth/evasions/navigator.languages',
  'stealth/evasions/navigator.permissions',
  'stealth/evasions/navigator.plugins',
  'stealth/evasions/navigator.webdriver',
  'stealth/evasions/sourceurl',
  'stealth/evasions/user-agent-override',
  'stealth/evasions/webgl.vendor',
  'stealth/evasions/window.outerdimensions'
} +3ms
  puppeteer-extra-plugin:base:stealth/evasions/chrome.app Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/chrome.app +5ms
  puppeteer-extra-plugin:base:stealth/evasions/chrome.csi Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/chrome.csi +2ms
  puppeteer-extra-plugin:base:stealth/evasions/chrome.loadTimes Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/chrome.loadTimes +2ms
  puppeteer-extra-plugin:base:stealth/evasions/chrome.runtime Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/chrome.runtime +2ms
  puppeteer-extra-plugin:base:stealth/evasions/defaultArgs Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/defaultArgs +1ms
  puppeteer-extra-plugin:base:stealth/evasions/iframe.contentWindow Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/iframe.contentWindow +2ms
  puppeteer-extra-plugin:base:stealth/evasions/media.codecs Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/media.codecs +2ms
  puppeteer-extra-plugin:base:stealth/evasions/navigator.hardwareConcurrency Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/navigator.hardwareConcurrency +2ms
  puppeteer-extra-plugin:base:stealth/evasions/navigator.languages Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/navigator.languages +2ms
  puppeteer-extra-plugin:base:stealth/evasions/navigator.permissions Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/navigator.permissions +1ms
  puppeteer-extra-plugin:base:stealth/evasions/navigator.plugins Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/navigator.plugins +6ms
  puppeteer-extra-plugin:base:stealth/evasions/navigator.webdriver Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/navigator.webdriver +2ms
  puppeteer-extra-plugin:base:stealth/evasions/sourceurl Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/sourceurl +2ms
  puppeteer-extra-plugin:base:stealth/evasions/user-agent-override Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/user-agent-override +2ms
  puppeteer-extra dependencies missing Set(3) {
  'stealth/evasions/webgl.vendor',
  'stealth/evasions/window.outerdimensions',
  'user-preferences'
} +1ms
  puppeteer-extra-plugin:base:stealth/evasions/webgl.vendor Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/webgl.vendor +2ms
  puppeteer-extra-plugin:base:stealth/evasions/window.outerdimensions Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/window.outerdimensions +2ms
  puppeteer-extra-plugin:base:user-preferences Initialized. +0ms
  puppeteer-extra plugin registered user-preferences +3ms
  puppeteer-extra dependencies missing Set(1) { 'user-data-dir' } +0ms
  puppeteer-extra-plugin:base:user-data-dir Initialized. +0ms
  puppeteer-extra-plugin:user-data-dir initialized {
  deleteTemporary: true,
  deleteExisting: false,
  files: [],
  folderPath: 'C:\\Users\\Alex\\AppData\\Local\\Temp',
  folderPrefix: 'puppeteer_dev_profile-'
} +0ms
  puppeteer-extra plugin registered user-data-dir +47ms
  puppeteer-extra ignoring dependency 'stealth/evasions/webgl.vendor', which has been required already. +0ms
  puppeteer-extra ignoring dependency 'stealth/evasions/window.outerdimensions', which has been required already.
 +0ms
  puppeteer-extra orderPlugins:before [
  'stealth',
  'stealth/evasions/chrome.app',
  'stealth/evasions/chrome.csi',
  'stealth/evasions/chrome.loadTimes',
  'stealth/evasions/chrome.runtime',
  'stealth/evasions/defaultArgs',
  'stealth/evasions/iframe.contentWindow',
  'stealth/evasions/media.codecs',
  'stealth/evasions/navigator.hardwareConcurrency',
  'stealth/evasions/navigator.languages',
  'stealth/evasions/navigator.permissions',
  'stealth/evasions/navigator.plugins',
  'stealth/evasions/navigator.webdriver',
  'stealth/evasions/sourceurl',
  'stealth/evasions/user-agent-override',
  'stealth/evasions/webgl.vendor',
  'stealth/evasions/window.outerdimensions',
  'user-preferences',
  'user-data-dir'
] +1ms
  puppeteer-extra orderPlugins:after [
  'stealth',
  'stealth/evasions/chrome.app',
  'stealth/evasions/chrome.csi',
  'stealth/evasions/chrome.loadTimes',
  'stealth/evasions/chrome.runtime',
  'stealth/evasions/media.codecs',
  'stealth/evasions/navigator.hardwareConcurrency',
  'stealth/evasions/navigator.languages',
  'stealth/evasions/navigator.permissions',
  'stealth/evasions/navigator.plugins',
  'stealth/evasions/navigator.webdriver',
  'stealth/evasions/sourceurl',
  'stealth/evasions/user-agent-override',
  'stealth/evasions/webgl.vendor',
  'stealth/evasions/window.outerdimensions',
  'stealth/evasions/defaultArgs',
  'stealth/evasions/iframe.contentWindow',
  'user-preferences',
  'user-data-dir'
] +1ms
  puppeteer-extra-plugin:user-preferences _userPrefsFromPlugins { intl: { accept_languages: 'en-US,en' } } +0ms
  puppeteer-extra-plugin:user-data-dir created custom dir C:\Users\Alex\AppData\Local\Temp\puppeteer_dev_profile-
wnWjGk +13ms
  puppeteer-extra-plugin:user-data-dir Wrote file C:\Users\Alex\AppData\Local\Temp\puppeteer_dev_profile-wnWjGk\D
efault\Preferences +6ms
Running tests..
  puppeteer-extra-plugin:stealth/evasions/user-agent-override onPageCreated - Will set these user agent options {
  override: {
    userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0
 Safari/537.36',
    platform: 'Win32',
    userAgentMetadata: {
      brands: [Array],
      fullVersion: '112.0.0.0',
      platform: 'Windows',
      platformVersion: '10.0',
      architecture: 'x86',
      model: '',
      mobile: false
    },
    acceptLanguage: 'en-US,en'
  },
  opts: { userAgent: null, locale: 'en-US,en', maskLinux: true }
} +0ms
  puppeteer-extra-plugin:user-data-dir onDisconnected +8s
  puppeteer-extra-plugin:user-data-dir removeUserDataDir C:\Users\Alex\AppData\Local\Temp\puppeteer_dev_profile-w
nWjGk +1ms
All done, check the screenshot: storage/stealth_test.png
  puppeteer-extra-plugin:user-data-dir null +441ms

Remote:

  System:
    OS: Linux 5.15 Debian GNU/Linux 11 (bullseye) 11 (bullseye)
    CPU: (8) x64 AMD Ryzen 9 5950X 16-Core Processor
    Memory: 22.94 GB / 32.00 GB
    Container: Yes
    Shell: 5.1.4 - /bin/bash
  Binaries:
    Node: 19.3.0 - /usr/bin/node
    npm: 6.14.18 - /var/www/************/node_modules/.bin/npm
  npmPackages:
    puppeteer: ^19.5.2 => 19.11.1 
    puppeteer-extra: ^3.3.4 => 3.3.6 
    puppeteer-extra-plugin-anonymize-ua: ^2.4.4 => 2.4.6 
    puppeteer-extra-plugin-stealth: ^2.11.1 => 2.11.2
Screenshot with result: protection is NOT passed

stealth_test__headless-true

Debug info from console 
# sudo -u www-data DEBUG=puppeteer-extra,puppeteer-extra-plugin:* node resources/puppeteer/puppeteer-extra-plugin-stealth.js

  puppeteer-extra-plugin:base:stealth Initialized. +0ms
  puppeteer-extra plugin registered stealth +0ms
  puppeteer-extra dependencies missing Set(16) {
  'stealth/evasions/chrome.app',
  'stealth/evasions/chrome.csi',
  'stealth/evasions/chrome.loadTimes',
  'stealth/evasions/chrome.runtime',
  'stealth/evasions/defaultArgs',
  'stealth/evasions/iframe.contentWindow',
  'stealth/evasions/media.codecs',
  'stealth/evasions/navigator.hardwareConcurrency',
  'stealth/evasions/navigator.languages',
  'stealth/evasions/navigator.permissions',
  'stealth/evasions/navigator.plugins',
  'stealth/evasions/navigator.webdriver',
  'stealth/evasions/sourceurl',
  'stealth/evasions/user-agent-override',
  'stealth/evasions/webgl.vendor',
  'stealth/evasions/window.outerdimensions'
} +1ms
  puppeteer-extra-plugin:base:stealth/evasions/chrome.app Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/chrome.app +2ms
  puppeteer-extra-plugin:base:stealth/evasions/chrome.csi Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/chrome.csi +0ms
  puppeteer-extra-plugin:base:stealth/evasions/chrome.loadTimes Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/chrome.loadTimes +1ms
  puppeteer-extra-plugin:base:stealth/evasions/chrome.runtime Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/chrome.runtime +1ms
  puppeteer-extra-plugin:base:stealth/evasions/defaultArgs Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/defaultArgs +0ms
  puppeteer-extra-plugin:base:stealth/evasions/iframe.contentWindow Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/iframe.contentWindow +0ms
  puppeteer-extra-plugin:base:stealth/evasions/media.codecs Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/media.codecs +1ms
  puppeteer-extra-plugin:base:stealth/evasions/navigator.hardwareConcurrency Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/navigator.hardwareConcurrency +0ms
  puppeteer-extra-plugin:base:stealth/evasions/navigator.languages Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/navigator.languages +1ms
  puppeteer-extra-plugin:base:stealth/evasions/navigator.permissions Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/navigator.permissions +0ms
  puppeteer-extra-plugin:base:stealth/evasions/navigator.plugins Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/navigator.plugins +1ms
  puppeteer-extra-plugin:base:stealth/evasions/navigator.webdriver Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/navigator.webdriver +1ms
  puppeteer-extra-plugin:base:stealth/evasions/sourceurl Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/sourceurl +0ms
  puppeteer-extra-plugin:base:stealth/evasions/user-agent-override Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/user-agent-override +1ms
  puppeteer-extra dependencies missing Set(3) {
  'stealth/evasions/webgl.vendor',
  'stealth/evasions/window.outerdimensions',
  'user-preferences'
} +0ms
  puppeteer-extra-plugin:base:stealth/evasions/webgl.vendor Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/webgl.vendor +0ms
  puppeteer-extra-plugin:base:stealth/evasions/window.outerdimensions Initialized. +0ms
  puppeteer-extra plugin registered stealth/evasions/window.outerdimensions +1ms
  puppeteer-extra-plugin:base:user-preferences Initialized. +0ms
  puppeteer-extra plugin registered user-preferences +1ms
  puppeteer-extra dependencies missing Set(1) { 'user-data-dir' } +0ms
  puppeteer-extra-plugin:base:user-data-dir Initialized. +0ms
  puppeteer-extra-plugin:user-data-dir initialized {
  deleteTemporary: true,
  deleteExisting: false,
  files: [],
  folderPath: '/tmp',
  folderPrefix: 'puppeteer_dev_profile-'
} +0ms
  puppeteer-extra plugin registered user-data-dir +16ms
  puppeteer-extra ignoring dependency 'stealth/evasions/webgl.vendor', which has been required already. +0ms
  puppeteer-extra ignoring dependency 'stealth/evasions/window.outerdimensions', which has been required already. +0ms
  puppeteer-extra orderPlugins:before [
  'stealth',
  'stealth/evasions/chrome.app',
  'stealth/evasions/chrome.csi',
  'stealth/evasions/chrome.loadTimes',
  'stealth/evasions/chrome.runtime',
  'stealth/evasions/iframe.contentWindow',
  'stealth/evasions/media.codecs',
  'stealth/evasions/navigator.hardwareConcurrency',
  'stealth/evasions/navigator.languages',
  'stealth/evasions/navigator.permissions',
  'stealth/evasions/navigator.plugins',
  'stealth/evasions/navigator.webdriver',
  'stealth/evasions/sourceurl',
  'stealth/evasions/user-agent-override',
  'stealth/evasions/webgl.vendor',
  'stealth/evasions/window.outerdimensions',
  'user-preferences',
  'user-data-dir'
] +0ms
  puppeteer-extra orderPlugins:after [
  'stealth',
  'stealth/evasions/chrome.app',
  'stealth/evasions/chrome.csi',
  'stealth/evasions/chrome.loadTimes',
  'stealth/evasions/chrome.runtime',
  'stealth/evasions/media.codecs',
  'stealth/evasions/navigator.hardwareConcurrency',
  'stealth/evasions/navigator.languages',
  'stealth/evasions/navigator.permissions',
  'stealth/evasions/navigator.plugins',
  'stealth/evasions/navigator.webdriver',
  'stealth/evasions/sourceurl',
  'stealth/evasions/user-agent-override',
  'stealth/evasions/webgl.vendor',
  'stealth/evasions/window.outerdimensions',
  'stealth/evasions/defaultArgs',
  'stealth/evasions/iframe.contentWindow',
  'user-preferences',
  'user-data-dir'
] +0ms
  puppeteer-extra-plugin:user-preferences _userPrefsFromPlugins { intl: { accept_languages: 'en-US,en' } } +0ms
  puppeteer-extra-plugin:user-data-dir created custom dir /tmp/puppeteer_dev_profile-DmMi0c +4ms
  puppeteer-extra-plugin:user-data-dir Wrote file /tmp/puppeteer_dev_profile-DmMi0c/Default/Preferences +5ms
Running tests..
  puppeteer-extra-plugin:stealth/evasions/user-agent-override onPageCreated - Will set these user agent options {
  override: {
    userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36',    platform: 'Win32',
    userAgentMetadata: {
      brands: [Array],
      fullVersion: '112.0.0.0',
      platform: 'Windows',
      platformVersion: '10.0',
      architecture: 'x86',
      model: '',
      mobile: false
    },
    acceptLanguage: 'en-US,en'
  },
  opts: { userAgent: null, locale: 'en-US,en', maskLinux: true }
} +0ms
  puppeteer-extra-plugin:user-data-dir onDisconnected +6s
  puppeteer-extra-plugin:user-data-dir removeUserDataDir /tmp/puppeteer_dev_profile-DmMi0c +0ms
All done, check the screenshot: storage/stealth_test.png
  puppeteer-extra-plugin:user-data-dir null +212ms

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Reactions: 12
  • Comments: 76 (1 by maintainers)

Most upvoted comments

It took some time to identify this issue, but I finally found a simple fix without any changes to Puppeteer.

Just add the following to puppeteer.connect or puppeteer.launch options:

targetFilter: (target) => !!target.url()

This does prevent puppeteer from clicking specific elements inside the captcha iframe, but directly clicking the iframe to pass the captcha works perfectly fine in my tests.

+11
fuzeman on Aug 8, 2023

Hey everyone, I believe this thing might be useful for you. https://github.com/CheshireCaat/puppeteer-with-fingerprints

+11
putuoka on Jul 27, 2023

Even though this is definitely not a great solution - I’ve found a workaround to get past Cloudflare for now.

They somehow detect if the browser is controlled by an automation script. So, the simplest approach would be to just… not do that. If we manually launch our browser with the --remote-debugging-port=PORT flag and connect to it via Puppeteer.connect we can Page.goto to our site of choice and if we call Browser.disconnect immediatly after Page.goto Cloudflare will let us through, as the browser is no longer controlled by Puppeteer.

After that we just need to confirm we actually got past Cloudflare (there are probably multiple better ways to accomplish this, I just open another headless browser and pull some data from http://127.0.0.1:PORT/json/list to see if it matches the site I’m trying to access). Then we can reconnect to our browser again using Puppeteer.connect and use Puppeteer like we normally would.

You could also inject javascript into the page before you disconnect using Page.evaluateOnNewDocument.

This probably doesn’t work well for all usecases, but for what I’m doing it’s okay. //Edit: Forgot to mention, I’m doing this with headful chromium.

+7
Av4aR on Jul 21, 2023

No need for Discord. There it is :

import puppeteer from 'puppeteer-extra';
import createPuppeteerStealth from 'puppeteer-extra-plugin-stealth';

puppeteer.use(createPuppeteerStealth());

const
    browser = await puppeteer.launch({
        headless: false,
        targetFilter: target => target.type() !== 'other'
    }),
    defaultPage = (await browser.pages())[0]; // <-- bypasses Cloudflare

await defaultPage.goto('https://1337x.to');
await defaultPage.waitForSelector('#autocomplete');
await new Promise(resolve => setTimeout(resolve, 1000));

const newPage = await browser.newPage(); // <-- fails to bypass Cloudflare

await newPage.goto('https://1337x.to');

+5
KaKi87 on Mar 2, 2024

@Linzh1998 hmm, it’s seems like your solution for manual solving of the captcha is working in combination with other advises from this thread

This is my modified snippet which allow to pass Cloudflare protection and solve the captcha on the local laptop (Windows) and remote server (Debian & xvfb) at this moment:

/**
 * @see https://github.com/berstend/puppeteer-extra/issues/817
 */
const puppeteer = require('puppeteer-extra')
const {executablePath} = require('puppeteer')

const StealthPlugin = require('puppeteer-extra-plugin-stealth')
const stealth = StealthPlugin()
puppeteer.use(stealth)

let resultImagePath = 'storage/stealth_test.png'

puppeteer.launch({
    executablePath: executablePath(),
    readTimeout: 5 * 60 * 1000,
    headless: 'new',
    targetFilter: (target) => !!target.url,
}).then(async browser => {
    console.log('Running tests..')
    let page = (await browser.pages())[0]; // Instead of using browser.newPage()

    await page.goto('https://nowsecure.nl')
    await page.waitForTimeout(5000)

    // try to solve the captcha manually
    console.log(`Try to solve the captcha manually...`)

    await page.click('body')
    await page.waitForTimeout(500)

    await page.keyboard.press('Tab')
    await page.waitForTimeout(500)

    await page.keyboard.press('Space')
    await page.waitForTimeout(10000)
    // finish

    await page.screenshot({
        path: resultImagePath,
        fullPage: true
    })
    await browser.close()
    console.log(`All done, check the screenshot: ` + resultImagePath)
})
+5
sngrl2 on Oct 11, 2023

Something is inconsistent in your fingerprint. So you get captcha. This is probably because you use Linux but pretend to be windows.

Indeed, I’m not encountering the issue on Windows.

get list of enabled evasions and disable them one by one until find what will pass you I would start with removal of iframe evasion

That wasn’t the one, but as you mentioned, puppeteer-extra generates a Windows user agent despite being on Linux, so I deleted user-agent-override and am no longer experiencing the issue.

Thanks you ❤️

import puppeteer from 'puppeteer-extra';
import createPuppeteerStealth from 'puppeteer-extra-plugin-stealth';

const puppeteerStealth = createPuppeteerStealth();
puppeteerStealth.enabledEvasions.delete('user-agent-override');
puppeteer.use(puppeteerStealth);

const
    browser = await puppeteer.launch({
        headless: false,
        targetFilter: target => target.type() !== 'other'
    }),
    newPage = await browser.newPage();
await newPage.goto('https://1337x.to');
+3
KaKi87 on Mar 2, 2024

@fuzeman

It took some time to identify this issue, but I finally found a simple fix without any changes to Puppeteer.

Just add the following to puppeteer.connect or puppeteer.launch options:

targetFilter: (target) => !!target.url()

This does prevent puppeteer from clicking specific elements inside the captcha iframe, but directly clicking the iframe to pass the captcha works perfectly fine in my tests.

After 3 months Cloudflare started to show captcha on my local laptop, and your snippet helps me to solve it 🥳 But not on remote server with Debian & xvfb

+3
sngrl2 on Oct 11, 2023

There is a special trick they use to prevent you from clicking in iframe via puppeteer. They detect that you disable cors.

+2
vladtreny on Apr 15, 2024

@sngrl2 I found that headless: false passes just fine. If I set headless: ‘new’ it stops to work even on Windows. So, I think there is a difference between headless and non-headless modes.

+1
AHgPuK on Jan 30, 2024

Hello, https://github.com/zfcsoftware/youtube_lessons_resources/blob/main/cloudflare_bypass/index.js You can get cookie and agent with cloudflare scraper like here and send requests with that information. If you need a proxy: https://github.com/zfcsoftware/puppeteer-real-browser you can use this.

+1
zfcsoftware on Jan 3, 2024

This does not work for me locally (Windows). I tried the newest puppeteer version and multiple different configurations which helped previously. Seems to me like Cloudflare more or less recently found another way of detecting puppeteer.

+1
atalantus on Oct 11, 2023

@luluhoc Looks like targetFilter was changed in puppeteer v21 (https://github.com/puppeteer/puppeteer/commit/44712d1e6efcb3fa49c27b1195d17c0c1c92a0ca).

I think this will work with older versions of puppeteer (haven’t tested myself):


targetFilter: (target) => !!target.url

@fuzeman the solution doesn’t give me error anymore but it doesn’t allow me to pass CF

+1
luluhoc on Aug 8, 2023

@luluhoc Looks like targetFilter was changed in puppeteer v21 (https://github.com/puppeteer/puppeteer/commit/44712d1e6efcb3fa49c27b1195d17c0c1c92a0ca).

I think this will work with older versions of puppeteer (haven’t tested myself):

targetFilter: (target) => !!target.url
+1
fuzeman on Aug 8, 2023

I found why, i was creating a new browser and then a new tab where i go to the protected link. By reusing the default about:blank page it’s working. let page = (await browser.pages())[0]; // Instead of using browser.newPage()

+1
Roncarlos on Jul 18, 2023
Read more comments on GitHub
← puppeteer-extra: [Bug] captcha error: TypeError [ERR_INVALID_PROTOCOL]: Protocol "https:" not supported. Expected "http:"
puppeteer-extra: [Bug] devtools plugin not working in headless mode →