puppeteer-extra: [Bug] bet365.com stopped working 2 days ago

Hello friends,

Here I’ve created a temporary fix for bet365

Run this and it should work

        await page.evaluateOnNewDocument(() => {
            Object.defineProperty(navigator, 'maxTouchPoints', {
                get() {
                    return 1;
                },
            });

        });

Will take a closer look on free time

Removing stealth plugin and adding ‘–disable-blink-features=AutomationControlled’ its working again

Hello friends, I’ve created a new fix.

await page.evaluateOnNewDocument(() => {

    Object.defineProperty(navigator, 'maxTouchPoints', {
        get() {
            "¯\_(ツ)_/¯";
            return 1;
        },
    });

    "✌(-‿-)✌";
    navigator.permissions.query = i => ({then: f => f({state: "prompt", onchange: null})});

});

For Python users, put it to like “Page.addScriptToEvaluateOnNewDocument”

The bet365 site has very strict IP checks. If you open the site with an:

• IP from a country that is not supported: you get this error - https://www.bet365.com/Members/helpers/error.aspx
• IP from a datacenter range: they load a JS script that is doing a bunch of calls to localhost which are blocked by Chrome. I tested a non-puppeteer Chrome and Safari but I have the same issue there.
• IP from a residential range: it hangs loading when using puppeteer, no matter if stealth is enabled or not.

Some more observations:

• Using the bundled Chromium from puppeteer directly works fine:

./node_modules/puppeteer/.local-chromium/mac-818858/chrome-mac/Chromium.app/Contents/MacOS/Chromium https://www.bet365.com/ --no-first-run --user-data-dir=$(mktemp)

• Connecting puppeteer to an existing running browser (started with --remote-debugging-port=9992) with puppeteer.connect({browserURL: 'http://localhost:9992'}): it’s again stuck on loading the site. This means they detect something in the way the page is opened.
• If you open an existing page (so not opened by puppeteer) in a browser opened with stealth (to avoid the webdriver property) it works fine - obviously page-level stealth evasions are not applied but it’s working fine. Here’s working sample code:

const puppeteer = require('puppeteer-extra')
puppeteer.use(require('puppeteer-extra-plugin-stealth')())

puppeteer.launch({ headless: false }).then(async browser => {
  const page = (await browser.pages())[0];

  await page.goto('https://www.bet365.com/');
});

We need to investigate what they detect exactly in newly opened pages.

@berstend Yes,I tried all the alternatives on this topic, until today in the morning it was working perfectly.

None of this you mentioned in your first comment, I recommend adding as much context as possible if you depend on others helping you.

example simulating the problem.

Try to purge or remove userDataDir .

The programmer of bet365 is monitoring this issue. Everyone please do not update solution here.

@FabianoDevX either you didn’t try the querySelector workaround, you didn’t read the other comments here or you didn’t mention trying it - none of these are good netiquette. 😉

Im having troubles with this workaround…

When I try to make an a evaluate with queryselector or queryselectorall the pages freeze automatically and chromium begin to leak memory as fuck. I cant use the console webbrowser, i cant retrieve any data from the website…

Anyone has this trouble?

Try on await page.evaluateOnNewDocument(() => { add windows.qs = document.querySelector

then, on every evaluate

.evaluate(()=>{
document.querySelector = windows.qs
document.querySelector('...')

Friends, you can also use this one

            let n = 1;
            Object.defineProperty(navigator, 'maxTouchPoints', {
                get() {
                    setTimeout(() => n = 0, 0);
                    return n;
                },
            });

Im having troubles with this workaround…

When I try to make an a evaluate with queryselector or queryselectorall the pages freeze automatically and chromium begin to leak memory as fuck. I cant use the console webbrowser, i cant retrieve any data from the website…

Anyone has this trouble?

It’s not working for me. Can you share the code you’re using, please?

const puppeteer = require(‘puppeteer-extra’); const StealthPlugin = require(‘puppeteer-extra-plugin-stealth’); const stealth = StealthPlugin(); stealth.enabledEvasions.delete(‘chrome.runtime’) stealth.enabledEvasions.delete(‘iframe.contentWindow’) puppeteer.use(stealth);

puppeteer.launch({ executablePath: ‘/usr/bin/chromium-browser’, headless: true, args: [‘–no-sandbox’, ‘–disable-blink-features=AutomationControlled’] }).then(async browser => {

console.log('Running tests..');
const page = await browser.newPage();

await page.goto('https://bet365.com');

await page.evaluateOnNewDocument(() => {
    Object.defineProperty(navigator, 'maxTouchPoints', {
        get() {
            return 1;
        },
    });

});

await page.screenshot({ path: 'bet365.png', fullPage: true });
await browser.close();
console.log('All done, check the screenshot.');

});

You need to run await page.evaluateOnNewDocument(() => { before await page.goto('https://bet365.com');

Workaround for the time being:

const stealth = StealthPlugin();
stealth.enabledEvasions.delete('chrome.runtime')
stealth.enabledEvasions.delete('iframe.contentWindow')
puppeteer.use(stealth);