isbot: "Invalid regular expression" on older ES versions (lack of negative lookbehind support)

Steps to reproduce

create a file with the regex from the release:

var regex = / daum[ /]| deusu\/| yadirectfetcher|(?:^| )site|(?:^|[^g])news|(?<! (?:channel\/|google\/))google(?!(app|\/google| pixel))|(?<! cu)bot(?:[^\w]|_|$)|(?<! ya(?:yandex)?)search|(?<!(?:lib))http|(?<![hg]m)score|@[a-z]|\(at\)[a-z]|\(github\.com\/|\[at\][a-z]|^12345|^<|^[\w \.\-\(?:\):]+(?:\/v?\d+(\.\d+)?(?:\.\d{1,10})?)?(?:,|$)|^[^ ]{50,}$|^active|^ad muncher|^amaya|^anglesharp\/|^avsdevicesdk\/|^bidtellect\/|^biglotron|^bot|^btwebclient\/|^clamav[ /]|^client\/|^cobweb\/|^coccoc|^custom|^ddg[_-]android|^discourse|^dispatch\/\d|^downcast\/|^duckduckgo|^facebook|^fdm[ /]\d|^getright\/|^gozilla\/|^hatena|^hobbit|^hotzonu|^hwcdn\/|^jeode\/|^jetty\/|^jigsaw|^linkdex|^metauri|^microsoft bits|^movabletype|^mozilla\/\d\.\d \(compatible;?\)$|^mozilla\/\d\.\d \w*$|^navermailapp|^netsurf|^offline explorer|^php|^postman|^postrank|^python|^rank|^read|^reed|^rest|^snapchat|^space bison|^svn|^swcd |^taringa|^thumbor\/|^tumblr\/|^user-agent:|^valid|^venus\/fedoraplanet|^w3c|^webbandit\/|^webcopier|^wget|^whatsapp|^xenu link sleuth|^yahoo|^yandex|^zdm\/\d|^zoom marketplace\/|^{{.*}}$|adbeat\.com|appinsights|archive|ask jeeves\/teoma|bit\.ly\/|bluecoat drtr|browsex|burpcollaborator|capture|catch|check|chrome-lighthouse|chromeframe|classifier|cloud|crawl|cryptoapi|dareboost|datanyze|dataprovider|dejaclick|dmbrowser|download|evc-batch\/|feed|firephp|freesafeip|gomezagent|headless|httrack|hubspot marketing grader|hydra|ibisbrowser|images|inspect|iplabel|ips-agent|java(?!;)|library|mail\.ru\/|manager|monitor|neustar wpm|nutch|offbyone|optimize|pageburst|pagespeed|parser|perl|phantom|pingdom|powermarks|preview|proxy|ptst[ /]\d|reader|reputation|resolver|retriever|rexx;|rigor|robot|rss|scan|scrape|server|sogou|sparkler\/|speedcurve|spider|splash|statuscake|stumbleupon\.com|supercleaner|synapse|synthetic|torrent|tracemyfile|transcoder|twingly recon|url|virtuoso|wappalyzer|webglance|webkit2png|whatcms\/|wordpress|zgrab/i;

run es-check es2017 on that file

Expected behaviour

It should pass the check, or the README should explicitely say that older browsers are not supported.

Actual behaviour

It does not pass the check

About this issue

Original URL
State: closed
Created 6 months ago
Reactions: 1
Comments: 17 (9 by maintainers)

Most upvoted comments

Can confirm, isbot@5 is working now for my Safari 15.5 use-case. Thanks again for the quick turnaround on that!

bfaulk96 on Feb 16, 2024

Thank you for fixing this so quickly, this makes it possible for us to upgrade to v4 again. 😃

akselinurmio on Jan 17, 2024

Version 4.4.0 should be okay when importing isbot.

import { isbot } from  "isbot"

I greatly appreciate your involvement in this issue. It helped me reach a better outcome.

omrilotan on Jan 12, 2024

That’s a very good question. The non-lookbehind patterns include some false-positives. For example, a lot of android apps include the webview substring “Channel/googleplay”, where the simplistic “google” substring pattern will recognise as a bot. That’s why I think supporting version 3 pattern updates is not a priority.

Regular expression lookbehind is now widely supported.

True, although Safari only started supporting it in 2023…

A quarter of our users seem to be on older browser versions (ie over 4 versions old), for which we serve a less optimized bundle. As we’re providing a platform for social change, we want to be mindful of the fact that not everyone have access to modern browsers, but we still want them to be able to use the platform.

Instead - I might create a shorter, more simplistic and more optimistic option for engines that do not support lookbehind as a fallback. This could mitigate your case as well.

For example:
let pattern;
try {
 pattern = new Regexp("(?<! cu)bot(?:[^\\w]|_|$)", "i");
} catch (error) {
 pattern = /bot/i;
}
What is your opinion about this feature?

I like this idea!

jlowcs on Jan 9, 2024