torrent-search-api: Yggtorrent - CaptchaError: captcha

It’s not that simple, the alternative package (hooman) only supports one kind of cloudfare challenge and cloudscraper handle multiple challenges and can chain them. So hooman may fix Yggtorrent and break other providers.

I’d rather prefer a fork of cloudscraper with the fix used in this package. Furthermore, there is a new type of challenge not supported by any of the 2 packages, so it will not work in all cases.

I’ll check if I can fix cloudscraper, and If someone wants to make a PR with hooman, I could publish it under an experimental tag (ex: “torrent-search-api@2.1.1-experimental”)

UPDATE: cloudscraper with the fix evoked before is equivalent to hooman, I obtained the same result, a 403 due to a captcha to solve.

I got fed up with the Cloudflare anti bot, so I created a proxy that internally use Chromium to fetch the page: https://github.com/unixfox/pupflare.

It works pretty great and there is no need to set any “cloudflare cookie” because just like a normal Chromium it stores the needed cookies while the server is up.

If you would like to use it with torrent-search-api library modify the base URL and disable the internal cloudscraper like this:

const yggTorrentProvider = TorrentSearchApi.getProvider('Yggtorrent');
yggTorrentProvider.baseUrl = "http://localhost:3000/?url=https://www2.yggtorrent.se";
yggTorrentProvider.enableCloudFareBypass = false;

EDIT & Update: I added the ability to do POST requests and handle the files like torrents (thanks Google Chrome, its was a nightmare to implement that) so it works perfectly with all the functionalities of torrent-search-api now.

The cloudflare protection has been deactivated, it may not last long but we’re back in business, Yggtorrent provider is working.

If you want a temporary solution, try this patch .

It works for me, but there’s a new cloudfare challenge so it may fail in some cases.

The fix doesn’t seem to work anymore…

I added a new experimental cloudflare bypass. It’s using puppeteer, like pupflare but only for grabbing the “cf_clearance” cookie.

To use it, take the latest version of “torrent-search-api” and also add “cloudflare-scraper” to your project dependencies (It will remain as an optional dependency to reduce the size of the dependencies if you doesn’t want to use a chromium instance).

I tried to fix cloudscraper but it’s tricky, I’m kind of stuck on the captcha part.

Even If I think it’s possible to bypass either cloudfare bots and hcaptcha service, I don’t think It will last long term and I don’t have the energy to patch it every week. It’s a cat and mouse game and the cloudfare team is reactive.

For a more reliable version, but also more resource consuming, I’d use puppeteer (chrome headless) to retrieve the “cf_clearance” cookie from cloudfare. It means that you’d have to fire up an instance of chrome every 24h (cookie expiration time) for a short period. Once you have this cookie, it’s pretty easy to inject it in “torrent-search-api”.

I tried the following code and It seems to work fine:

const yggTorrentProvier = TorrentSearchApi.getProvider('Yggtorrent');
// Fix 1010 error
yggTorrentProvier.headers = {
      'User-Agent':
        'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'
    };
// Avoid cf challenges
yggTorrentProvier.setCookies(
      ['cf_clearance=XXXX09b8166c53752ddaf28ecd1a209cea5b5237-1589898270-0-150;']
);

I’ll swap the cloudscraper package If there’s a new player in this game (hooman could be a good candidate but it doesn’t fix our issue for now).