youtube-dl: [Crunchyroll] ERROR: Unable to download webpage: HTTP Error 403: Forbidden

Checklist

I’m reporting a broken site support
I’ve verified that I’m running youtube-dl version 2021.03.03
I’ve checked that all provided URLs are alive and playable in a browser
I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
I’ve searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-f', 'best[format_id*=enUS]', '--playlist-start', '68', 'https://www.crunchyroll.com/attack-on-titan']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.03.03
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg N-93542-gecdaa4b4fa, ffprobe N-93542-gecdaa4b4fa
[debug] Proxy map: {}
[crunchyroll:playlist] attack-on-titan: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpvjygsik_\build\youtube_dl\extractor\common.py", line 632, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpvjygsik_\build\youtube_dl\YoutubeDL.py", line 2275, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

Description

I came across this issue not too long ago. Haven’t been able to fix it on my own, so here I am. I’ve tried adding cookies.txt to the command line as well as the user agent, neither solved the issue.

About this issue

Original URL
State: open
Created 3 years ago
Comments: 19 (3 by maintainers)

Commits related to this issue

See commit description - Add workaround for https://github.com/ytdl-org/youtube-dl/issues/28398 - Refactor some code - Use PrettyTable on anime selection — committed to Trigus42/crunchyroll-dl by Trigus42 3 years ago
See commit description - Add workaround for https://github.com/ytdl-org/youtube-dl/issues/28398 - Refactor some code - Use PrettyTable on anime selection — committed to Trigus42/crunchyroll-dl by Trigus42 3 years ago

Most upvoted comments

It seems to be the __cf_bm cookie which is required. I generated a cookie file and removed them one by one until youtube-dl stopped working. I then removed all of the others and left just this cookie and it worked fine.

Seems this is a cloudflare cookie: https://support.cloudflare.com/hc/en-us/articles/200170156-Understanding-the-Cloudflare-Cookies#12345681

6cUbi57z on Apr 4, 2021

The problem seems to be that, when ytdl tries to GET i.e. /darling-in-the-franxx, CR (or CF) returns 403. This does not happen in the browser for the same URL.
Same is true when using a cookie exported from my browser session.

After a lot of trial and error, I’ve found a workaround though:
Set the --user-agent to your browser’s and pass your browser’s CR cookies to ytdl.

It’s fairly fragile (one slight difference in the UA string makes it 403 again and also invalidates the cookie file?) but looks to be working well enough.

Seems like CR is getting more sophisticated with blocking automated tools and they’ve had a DRM rollout looming for a while too…

Atemu on Apr 1, 2021