yt-dlp: [crunchyroll] Error 403 Forbidden for all URLs

The initial state extractor bug is being tracked in its own issue, and there is a patch available here: https://github.com/yt-dlp/yt-dlp/issues/7624#issuecomment-1638912586

The 403 errors are coming from Cloudflare; their fingerprinting is determining that yt-dlp is not a web browser and blocking its requests to Crunchyroll.

There is a workaround for this. It requires cookies from a browser with the same IP address that you will be using with yt-dlp. Here are the steps:

1. Refresh your cookies in browser by loading a Crunchyroll video page until the video starts playing. If you have not done this within the past 30 minutes, you will need to do it again

2. Find your browser’s User-Agent string; you need the entire string (starting with Mozilla/5.0) and it needs to be up-to-date (i.e. if your browser updates, you’ll need to get its new UA string with the current version)

   • the simplest way is to type “my user-agent” into duckduckgo, google, etc.
   • for Firefox browsers, you can find the user-agent by navigating to about:support
   • for Chromium browsers, you can find it in chrome://version (or your vendor prefix like brave://version, etc)

3. Pass your browser’s user-agent to yt-dlp with --user-agent "USERAGENT" along with --cookies-from-browser firefox

   • replace USERAGENT with your actual full user-agent string you got from step 2
   • replace firefox with the browser that you are using to browse Crunchyroll
   • Or, if you’d rather use a cookies file instead of having yt-dlp extract cookies from your browser, you can use the --cookies option, e.g. --cookies cookies.txt. Note that the cookies need to be exported from a fresh browser session (see step 1) within the past 30 minutes. See the wiki for browser extension recommendations

PS: If you get an Unable to extract initial state error, you need to run yt-dlp --update-to nightly or install master branch with pip

PPS: This appears to not work with MS edge. Use chrome/firefox

@Killua-Zolldyck a proper fix is being worked on currently

@Killua-Zolldyck having the same problem

Hmm. Crunchy went through a maintenance outage about an hour ago. Now I get this error message, no matter what I try. Looks like support for Crunchy is gone for the moment…

ERROR: [crunchyroll] G14U4WXJ9: Unable to extract initial state; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U

Anyone else having this problem or is it just me?

EDIT: I’ve just opened Crunchy in my Edge Browser and the Crunchy Website displayed a message “Your device has been trusted.” or something like that (already closed the message and can’t reproduce it).

To clarify it to everyone: Crunchyroll has changed the access to the videos. You need a cookie that is at max. 30 minutes old and includes your login to the page. The cookie itself is updated everytime you open a CR Website in your browser. If you open a link with yt-dlp the cookies aren’t updated, because yt-dlp is not a browser itself and just uses the cookies of the browser you give in the options. And at least on my system, I have to use Firefox at the moment, because Edge does not work, even if I use --cookies-from-browser edge and the correct --user-agent option for my browser.

I don’t think, this can be updated in yt-dlp easily to work like before. yt-dlp isn’t a full feature web-browser and I wonder if it will be possible at all to keep the cookie updated by yt-dlp.

So as far as I see, there’s only one option to download from CR at this time:

• use Firefox
• have an up to date cookie for CR website (max. 30 minutes old)
• use --cookies-from-browser firefox
• use --user-agent “…”
• the value of the --user-agent option must fit to the browser, you generated the cookie with

If you know any other working scenario, feel free to report here.

@tamodolo are you still able to download from CR? I’ve tried downloading firefox to test on it but still getting 403.

cmd line: yt-dlp -vU “https://www.crunchyroll.com/pt-br/watch/GJWU2EK7Z/sparkling-fresh” --user-agent {myUserAgent} --cookie-from-web firefox

doesn’t matter firefox // chrome // edge they are the same issue.

@BraveSail got a working solution for that. It’s was the same here until I opened firefox.

@livelife68 I run 2 downloads at once. But not dare to try more than that.

edit: nightly release broke the season alias…

Switching from Edge to a clean install of Firefox did solve the problem for me. Just to let you know.

Edit: The usage of --user-agent was necessary to make it work on Firefox! Using --user-agent with Edge did not solve the problem.

https://github.com/yt-dlp/yt-dlp/commit/9b16762f48914de9ac914601769c76668e433325 has already been merged instead to solve the initial state removal and to hint towards the workaround.

A release is on it’s way, but like said above, nightly builds can be used instead if required. Use these if required.

Thanks @coletdjnz, this version https://github.com/coletdjnz/yt-dlp-dev/releases works great i can login and download premium content without worrying about cookies expiration or changing useragent on every update i used --impersonate chrome110, and i can confirm that its working with Firefox cookies text file (expires every 30 min) the only problem is with chrome cookies which i think Crunchyroll give different cookies for.

EDIT: --cookies-from-browser firefox never expires and you don’t need to open the browser

Found some issues in #7595 :

1. pypy with arm64 environment need to compile curl_cffi locally to install it. Maybe same as CPython? I’m not sure since I tested pypy first.
2. pypy can not load the curl_cffi module even it have successfully installed, CPython is ok.
3. Hidive is 403 now at video.hidive.com when downloading the stream segment, even though hidive is not using cloudflare. (stable version is ok)

[debug] Command-line config: ['-v', '-R', 'infinite', '--force-overwrites', '--fragment-retries', 'infinite', '--no-mtime', '--proxy', 'http://127.0.0.1:1080', 'https://www.hidive.com/stream/otaku-elf/s01e012', '-u', 'PRIVATE', '-p', 'PRIVATE', '-N', '16', '--user-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:103.0) Gecko/20100101 Firefox/103.0', '--embed-subs', '--convert-subs', 'ass', '--sub-langs', 'all', '--remux-video', 'mkv', '-o', '/root/bangumi/[GJ.Y] Edomae Elf - 12 (Sentai .mkv']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8 (No ANSI), error utf-8 (No ANSI), screen utf-8 (No ANSI)
[debug] yt-dlp version stable@2023.07.06 [b532a3481]
[debug] Python 3.9.5 (CPython aarch64 64bit) - Linux-5.10.0-21-arm64-aarch64-with-glibc2.36 (OpenSSL 1.1.1n  15 Mar 2022, glibc 2.36)
[debug] exe versions: ffmpeg 4.3.5-0, ffprobe 4.3.5-0
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, curl_cffi-None, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {'all': 'http://127.0.0.1:1080'}
[debug] Request Handlers: curl_cffi, urllib
[debug] Loaded 1767 extractors
[HiDive] None: Downloading webpage
[HiDive] Logging in
[HiDive] Extracting URL: https://www.hidive.com/stream/otaku-elf/s01e012
[HiDive] otaku-elf/s01e012: Downloading JSON metadata
[HiDive] otaku-elf/s01e012: Downloading m3u8 information
[info] otaku-elf/s01e012: Downloading subtitles: en
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] otaku-elf/s01e012: Downloading 1 format(s): ja_br_or-8361
[info] Writing video subtitles to: /root/bangumi/[GJ.Y] Edomae Elf - 12 (Sentai .en.vtt
[debug] Invoking http downloader on "https://www.hidive.com/caption/vtt/259c7570ee49a3c9c25098de73f85f1c02cd4a6e/OTK_s01e012_tv_br_or_ja_en_v01.vtt"
[download] Destination: /root/bangumi/[GJ.Y] Edomae Elf - 12 (Sentai .en.vtt
[download] 100% of   47.17KiB in 00:00:00 at 135.04KiB/s   
[SubtitlesConvertor] Converting subtitles
[INFO] Converting: [GJ.Y] Edomae Elf - 12 (Sentai .en.vtt ...
Deleting original file /root/bangumi/[GJ.Y] Edomae Elf - 12 (Sentai .en.vtt (pass -k to keep)
[debug] Invoking hlsnative downloader on "https://www.hidive.com/manifest/child/259c7570ee49a3c9c25098de73f85f1c02cd4a6e/OTK_s01e012_tv_br_or_ja_xx_HLS1080p_16x9_02_v.m3u8?#="
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 147
[download] Destination: /root/bangumi/[GJ.Y] Edomae Elf - 12 (Sentai .mp4
WARNING: The download speed shown is only of one thread. This is a known issue
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 6 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 4 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 10 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 2 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 9 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 5 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 3 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 8 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 7 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 11 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 15 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 16 (1/inf)...
[download] Got error: HTTP Error 403: Forbidden. Retrying fragment 14 (1/inf)...

#7632 is working for me using “–extractor-arg crunchyrollbeta:ua_workaround” option. Thanks a lot!

So I tried it the way I used to before, and it still gave an error.

@Rafawell you need to add ua_workaround to your crunchyrollbeta extractor-args for the “old” way to work with this build e.g. --extractor-args "crunchyrollbeta:format=vo_adaptive_dash;ua_workaround" or, if you aren’t passing any other extractor args: --extractor-args "crunchyrollbeta:ua_workaround"

Fixes for both the 403 and the initial state have been worked on in #7632, but they need testing.

A pre-release build is available: https://github.com/yt-dlp/yt-dlp/pull/7632#issuecomment-1641074939. Please test it and see if the fixes worked for you.

bashonly fix only the initial state, for 403 the workaround is to use a cookie form the browser.

Ah so the cookie generated by yt-dlp will not work? Also I notice it now that my cookie generated is edited by yt-dlp when I use --cookie ‘pathtocookie’… not sure why… I used to just create that file from my webview in my own interface so I’m not sure if I can still use my own or of I need to use yt-dlp generated file

yt-dlp isn’t a real browser so it won’t update the cookies ever. Use firefox instead and the commands I wrote in my previous comment.

Oh sorry, i was in a hurry. I have tryed it with multiple browsers: chrome, brave, firefox All are the same. The cookies work for 30min after the last request the browser made (refreshing/loading a link, getting a video fragment) and inport into ytdlp. That means if you keep a video playing in the background and make ytdlp reload the cookies every 30 min, it would work no prob.

Need to reinstall chrome first - haven’t used chrome since ages. I’ll check.

Update: Unfortunately (?) downloading a video using --cookies-from-browser chrome works for me as well.

Also doesent work for me 😕

Did you pass Brave’s --user-agent, after visiting Crunchyroll at least once?

It can’t be added until #2861 is merged, but that PR is actually extremely close to being ready to be merged!

It only adds the framework that would be needed to support this. Actually adding support would be quite a ways away.

I’m sure this is the correct solution

Yes, adding support for Curl Impersonate is the ultimate solution to this problem. It can’t be added until https://github.com/yt-dlp/yt-dlp/pull/2861 is merged, but that PR is actually extremely close to being ready to be merged!

Use cookies and user agent from a legit browser. That worked for me.

Strange, the error occurred again I thought it was resolved just by using the --user-agent, I have no idea what it could be.

You need to refresh the cookies directly on firefox. Open something and it’ll work again. yt-dlp need to implement this feature to work without this.

So i am on Windows 10 on Version 2023.06.22 of yt-dlp using following command (cookies on Firefox already remove and renewed for crunchyroll):

.\yt-dlp.exe -vU -f best --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0" --cookies-from-browser firefox --cookies .\auto_cookies.txt --ffmpeg-location .\ffmpeg\bin\ffmpeg.exe --batch-file .\links.txt

and trying to download some german dubbed episodes and getting the following error:

ERROR: [crunchyroll] GG1U2ZNEV: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 708, in extract
  File "yt_dlp\extractor\crunchyroll.py", line 320, in _real_extract
  File "yt_dlp\extractor\crunchyroll.py", line 114, in _call_api
  File "yt_dlp\extractor\crunchyroll.py", line 99, in _call_base_api
  File "yt_dlp\extractor\crunchyroll.py", line 69, in _update_query
  File "yt_dlp\extractor\common.py", line 1111, in _download_webpage
  File "yt_dlp\extractor\common.py", line 1062, in download_content
  File "yt_dlp\extractor\common.py", line 896, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 853, in _request_webpage

  File "yt_dlp\extractor\common.py", line 835, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3786, in urlopen
  File "urllib\request.py", line 531, in open
  File "urllib\request.py", line 640, in http_response
  File "urllib\request.py", line 569, in error
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 403: Forbidden

Anybody got any suggestions what the problem could be?

Update:

Ok it works now but not with the --cookies-from-browser firefox option. I’ve extracted the cookies with a browser addon on Firefox and used that cookie file and that worked. So the command that worked for me:

.\yt-dlp.exe -vU -f best --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0" --cookies .\cookies.txt --ffmpeg-location .\ffmpeg\bin\ffmpeg.exe --batch-file .\links.txt

I suspect that the Crunchyroll Cookies have been shortened to be valid for only 30 minutes. I was able to get downloads using the --user-agent “USERAGENT” along with --cookies-from-browser firefox, however it started to fail about 30 minutes in to downloads.

[download] 100% of 1.38GiB in 00:03:32 at 6.65MiB/s [Metadata] Adding metadata to "downloads\Wandering_Witch_-_The_Journey_of_Elaina\01\002 - The_Land_of_Mages(8).mp4" [crunchyroll] Extracting URL: https://www.crunchyroll.com/watch/G14U4E27X/the-girl-as-pretty-as-a-flower--bottled-happiness [crunchyroll] Authenticating with grant_type=etp_rt_cookie ERROR: [crunchyroll] G14U4E27X: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U

I then refreshed Firefox and played a short video segment and then it allowed me to download again for 30 minutes before failing again (Approx 8 episodes). Rinse and repeat the actions.

If you are playing through a recognized browser the browser would automatically update the authentication cookie upon trying to play a video. yt-dlp does not automatically refresh cookies from the site.

I am now limiting myself to 7 - 8 downloads at a time. [Batch list/ not concurrenty]