youtube-dl: Error 403 at Crunchyroll even with cookies (youtube-dl)
Checklist
- I’m reporting a broken site support issue
- I’ve verified that I’m running youtube-dl version 2021.12.17
- I’ve checked that all provided URLs are alive and playable in a browser
- I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
- I’ve searched the bugtracker for similar bug reports including closed ones
- I’ve read bugs section in FAQ
Verbose log
youtube-dl.exe --verbose --cookies "%CD%\cookies.txt" --get-filename -o "test video.%(ext)s" "https://www.crunchyroll.com/es-es/hundred/episode-1-little-garden-702479"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--cookies', 'C:\\Users\\Administrador\\Documents\\Visual Studio Projects\\CrunchyRoll Crawler\\CrunchyRoll Crawler\\Interop\\cookies.txt', '--get-filename', '-o', 'test video.%(ext)s', 'https://www.crunchyroll.com/es-es/hundred/episode-1-little-garden-702479']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg 2021-11-25-git-522f577d7e-full_build-www.gyan.dev, ffprobe 2021-11-25-git-522f577d7e-full_build-www.gyan.dev
[debug] Proxy map: {}
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 634, in _request_webpage
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 2288, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default
Description
I’m not sure whether this could be a bug.
I think this could be related with Cloudflare or reCaptcha for some reason.
Yesterday I used youtube-dl to download without no problems a entire non-premium non-login-required anime season from Crunchyroll website ( this anime: https://www.crunchyroll.com/es-es/hundred ). After that, youtube-dl stopped working when I tried to download more videos, giving me HTTP error 403 when I try to download any video from Crunchyroll…
So I took a research and investigated all related about cookies usage.
I can access Crunchyroll website with no problem under my local web-browser: Firefox DevEdition. But yesterday after I finished downloading all the 12 episodes with youtube-dl, and it stopped working giving me error 403, I noticed that Crunchyroll website asked me to solve a reCaptcha in my local Firefox DevEdition web-browser, but it happened only once.
If I delete all cookies from my Firefox DevEdition browser, and I access Crynchyroll website again, it asks me to resolve a reCapctha only once.
I also noticed that if I use CefSharp web-browser in a .NET Windows Forms application, the first time that I try to access Crunchyroll website it always ask me to resolve a reCaptcha. This happens once every time that I run a new instance of the application. After I solved the reCaptcha it does not ask me anymore while the application instance is running. This never happened to me before I downloaded the 12 episodes with youtube-dl.
In my Firefox DevEdition web-browser I tried to use cookies.txt extension to generate a cookies file from Crunchyroll website. But when I pass these cookies.txt file to youtube-dl it does not solve the problem, I still get error 403.
I tried other parameters like removing cache dir, check no certificate, setting the same user agent as my Firefox web-browser, and username + password.
What can I do to avoid this 403 error and finally let me download videos from Crunchyroll again?.
Cookies.txt file content
# Netscape HTTP Cookie File
# https://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file! Do not edit.
www.crunchyroll.com FALSE / FALSE 1652010928 cf_chl_2 9f3a7a1794ff871
www.crunchyroll.com FALSE / FALSE 1652011024 cf_chl_prog x14
#HttpOnly_.crunchyroll.com TRUE / TRUE 1683547024 cf_clearance ieG4GNX1pmXKTG6c2LcgKhvEkc8bvWWaisNvaM4zBxg-1652007424-0-250
#HttpOnly_.crunchyroll.com TRUE / FALSE 1667559424 c_visitor 4acfe514-2de9-4994-880f-36ded63f323e
#HttpOnly_.crunchyroll.com TRUE / TRUE 1652009224 __cf_bm D8vSxRBb4t9zm2X2NUwRnFjNzboYdfAE9Th9U4JVv_Y-1652007424-0-AWaAkapHhy9vfAZgOUnPtG3vf0ieECPCZrGyg/MI4DdOni7GKSaYaMNlmzVTh6XUAVYnKvA4o2dLmj7+loj+d8O2+v4tWi+jm2RTvjNYcxei
www.crunchyroll.com FALSE /es-es FALSE 1686221830 _evidon_consent_cookie {"consent_date":"2022-05-08T10:57:10.749Z","consent_type":1}
www.crunchyroll.com FALSE /es-es FALSE 1654599430 _evidon_suppress_gdpr_locale_cookie accepted
#HttpOnly_.crunchyroll.com TRUE / TRUE 0 __cfruid c91cf6b0af8161647b92804d2eee06e57bac830f-1652007529
#HttpOnly_.crunchyroll.com TRUE / TRUE 0 session_id f50cc1151f31662aada2cb5a7a6f0819
#HttpOnly_.crunchyroll.com TRUE / TRUE 1653217139 etp_rt d6746056-7de4-41be-92eb-3525679a704e
www.crunchyroll.com FALSE /es-es/hundred FALSE 1683543540 crunchyroll_beta_hide_banner 1
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 18 (7 by maintainers)
Did you use a VPN or a Proxy? Some of these servers are in a Cloudflare IP blocklist and will always get a forbidden response. If so try accessing one vid and if you succeed try another vpn server. Also you should not need your cookies if you want to download a free no-login show.
Regarding the User-Agent: I suggest useing the same User-Agent string as the Browser you exported the cookies from. Simply Type “my user agent” in google and you should get the correct user-agent.
I hope so. Some people care about their IP address. Possibly some site cookies may contain personal data, or username/password may be there if you logged in to the site. Try searching for strings you don’t want in the file.
http://www.softwareishard.com/blog/har-12-spec/
If you repeatably get 403 with yt-dl and the same page displays successfully in your browser, a HAR archive of the connection in the browser could be useful (which you can generate using the browser’s development tools). You may wish to sanitise any personal information if posting it in GH.
Yes. Try being an old iPhone.
Try
--user-agent "Mozilla/5.0":Some CDN gatekeeper like Cloudflare is breaking the web by blocking on user agent strings, before even looking at your cookies. Fortunately the latest specifications favour not sending an explicit user agent string and they don’t come less explicit than
Mozilla/5.0.i wa able to download episode by opening the dev tools - network - type mpd in the search box and use that link to download the episodes.
Thanks for answer. It still gets HTTP Error 403: Forbidden