Checklist
Verbose log
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2024.08.07.local
[debug] Python 3.8.5 (CPython x86_64 64bit) - macOS-10.16-x86_64-i386-64bit - OpenSSL 1.1.1h 22 Sep 2020
[debug] exe versions: ffmpeg 7.0.1-tessus, ffprobe 6.1.1, rtmpdump 2.4
[debug] Proxy map: {'http': 'http://127.0.0.1:58309', 'https': 'http://127.0.0.1:58309'}
Usage: youtube-dl [OPTIONS] URL [URL...]
youtube-dl: error: You must provide at least one URL.
Type youtube-dl --help to see a list of all options.
PASTE VERBOSE LOG HERE
Description
- The download will run normally for a short time, and then a 403 error will appear
youtube-dl 'https://www.youtube.com/watch?v=lLSkbZ3-EOs'
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 10 of 10)...
[download] Skipping fragment 41...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 4 of 10)...
I’m also experiencing this error, but when i changed the user-agent to: “Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.6533.103 Mobile Safari/537.36” (mobile user agent of the latest chrome version) it worked. Here is the exact line:
youtube-dl --verbose -x --user-agent "Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.6533.103 Mobile Safari/537.36"This just worked for me:
Try with
m.youtube.comor pass:--user-agent "Mozilla/5.0 (Android 14; Mobile; rv:128.0) Gecko/128.0 Firefox/128.0".Works (for now…)
Adding
--user-agent "Mozilla/5.0 (Android 14; Mobile; rv:128.0) Gecko/128.0 Firefox/128.0"worked for me just now on a previously failingwhere my alias yd is
Please don’t bother to supply any “me too” reports unless the log shows some novelty that may help with rectification. Just “Like”, or whatever, an existing similar report.
You can see how a
poTokenis being sent inPOSTdata by the browser in the Invidious code that shows how to capture the value. But I understood from yt-dlp discussions that apotquery parameter was used in the media links associated with the pot-ified session.@8ChanAnon’s algorithm is what is currently done for age-gate videos, up to the last step with
m.youtube.comwhich is new and interesting. What happens if you skip straight to that step?Step 2 will only work if
TVHTML5_SIMPLY_EMBEDDED_PLAYERis not pot-ified, and that seems to be in question.A tldr; may not be out of place, even if all this can be read above.
Many programs, including this one, use an internal YT API to provide services that cannot be supported through the official YT API. The internal API has been changed such that media links extracted using the
WEBclient (which, for now, we use) require a further challenge (thepoToken) to be solved to avoid 403 after the first ~1MB. This (with similar things) is why a YT web page is several MB instead of 10kB. Using m.youtube.com may be a work-around until after “for now”.Fixing this in line with the long established multi-client capability in yt-dlp is resulting in a long twisty set of commits that are not yet quite ready for PR.
To avoid 403 on the media download, you need a work-around, which is to use a UA like the one I showed, which causes YT to use its mobile web client. The downside is that the current extractor may not be able to extract playlists from the mobile page format. You could list the playlist IDs with
--flat-playlist --get-id, not specifying any UA (add--get-titlefor a hint as to the content associated with each ID); then get each wanted audio item using the mobile UA and other parameters similar to what you have above. You should scroll back in the issue, I’m afraid, for all the details.Eg, as recommended above:
The mobile UA work-around could cause metadata to be lost. Even if that doesn’t seem to be relevant, please check again with the real-soon-now PR and post any appropriate diagnostics there.
can confirm this works for me (youtube-dl proxy through tor-browser-bundle exit=de)
Please try https://github.com/ytdl-org/ytdl-nightly/releases/tag/2024.12.17 to resolve this issue for now.
@bstojkovic unless you are reporting the 403 in the original post, please get out.
This project seems to have slowed down a lot. @dirkf is overwhelmed by issues and a lot of features and fixes have yet to be backported.
I think if you are not a legacy system user, it might be better to use
yt-dlpinstead. Otherwise you will have to wait.Yes, because specifying a mobile UA redirects to m.youtube.com as above.
Mozilla/5.0 (Android 14; Mobile; rv:115.0) Gecko/115.0 Firefox/115.0is a shorter UA that has the same effect.However the
ytInitialDatain the mobile page is stringified JSON rather than actual JSON as in the desktop page, and this causes an additional API call (I suspect that no valid data is returned).Can confirm that a lot of the video-only formats are just being 403-ed in the middle with their downloads, resulting in me getting files that stop after about 10-20 minutes into the video, but still have full sized audio.
By now I have written something into my scripts to just pick format 18 as long as a flag is set, because i foresee this issue happening again in the future once it is eventually fixed… >.>
… Thanks a bunch 👍 ; can confirm that the latest nightly can indeed complete a download attempt:
😄
https://github.com/ytdl-org/ytdl-nightly/releases/tag/2025.01.23
@nicolaasjan wrote:
… But have you actually attempted a download? 😉
… so the answer is NO here 😢 …
When testing nowadays, there are 3 things to check rather than the 1 that used to be enough:
Sign in to confirm you're not a bot?It seems that the mobile UA work-around is failing now, though yt-dlp (2024-12-06) is able to pull the same listed formats. Testing video
lLSkbZ3-EOs, I found format 18 was downloadable with the PR #32987 code, but I think there some videos that don’t have this not-really-suitable-for-a-55"-TV format.At a guess, the other formats are no longer accessible in the mobile page, or the UA isn’t causing redirection: if they were found in the API JSON they would be listed but not downloadable because of not using
sts, as noted above. More investigation needed, but it shouldn’t de-rail the rewrite when I have time to get back to it.Use the nightly version (is built from current source): https://github.com/ytdl-org/ytdl-nightly
Anything new on this?
I am using the latest snapshot and it’s still not working, I’m getting the same errors as in this issue.
In addition other formats such as 140 or 18 aren’t working as well. I’m getting the
Unable to decode n-parameterthere.And interestingly,
yt-dlp 2024.04.09is working.Edited to add:
This is the commandline I’m using:
I created the cookies file using the chrome extension “Get cookies.txt” right before I executed the command after visiting the uri I was about to download.
@PatrickJRed it still wouldn’t help with this situation, but it’s pretty simple to add another curl downloader, you should open another issue or make the PR yourself
@dirkf
On a lot of websites I want to download from youtube-dl and curl dont even get the correct html, instead of the one i would get in my Browser , they receives a version that has the captcha.
Instead I have a bash-script that predownloads the non-captcha html via “https://github.com/lwthiker/curl-impersonate” (runs in docker container , i use tag: 0.5.2-ff-alpine)
AFAIK lwthiker/curl-impersonate is the only http client that completly impersonates an actual browser like firefox. A lot of the problems I had with 403 errors in youtube-dl where captchas triggered by http-client not being exactly like an official version of firefox or chrome 😃 (this might even be valid for fragment-downloads)
I don’t think it’s worth trying to get around the poToken, it will eventually be required in all clients.
I keep digging into
base.jswhen I get some time trying to understand how the token is created, it does seem to be extracted from the bytes of at least the first video fragment as far as I can tell, but not all fragments?..There’s a
Uint8Arraywhich appears to be the fragment response data?.. manipulated several times and then 82/84/68 bytes of that array are stored asplaybackCookiewhich is then sent in the POST dataUpdate:
I just tried again, first time today, and the audio failed the first attempt, but worked on the second, and the video worked on the first try. Using youtube-dl.
I think there is something going on here too. Yesterday I downloaded about four videos (format 18, 20+ minutes each, no rate limit) in rather quick succession, but when I got to the fifth video, I noticed that it served me a new javascript (b191cf34) and immediately got a 403. This was roughly 24 hours ago, by the way. The javascript today is called 962b571b and still a 403.
It is possible that I got IP-blocked but I can still download with yt-dlp with the same IP. Just tested with a single video.
Everything was working well for me yesterday on 2025.02.28 but today I’m getting the same 403 error trying to download videos on every channel I try.
403 is now expected when nsig processing fails. This is a new issue related to player
9c6dfc4a; I can reproduce it with the same test video. Using yt-dl, there isn’t any work-around at the moment.Apologies @dirkf, I suppose I somehow managed to get onto this topic by following other links but I can see now why the message I’m getting is actually in reference to #32960 and not this one. Thanks for the correction! If there’s anything I can do to assist, I’d be happy to try and lend a hand.
This isn’t happening systematically: it’s not happening for me with either Py3.11 or Py2.7. All the considerations from #32960 apply.
Can confirm release 2025.01.23 makes youtube downloadable again. (tested with >10 videos from different channels)
@Linux2010 @dirkf So this issue can be be closed as completed/fixed ?
Oops… You’re right . It doesn’t work.
@Rekrullurker posted
That’s because the sign-in error comes when fetching the JSON containing the download links. The nsig challenge, which can take seconds, is executed only when processing those links.
-vmakes this apparent.I can confirm that the last video example from @FMecha works with the youtube-dl version built from the latest master:
Seems to work (for now…). 😀️
OK, so that’s the version (2025.01.13) with the commit linked above, selecting the
tvInnerTube client, that I’m right now shoe-horning into yt-dl.At this point, YouTube is doing it intentionally, right? People report error 403, a fix/workaround is found, and then a few days later, 403 errors again.
https://github.com/yt-dlp/yt-dlp/commit/712d2abb32f59b2d246be2901255f84f1a4c30b3#diff-b7b9f6790de4427214b61939432e667d95b929d07fd918b9da1a36d7996cc506R259
LoL, but if you want, pay back to an open source program in some way, or somehow help a local charity.
Also, review #32960 for guidance on how to avoid further breakage.
Apparently the client parameters, including UA, need to be just right.
A temporary fix (that worked today) is included in PR #32987.
Yes, I am. After some more testing, it seems it only fails on some videos.
Here is the log from one such failure
Thank, that build worked great.
I have installed ffmpeg (6/20/2018) in Winds11 “Edit the sys environment variables” in “Environment variables” then “Path” then “edit” then added the last two as pictured. (youtube-dl.exe ver 2024.8.7 nightly build).
Now I am trying to only get mp3 files from YT and here is a screenshot of what issues I have in Windows Power Shell
@dirkf I tried that and it works, thank you.
Though just a note, I got an output that looks like this:
'Numb (Official Music Video) [4K UPGRADE] – Linkin Park.f18.mp4'. including the quotes, having to rename the file. Did I do something wrong?Also, how do I get the audio part, preferably mp3, from youtube-dl, or do I have to use ffmpeg manually?
PS. Nvm, sorry, I had to replace single quotes with double quotes and use
worstaudio.But when I try to download a playlist with this, I get the following:
come on,baby, we need your solution, hurry, very thanks!
Not “just” asking but imputing motives, which is not relevant.
yt-dlp reports may well offer good evidence of YT site behaviour (and its YT extractor can test the known InnerTube clients). There is no adverse competition with yt-dlp since each project has its own goals and code is compatibly (un)licensed.
For avoidance of doubt, that is the end of this meta-discussion.
@xyzxye
Update to 2024.08.06 or later. If the issue persists open an issue on the yt-dlp issue tracker.
Because someone needs to take the time and port the improvements made to yt-dlp, over to youtube-dl. yt-dlp already switched away from the web client ~1 month ago and even before that used iOS as a secondary client. If you require the Python 2 compatibility, you will have to wait for these improvements to be ported to youtube-dl (or even contribute something yourself).
I outlined the solution to this problem two weeks ago. Guess I have to prove that my method works. I put together a little demo:
https://8chananon.github.io/dl/yt-extract.htm
It runs in your browser. For that reason, I have to use a proxy server but my proxy is blocked by Youtube due to having an Amazon IP address so the proxy has to route through another proxy which isn’t banned by YT. Five proxies have been provided and the app will cycle to the next one if it detects blocking (or the proxy stops working).
Note that videos that are pegged to the IP address won’t work because of the proxy (like these two: 8Pa9x9fZBtY and 0pKfxbCHLoU).
You can input a full URL or just the video ID. The app just dumps all of the itags (except itag 18) as links to the actual video/audio. Click a link to play it in a new tab. You can play the audio and the video in different tabs and synchronize them (LOL).
The code is, of course, in Javascript. Sorry about that but I don’t do Python. Also, the download speed seems to be throttled but videos play fine.
If you want itag 18 or you just want to play the video to check it out:
https://8chananon.github.io/dl/yt-player.htm
Just confirming, my Script still works and downloads Videos without needing to change things up.
Still does use the legacy video formats, which kills my mobile based home data plan though, so I really hope there is going to be a fix so i can use 240p on the longer vods again. (not rushing you, because i know how complicated this stuff can be, but I’m annoyed at youtube itself for causing this in the first place)
Unfortunately, the yt-dlp stage is worse than that of this project. any ‘left-right’ steps and you have broken code. any support as well. so no
Apparently the latest fix worked for not even a day, that doesn’t bode well. Personally I keep getting “giving up after 0 fragment retries” in my python stuff.
From what I read in yesterdays thread, it seems like this will just not work out with fake JS interpretation if they try to combat this in the slightest. Like, that almost doesn’t deserve the name attack vector, that’s an attack landscape.
I have that with every single video I try. Curiously enough format ‘18’ work all the time. Other formats that work are ‘136’, ‘137’, ‘248’ and/or ‘160’, but it depends on video - not always the case. Still, format ‘18’ is the most reliable to work.