yt-dlp: [youtube] The following content is not available on this app.. Watch on the latest version of YouTube

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Please make sure the question is worded well enough to be understood

There is such an error, what can be done?

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://youtu.be/3z0IT3bXm_E?si=sJlxgc-qRI8mxT3U']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.05.27 from yt-dlp/yt-dlp [12b248ce6] (zip)
[debug] Python 3.8.10 (CPython x86_64 64bit) - Linux-5.15.0-107-generic-x86_64-with-glibc2.29 (OpenSSL 1.1.1f  31 Mar 2020, glibc 2.31)
[debug] exe versions: ffmpeg 4.2.7, ffprobe 4.2.7
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2019.11.28, mutagen-1.47.0, requests-2.31.0, sqlite3-3.31.1, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1820 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.05.27 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.05.27 from yt-dlp/yt-dlp)
[youtube] Extracting URL: https://youtu.be/3z0IT3bXm_E?si=sJlxgc-qRI8mxT3U
[youtube] 3z0IT3bXm_E: Downloading webpage
[youtube] 3z0IT3bXm_E: Downloading ios player API JSON
ERROR: [youtube] 3z0IT3bXm_E: The following content is not available on this app.. Watch on the latest version of YouTube.
  File "/home/user/.local/bin/yt-dlp/yt_dlp/extractor/common.py", line 734, in extract
    ie_result = self._real_extract(url)
  File "/home/user/.local/bin/yt-dlp/yt_dlp/extractor/youtube.py", line 4248, in _real_extract
    self.raise_no_formats(reason, expected=True)
  File "/home/user/.local/bin/yt-dlp/yt_dlp/extractor/common.py", line 1257, in raise_no_formats
    raise ExtractorError(msg, expected=expected, video_id=video_id)

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 18
  • Comments: 91 (9 by maintainers)

Most upvoted comments

This is not directed at anyone in particular, but this needs to be said before this issue thread gets completely unreadable & unmanageable:

An issue tracker is NOT a message board.

Even though Github’s issues UI looks essentially like a phpBB forum, this is not a phpBB forum.

Off-topic comments do not belong here. Rants against Youtube/Google/Brave/etc do not belong here (even if they are deserved). If a comment has been marked as off-topic/outdated/spam, do not engage with it.

“Me too”/“+1” comments are not useful after the issue has already been established to be widespread. Speculation based on evidence/history is OK, but wild speculation is misinformation.

Take the chit-chat to discord.

Issue threads need to be able to function as resources of information for:

  1. developers trying to fix the issue
  2. other impacted users looking for answers/workarounds/solutions

Maintainers can hide irrelevant comments, but maintainers can’t undo Github’s automatic hiding of potentially useful comments once the thread reaches a certain length. And the longer the thread gets, the more comments are automatically hidden by Github.

Please only comment if you have new/useful information pertaining this issue. Thanks in advance.

Experimental OAuth2 support for YouTube already exists for yt-dlp with https://github.com/coletdjnz/yt-dlp-youtube-oauth2

Though we have currently no indication to believe that it is any “safer” than cookies. The only thing it prevents is the regular cookie rotation that YouTube has been doing for quite a while now.

Is this related? https://support.google.com/youtube/thread/269521462?hl=en

I took one good look at it and it’s a blatant “screw you” to their users. YouTube just decided to lock themselves into a battle that they cannot win to “protect the creators” even though they do next to nothing for the creators themselves and allow scammers to run wild and free. They must have been trying to aim at NewPipe and third-party YouTube clients and yt-dlp got caught in the crossfire or has been targeted directly. What’s the point in trying to block third party clients when they can just patch their code to work around your blocks?

One workaround I can think of is creating a Firefox fork and stripping out Firefox until it’s just the bare browser itself and bundle that with yt-dlp. The idea is to use a modified web browser to avoid tripping this system by making it look like a legitimate browser connection has been made. But that’s complicated, and forging yt-dlps user agent to report as the user’s main everyday browser might work.

EDIT: The fix I mentioned above will no longer work as YouTube is now actually flagging and banning IP addresses. There is sadly no fix.

The message remains the same: YouTube hates you, even as a legitimate user or a paying customer.

Yes, this is how youtube’s IP banning is manifesting itself via yt-dlp output now

If it really was about stopping bots Google could just throw their “I’m not a robot” recaptcha.

  • you can configure that to do really abusive tests as well as rather modest

So I agree with those thinking this is mostly to stop downloaders. This benefit both Google and those producing content worth throwing ads at, if they manage to do that.

And if logged into account at Google they can do more statistics if they get people to sign in more.

But incredible devs at yt-dlp manage to work around things before, so probably will again.

Going with Brave browser give the same “sign in…bot”-suggestion so it’s not just yt-dlp, it’s general problem and wanting people to sign in more, as well, as I see it.

  • and just on some files, not all
  • I went back and forth a couple of times, and only certain files give “bot”-message

And I do vpn all the time, and swapping servers and it might go through in the end. And all cookies and history is cleared in Brave every time I start it. So new ip all the time and still get it mostly on some files.

Sign in to confirm you’re not a bot. This helps protect our community. Learn more

It seems that this error is displayed randomly, I’d say 1 in 5 times, at least according to my tests.

Getting something similar on my end, not sure if related

[youtube] Extracting URL: https://www.youtube.com/watch?v=c0_lGFdm_Ak
[youtube] Sleeping 1.0 seconds ...
[youtube] c0_lGFdm_Ak: Downloading webpage
WARNING: [youtube] Unable to download webpage: HTTP Error 429: Too Many Requests
[youtube] Sleeping 1.0 seconds ...
[youtube] c0_lGFdm_Ak: Downloading ios player API JSON
[youtube] Sleeping 1.0 seconds ...
[youtube] c0_lGFdm_Ak: Downloading iframe API JS
[youtube] Sleeping 1.0 seconds ...
[youtube] c0_lGFdm_Ak: Downloading web player API JSON
ERROR: [youtube] c0_lGFdm_Ak: Sign in to confirm you’re not a bot. This helps protect our community. Learn more
  File "/home/user/.local/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 734, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/yt_dlp/extractor/youtube.py", line 4248, in _real_extract
    self.raise_no_formats(reason, expected=True)
  File "/home/user/.local/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 1257, in raise_no_formats
    raise ExtractorError(msg, expected=expected, video_id=video_id)

It doesn’t matter if you use OAuth instead of cookies, it seems to be putting a complete ban on user who downloads “too much”. I have recently tried to download a playlist through plugins and it gave me this: ERROR: [youtube+oauth2] <videoID>: Video unavailable. This content isn't available. It blocked my throwaway account from downloading and watching videos. But before that i was getting Skipping player response on this video so that was apparently A/B block testing before completely shutting any downloads on JS level…

To emulate being embedded might be a way to download it?

Note that the site you linked uses youtube.com and not youtube-nocookie.com, so it’s using your account’s cookies if you’re signed in. In case you didn’t already, you might want to try a private window.

A priori what he said seems correct.

My IP is currently blocked too, if I’m not logged in then I can’t play videos.

I’ve tried going to the url mentioned without private browsing and with (without being logged into an account in both cases) and the video plays.

Hopefully Google find that side effects are worse and getting less views on content and thereby less ads played so they retract the idea.

That browsers don’t get access either will create resentment from content producers as well.

The cure become worse than the disease.

client does not matter, even youtube’s own web client will give you a sign in error if your ip (in this case of a vpn) isn’t deemed good enough: TeamNewPipe/NewPipe#11139 (comment)

An edit to that comment says the same video then plays again without demanding a login if the IP address changes (VPN is disabled). So it seems like what’s critical here is not causing the IP address to get blocked in the first place.

I think there’s a reasonable case for personal noncommercial downloads of YouTube videos and content from other streaming sites — not everyone is blessed with the kind of bleeding edge bandwidth so many sites demand to watch content at a good quality — and for data preservation as well.

If someone is downloading massive quantities of content quickly, though, I can see a reasonable case for whoever runs that infrastructure to want the requests to be throttled so it doesn’t deny service to others. I know that’s only the fig leaf YouTube would use here to justify what their motives really are, but the fig leaf in itself isn’t entirely unreasonable.

Is it worth having some kind of throttling as the default setting (at least until this is figured out) and letting the user remove it with a command switch? Or have that as the default for whole playlist downloads, even if not for individual video downloads?

Yes, this is how youtube’s IP banning is manifesting itself via yt-dlp output now

YouTube is harming legitimate users and data preservation efforts by doing this yet failing to address scammers. They must REALLY hate their user base if they are going this far. People are just going to work around these blocks; it’s a waste of time. How they are still doing this baffles me. Probably because it harms their bottom line.

Go the “The following content is not available on this app… Watch on the latest version of YouTube” error. Adding --cookies and --user-agent solved it for me.

Oh true, I forgot that small instances tend to be run on VPSs. Maybe it could just be a maximum download count then.

If you don’t download frequently or download with self-restraint then you should be unaffected. This is only affecting those who download frequently. Those who download in bulk or download playlists have the greatest risk of facing this block. YouTube is harming both personal non-commercial use of YouTube videos and data preservation efforts, and they will very likely continue these practices. This is anti-consumer and anti-user, and shouldn’t be legal.

it happened on their private instances as well, with barely anyone using it

that’s because it’s (seemingly) ip-based a/b testing. cloud provider’s ips are probably more likely to be affected than residential ips.

client does not matter, even youtube’s own web client will give you a sign in error if your ip (in this case of a vpn) isn’t deemed good enough: https://github.com/TeamNewPipe/NewPipe/issues/11139#issuecomment-2154637732

Same problem here:

DownloadError: ERROR: [youtube] ... : The following content is not available on this app.. Watch on the latest version of YouTube.

Hello everyone! I’ve encountered this issue too, when I was downloading videos from multiple user accounts for archival purposes. I have a fixed IP address, so changing it isn’t an option for me. I was looking for a quick, free solution. I don’t want to advertise, but the free version of ProtonVPN completely solved the problem. I use it on an Ubuntu server with an ovpn file, since there’s no GUI. It’s also a benefit that the VPN connection reduces the bandwidth. I’d like to ask if there’s a way to create a service that disconnects the existing VPN connection and takes the next VPN config file to establish a new connection upon receiving a specific error signal. Does yt-dlp have a return code that would indicate this type of error separately? Thanks in advance!

There is no such thing as a “free” VPN. Nothing in life is free, and they are likely selling your data to recuperate losses. Thats if they don’t throttle you to like 56K or have a 500 MiB data limit. Even then, this is far beyond the scope of this issue. Please read this. You’re making it harder for the yt-dlp devs to actually fix this issue.

Contributors, lock the issue.

Besides the number of videos downloaded within an hour, concurrency of downloads (and of their fragments) and rate-limiting or lack thereof may also have an influence.

I strongly rate-limit my downloads (<10Mbps) since a few years when Youtube started aggressively throttling them if I didn’t, have some sleep time between individual downloads and download exclusively serially, one at a time, and I’ve yet to be affected by this current issue (besides the various Invidious instances I used no longer working).

This is likely related to BotGuard where YouTube might be A/B testing/rolling some changes server-side. We are currently investigating this.

You’ve hit the request limit… Your limits will reset soon, usually within an hour.

At least it gave some more information. What happened after an hour?

Is this a clue, maybe?

Using an embedded youtube video it works like here with browser https://riks.se/riksronden-ska-eu-forskare-och-experter-styra-sverige/

but if I search that on youtube and access it need login.

  • either just clicking to watch it directly on youtube or search separately on youtube
  • as does yt-dlp also does needing sign in

To emulate being embedded might be a way to download it?

Just a thought…

If it really was about stopping bots Google could just throw their “I’m not a robot” recaptcha.

  • you can configure that to do really abusive tests as well as rather modest

So I agree with those thinking this is mostly to stop downloaders. This benefit both Google and those producing content worth throwing ads at, if they manage to do that.

And if logged into account at Google they can do more statistics if they get people to sign in more.

But incredible devs at yt-dlp manage to work around things before, so probably will again.

Going with Brave browser give the same “sign in…bot”-suggestion so it’s not just yt-dlp, it’s general problem and wanting people to sign in more, as well, as I see it.

  • and just on some files, not all
  • I went back and forth a couple of times, and only certain files give “bot”-message

And I do vpn all the time, and swapping servers and it might go through in the end. And all cookies and history is cleared in Brave every time I start it. So new ip all the time and still get it mostly on some files.

And the only reason why they are stopping downloaders is because it harms their bottom line. They went on a crusade against adblock and failed. This issue with them blocking downloaders is already sparking controversy as it broke music bots on Discord and it won’t be long before this (hopefully) reaches the mainstream.

Interested in how they’re identifying that yt-dlp requests are not the same as regular requests from their app or the browser 🤔 (presuming the same thing doesn’t happen if you watch too many videos in your browser)

You may have potentially found the key to working around this blockade that YouTube set up. yt-dlp may need to imitate and forge proper requests. But that may still not work; and the best solution is to avoid the block in the first place which sadly may mean slowing the connection or setting a counter for how many downloads you can do per day. It will cripple yt-dlp but at least work around the banning.

Might wanna see this. Using browser cookies is probably a terrible idea.

Thank you - sorry, I hadn’t seen that (or I wouldn’t have suggested it). Yes it does sound like a terrible idea 🤦

Interested in how they’re identifying that yt-dlp requests are not the same as regular requests from their app or the browser 🤔 (presuming the same thing doesn’t happen if you watch too many videos in your browser)

Works correctly for me on latest nightly version

I don’t think you’re affected by this problem because there’s been no change between the release and the nightly. I have the same error on the last release and the nightly as well.

I dont see any other workaround then just proxying/vpn it. Using yt-dlp WILL blacklist your ip, i am not sure for how long. Also do not use --cookies-from-browser, it will load so slowly if you use a web client then speeds up after about 3-4 hours.

How would i download youtube videos that require account cookies to access the page if i am not using cookies?

DO NOT PASS COOKIES. YOU WILL GET COOKIEBANNED.

UPDATE: I was able to get away with downloading ten videos in a row without encountering the block. Either I am extremely lucky, YouTube isn’t targeting people in the Midwestern United States or are not targeting Starlink customers.

However, I’m laying down the baseline that one video is fine, five is pushing it and ten+ is risky. The more you download, the more likely you may face this block.

Am still keen to hear what happened after the hour mentioned in the limit warning with oauth2, and also how many videos people are downloading before they run into this problem. @TheSystemGuy1337 mentioned they’d downloaded 2 or 3 but it wasn’t clear whether they encountered the block immediately after that. If the block can be lifted after an hour, and you have to download a really large number of videos before it’s applied, that makes addressing this a different priority to if it’s affecting people after only a few downloads and the block isn’t lifted after the hour.

I have not faced this block yet and I’ve downloaded around 10 to 15 videos over the past week. For whatever reason I tried replying to this via email and it didn’t go though, so if it dupes blame Gmail. I spread out the downloads over hours to days which is likely why I haven’t been ICBM’d by this block yet. If you don’t download frequently or spread them out then you won’t be targeted.

It seems to be easier/faster to get accounts blocked, than IPs, though. Which means there is something about the cookies and logged-in state specifically that is worth investigating.

I have doubts on referer; I haven’t noticed any issues using YouTube with a live browser (YET) and I often open links in a private/incognito tab.

Obviously VPNs are going to be an issue (as they already are for privacy apps for other social networks) given the much higher volume. How does Google tell the difference between a previously unknown VPN IP and a CGNAT IP, given how much CGNAT junk is going on these days?

have we considered also downloading an ad and waiting the length of the ad before buffering the main video? (hell maybe even feeding some dummy analytics to guise as a normal browser?)

When viewing videos through youtube.com with an adblocker like uBlock, I don’t get ads and I also don’t get the “please login” block.

That would indicate that the detection actually doesn’t take into account the ad-playing delay ?

have we considered also downloading an ad and waiting the length of the ad before buffering the main video? (hell maybe even feeding some dummy analytics to guise as a normal browser?)

implementing this stuff sounds like it’d be quite the feat but it might help with not triggering youtube’s ytdlp detection in the long run, and we can just put it behind a command line argument for people who don’t want to wait and are okay with risking being blocked

I’ve seen people say setting the user agent to edge helps avoid the a/b testing but I haven’t tested that with yt-dlp

oauth2 doesn’t seem to be a solution for downloading large numbers of youtube videos.

You’ve hit the request limit.. Your limits will reset soon, usually within an hour.

I’ll keep track. I have downloaded about 2 or 3 videos today as far as my memory goes

I tested further and seems to be about ip-addresses used and identified as malicious or similar

  • the regular vpn servers I use are all blocked
  • but picking one I never used before like Andorra went through with Duck player at least So vpn servers used by bots might be targeted and you as private user can be affected too.

It happends that protective scripts show a recaptcha to go through at various places using these scripts.

  • sometimes message as title of this thread
  • sometime just that you are blocked Vpn services are used by bots as well and create issues for all of us.

i would be curious whether this change affects things like DuckDuckGo’s youtube video embeds when you search for video results - if they somehow get an exception to this maybe that could lead to a workaround

To emulate being embedded might be a way to download it?

Note that the site you linked uses youtube.com and not youtube-nocookie.com, so it’s using your account’s cookies if you’re signed in. In case you didn’t already, you might want to try a private window.

I have not been signed into youtube for last 5 years, so I am not now.

I tried the /embed/ url thingy from view source, but that was not enough.

So something how it’s embedded make a difference it seems.

And again - it’s even with browser it refuses, not only yt-dlp

Thank you - sorry, I hadn’t seen that (or I wouldn’t have suggested it). Yes it does sound like a terrible idea 🤦

That was in a different repo, so you probably wouldn’t have found it anyways. No need to apologise

Interested in how they’re identifying that yt-dlp requests are not the same as regular requests from their app or the browser

My guess is either user agent, or what Kane suggested above

the browser might also send requests to the /api/stats/watchtime endpoint

Although ad blockers exist, so who knows

pass the browser cookies as a parameter?

Might wanna see this. Using browser cookies is probably a terrible idea.

Works correctly for me on latest nightly version