yt-dlp: [twitter] x.com links give Unsupported URL error

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Region

Australia

Provide a description that is worded well enough to be understood

https://twitter.com has been changed to https://x.com, but yt-dlp is still performing a redirect to the former URL it does not need to, which seems to break the download.

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://x.com/historyinmemes/status/1790637656616943991']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.04.09 from yt-dlp/yt-dlp [ff0779267] (pip)
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-6.5.0-35-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2020.06.20, mutagen-1.47.0, pyxattr-0.7.2, requests-2.31.0, secretstorage-3.3.1, sqlite3-3.37.2, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1810 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.04.09 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.04.09 from yt-dlp/yt-dlp)
[generic] Extracting URL: https://x.com/historyinmemes/status/1790637656616943991
[generic] 1790637656616943991: Downloading webpage
[redirect] Following redirect to https://twitter.com/x/migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P
[generic] Extracting URL: https://twitter.com/x/migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P
[generic] migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL: https://twitter.com/x/migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P
Traceback (most recent call last):
  File "/home/christian/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1606, in wrapper
    return func(self, *args, **kwargs)
  File "/home/christian/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1741, in __extract_info
    ie_result = ie.extract(url)
  File "/home/christian/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 734, in extract
    ie_result = self._real_extract(url)
  File "/home/christian/.local/lib/python3.10/site-packages/yt_dlp/extractor/generic.py", line 2514, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://twitter.com/x/migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 2
  • Comments: 35 (15 by maintainers)

Commits related to this issue

Most upvoted comments

its high priority bugfix because x.com is very popular site, probably second only to youtube

new yt-dlp version should be published.

i dont think it’s been said explicitly: substituting x.com with twitter.com seems to work

& it seems to be broken again.

I installed the nightly and public videos work fine, however, I cannot get it to download a protected video by passing a cookies.txt file from x.com (this is after a very recent change to their login system). I bet I’m not the only one affected and I don’t want to give out someone’s private link so here is just the error message without the verbose. Let me know if a verbose is truly necessary and I would have to set up a dummy protected acct to get one.

ERROR: [twitter] _________________: You are not authorized to view this protected tweet. Use --cookies, --cookies-from-browser, --username and --password, --netrc-cmd, or --netrc (twitter) to provide account credentials

Internally, the site still uses twitter domains. All we need to do is add x.com to the _VALID_URLs

I dont understand your last suggestion, I thought it might be a flag I could pass?

By “don’t use lazy extractors” I meant remove yt_dlp/extractor/lazy_extractors.py and don’t make it again

@BETLOG Just update to the latest version

@ggous

you need to have updated to nightly

yt-dlp --update-to nightly

If it still doesn’t work, then show a verbose log: add -v to your download command and copy/paste the full output

@InklingGirl already has: python -m pip install -U --pre "yt-dlp[default]"