yt-dlp: [twitter] x.com links give Unsupported URL error
DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
- I understand that I will be blocked if I intentionally remove or skip any mandatory* field
Checklist
- I’m reporting that yt-dlp is broken on a supported site
- I’ve verified that I have updated yt-dlp to nightly or master (update instructions)
- I’ve checked that all provided URLs are playable in a browser with the same IP and same login details
- I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
- I’ve searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
- I’ve read the guidelines for opening an issue
- I’ve read about sharing account credentials and I’m willing to share it if required
Region
Australia
Provide a description that is worded well enough to be understood
https://twitter.com has been changed to https://x.com, but yt-dlp is still performing a redirect to the former URL it does not need to, which seems to break the download.
Provide verbose output that clearly demonstrates the problem
- Run your yt-dlp command with -vU flag added (
yt-dlp -vU <your command line>) - If using API, add
'verbose': TruetoYoutubeDLparams instead - Copy the WHOLE output (starting with
[debug] Command-line config) and insert it below
Complete Verbose Output
[debug] Command-line config: ['-vU', 'https://x.com/historyinmemes/status/1790637656616943991']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.04.09 from yt-dlp/yt-dlp [ff0779267] (pip)
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-6.5.0-35-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2020.06.20, mutagen-1.47.0, pyxattr-0.7.2, requests-2.31.0, secretstorage-3.3.1, sqlite3-3.37.2, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1810 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.04.09 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.04.09 from yt-dlp/yt-dlp)
[generic] Extracting URL: https://x.com/historyinmemes/status/1790637656616943991
[generic] 1790637656616943991: Downloading webpage
[redirect] Following redirect to https://twitter.com/x/migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P
[generic] Extracting URL: https://twitter.com/x/migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P
[generic] migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL: https://twitter.com/x/migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P
Traceback (most recent call last):
File "/home/christian/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1606, in wrapper
return func(self, *args, **kwargs)
File "/home/christian/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1741, in __extract_info
ie_result = ie.extract(url)
File "/home/christian/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 734, in extract
ie_result = self._real_extract(url)
File "/home/christian/.local/lib/python3.10/site-packages/yt_dlp/extractor/generic.py", line 2514, in _real_extract
raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://twitter.com/x/migrate?tok=eyJlIjoiL2hpc3Rvcnlpbm1lbWVzL3N0YXR1cy8xNzkwNjM3NjU2NjE2OTQzOTkxIiwidCI6MTcxNTc2OTk4MH26emBettwJjp9vBvDz7z1P
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 2
- Comments: 35 (15 by maintainers)
Commits related to this issue
- [ie/twitter] Support x.com URLs (#9926) Closes #9923 Authored by: bashonly — committed to yt-dlp/yt-dlp by bashonly 3 years ago
its high priority bugfix because x.com is very popular site, probably second only to youtube
new yt-dlp version should be published.
i dont think it’s been said explicitly: substituting x.com with twitter.com seems to work
& it seems to be broken again.
I installed the nightly and public videos work fine, however, I cannot get it to download a protected video by passing a cookies.txt file from x.com (this is after a very recent change to their login system). I bet I’m not the only one affected and I don’t want to give out someone’s private link so here is just the error message without the verbose. Let me know if a verbose is truly necessary and I would have to set up a dummy protected acct to get one.
ERROR: [twitter] _________________: You are not authorized to view this protected tweet. Use --cookies, --cookies-from-browser, --username and --password, --netrc-cmd, or --netrc (twitter) to provide account credentials
Internally, the site still uses twitter domains. All we need to do is add
x.comto the_VALID_URLsBy “don’t use lazy extractors” I meant remove
yt_dlp/extractor/lazy_extractors.pyand don’t make it again@BETLOG Just update to the latest version
@ggous
you need to have updated to nightly
yt-dlp --update-to nightlyIf it still doesn’t work, then show a verbose log: add
-vto your download command and copy/paste the full output@InklingGirl already has:
python -m pip install -U --pre "yt-dlp[default]"