yt-dlp: [MSN] Download videos from msn failed with error "Unable to extract error".
Checklist
- I’m reporting a broken site
- I’ve verified that I’m running yt-dlp version 2022.03.08.1. (update instructions)
- I’ve checked that all provided URLs are alive and playable in a browser
- I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
- I’ve searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
- I’ve read the guidelines for opening an issue
- I’ve read about sharing account credentials and I’m willing to share it if required
Region
Singapore
Description
Could anyone help me with this? Thanks.
PS C:\Users\v-owendeng> yt-dlp https://www.msn.com/en-us/news/local/jeffco-public-schools-says-no-limit-on-guests-for-end-of-year-events/vi-BB1gIARh [MSN] jeffco-public-schools-says-no-limit-on-guests-for-end-of-year-events: Downloading webpage ERROR: [MSN] BB1gIARh: Unable to extract error; please report this issue on https://github.com/yt-dlp/yt-dlp , filling out the “Broken site” issue template properly. Confirm you are on the latest version using yt-dlp -U
Here are 3 test URLs: https://www.msn.com/en-us/video/cardio/crash-shuts-down-portion-of-brownsville-road/vi-AAL8z9q https://www.msn.com/en-us/news/local/jeffco-public-schools-says-no-limit-on-guests-for-end-of-year-events/vi-BB1gIARh https://www.msn.com/en-us/video/peopleandplaces/boston-mayor-janey-to-sign-measure-limiting-police-use-of-tear-gas/vp-BB1gchKd
Verbose log
PS C:\Users\v-owendeng> yt-dlp -vU "https://www.msn.com/en-us/video/cardio/crash-shuts-down-portion-of-brownsville-road/vi-AAL8z9q"
[debug] Command-line config: ['-vU', 'https://www.msn.com/en-us/video/cardio/crash-shuts-down-portion-of-brownsville-road/vi-AAL8z9q']
[debug] Encodings: locale cp936, fs utf-8, out utf-8, err utf-8, pref cp936
[debug] yt-dlp version 2022.03.08.1 [c0c2c57] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] exe versions: ffmpeg 2022-03-17-git-242c07982a-full_build-www.gyan.dev (setts), ffprobe 2022-03-17-git-242c07982a-full_build-www.gyan.dev
[debug] Optional libraries: brotli, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2022.03.08.1, Current version: 2022.03.08.1
yt-dlp is up to date (2022.03.08.1)
[debug] [MSN] Extracting URL: https://www.msn.com/en-us/video/cardio/crash-shuts-down-portion-of-brownsville-road/vi-AAL8z9q
[MSN] crash-shuts-down-portion-of-brownsville-road: Downloading webpage
ERROR: [MSN] AAL8z9q: Unable to extract error; please report this issue on https://github.com/yt-dlp/yt-dlp , filling out the "Broken site" issue template properly. Confirm you are on the latest version using yt-dlp -U
File "yt_dlp\extractor\common.py", line 617, in extract
File "yt_dlp\extractor\msn.py", line 166, in _real_extract
File "yt_dlp\extractor\common.py", line 1192, in _search_regex
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 20 (10 by maintainers)
Commits related to this issue
- [ie/msn] Rework extractor (#12513) Closes #3225 Authored by: thedenv, seproDev Co-authored-by: sepro <sepro@sepr0.com> — committed to vstavrinov/yt-dlp by thedenv 2 years ago
- [ie/msn] Rework extractor (#12513) Closes #3225 Authored by: thedenv, seproDev Co-authored-by: sepro <sepro@sepr0.com> — committed to iribeirocampos/yt-dlp by thedenv 2 years ago
- [ie/msn] Rework extractor (#12513) Closes #3225 Authored by: thedenv, seproDev Co-authored-by: sepro <sepro@sepr0.com> — committed to subrat-lima/yt-dlp by thedenv 2 years ago
That seems to apply here exactly.
In this case it seems like the URL pattern should be tweaked to extract
localeas the xx-xx initial path component and then, instead of fetching the webpage, get the JSON fromf'https://assets.msn.com/content/view/v2/Detail/{locale}/{page_id)'.The page from https://github.com/yt-dlp/yt-dlp/issues/3225#issuecomment-1506285806 gets 410 Gone for this; MSN still serves a page full of clickbait and ads without telling you that it’s not the page you expected.
A current URL in the same area (https://www.msn.com/en-us/weather/topstories/midwest-northeast-bracing-for-potentially-dangerous-long-duration-heat-wave-next-week/vi-BB1oaoek?ocid=windirect&cvid=981228a2079b4d07908de15c5820a4e2&ei=98) with
localeen-usandpage_idBB1oaoekyields valid JSON. In the JSON, there are these potentially useful values:Then (using upstream code):
@nealmcb PhantomJS is only used by 4 extractors, and it’s only used as a last resort when there is no other path to extraction except by actually executing the site’s javascript. (And for the most widely used of those 4 extractors, there is an alternative available). PhantomJS is likely not relevant to this issue.
If you’re writing an extractor and the information you need isn’t found in the page source/html, then what you should try to do is to find out what the javascript is doing to get the required info and recreate that in the extractor code; this usually amounts to making an API request (or multiple). In your browser’s dev tools you can monitor all network requests being made as the page/video loads. Start there; more often than not the XHR calls will tell you all you need to know and you won’t even need to look at the javascript code itself.
Congratulations and thanks to you all for helping improve yt-dlp. SInce there are so many PRs active, for the record, here is the MSN PR: https://github.com/yt-dlp/yt-dlp/pull/12513
On Sun, Mar 2, 2025 at 7:21 AM thedenv @.***> wrote:
– Neal McBurnett https http://neal.mcburnett.org/😕/ bcn.boulder.co.us/~neal/
You have still not created a Pull Request. Go here https://github.com/yt-dlp/yt-dlp/compare/master...thedenv:yt-dlp:master and select the branch you want to PR. Me or another maintainer will then review your code.
Yes you would fork yt-dlp, create a new branch, then modify
yt-dlp/extractor/msn.py, and then create a pull request. Do take a look at the yt-dlp coding conventions: https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions You will need to adjust the logic of your script a bit. For example in yt-dlp you can not import requests directly, but would use helper functions such asself._download_json@thedenv Nice work. Instead of having this be an external script, consider adjusting the MSN extractor in yt-dlp and opening a PR. That way people can just use yt-dlp directly instead of having to download and setup your script.
The problem MSN URLs mentioned in existing issues no longer lead to the expected content as far as I can tell.
Someone could open an issue upstream (or post here) suggesting various currently valid MSN URLs with playable media that should be supported, describing for each what is expected to be found (one MSN video, as above, a video from YT or DM or some other external host, a playlist of any of these). Suggestions as to what values should be expected for the standard JSON extraction parameters would also be useful.
What the video above gets
Then extractor patches can be validated against those and via a PR upstream and the process of pulling upstream changes to yt-dlp (or a PR here if someone makes one) your wishes could be realised.