yt-dlp: [9now.com.au] Unable to extract page data
Checklist
- I’m reporting that yt-dlp is broken on a supported site
- I’ve verified that I have updated yt-dlp to nightly or master (update instructions)
- I’ve checked that all provided URLs are playable in a browser with the same IP and same login details
- I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
- I’ve searched known issues, the FAQ, and the bugtracker for similar issues including closed ones. DO NOT post duplicates
- I’ve read about sharing account credentials and I’m willing to share it if required
Region
Australia
Provide a description that is worded well enough to be understood
yt-dlp now outputs error when tring to download webpage for data scrape.
If you go into devtools the m3u8 is still visible and that can be inputted into yt-dlp without issue. Looks to simple be webpage download step that is failing.
Provide verbose output that clearly demonstrates the problem
- Run your yt-dlp command with -vU flag added (
yt-dlp -vU <your command line>) - If using API, add
'verbose': TruetoYoutubeDLparams instead - Copy the WHOLE output (starting with
[debug] Command-line config) and insert it below
Complete Verbose Output
[debug] Command-line config: ['-vU', 'https://www.9now.com.au/married-at-first-sight/season-12/episode-1']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2025.03.07.232704 from yt-dlp/yt-dlp-nightly-builds [05c8023a2] (win_exe)
[debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.22631-SP0 (OpenSSL 1.1.1t 7 Feb 2023)
[debug] exe versions: ffmpeg N-116754-ga754ee0844-20240821 (setts), ffprobe N-116754-ga754ee0844-20240821
[debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2025.01.31, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.3.0, websockets-15.0.1
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Plugin directories: none
[debug] Loaded 1844 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Latest version: nightly@2025.03.07.232704 from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@2025.03.07.232704 from yt-dlp/yt-dlp-nightly-builds)
[debug] Using fake IP 1.134.182.26 (AU) as X-Forwarded-For
[9now.com.au] Extracting URL: https://www.9now.com.au/married-at-first-sight/season-12/episode-1
[9now.com.au] episode-1: Downloading webpage
ERROR: [9now.com.au] episode-1: Unable to extract page data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "yt_dlp\extractor\common.py", line 747, in extract
File "yt_dlp\extractor\ninenow.py", line 68, in _real_extract
File "yt_dlp\extractor\common.py", line 1338, in _search_regex
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 3
- Comments: 46 (9 by maintainers)
Commits related to this issue
- [ie/9now.com.au] Fix extractor (#12702) Closes #12591 Authored by: bashonly — committed to yt-dlp/yt-dlp by bashonly 2 years ago
- [ie/9now.com.au] Fix extractor (#12702) Closes #12591 Authored by: bashonly — committed to subrat-lima/yt-dlp by bashonly 2 years ago
This issue is happening again
Yes, as long as the
episodesorclipsJSON isn’t split between chunks, this can be done. There are many tens ofbrightcoveIds in (eg) the page for MAFS s12e2, but what we want is the JSON withitemsthat list the episodes (or clips if it’s a clip page) for the show, specifically the item whose.link.webUrlmatches the path part of the URL. That JSON is equivalent to what the old extraction was finding.Here's a patch that might implement the above idea, but it's quite extensive:
(Style tweaked for
.encode())I’m having that same problem to. Also it worked for me to yesterday and other previous days
Thanks. It fully working now. Using the lastest master version source
For those that need it in clearer instructions on how to download while this issue is happening, see the video attached. The quality isn’t the best but gives enough.
In chrome: ChromeDev tools (right click Inspect), select the Network tab, type m3u8 in the filter and select Fetch/XHR to help filter it more. Click Stop recording network log or press Ctrl/cmd + E Click Clear network log or Ctrl/cmd + L Click Start recording network log or press Ctrl/cmd + E Refresh the page.
Wait till you see the master.m3u8 file in the network log Click it, then to the right you’ll see Request URL. Copy the whole url, not just to the end of .m3u8
I had issues playing the files in Plex when using “-f bestvideo+bestaudio --merge-output-format mp4”
This is the command I use:
yt-dlp -S “ext,height:720” --referer “The base domain url the video is hosted on” “The m3u8 Request URL”
This is MAFS episode 31 with the full m3u8 link as an example:
yt-dlp -S “ext,height:720” --referer “https://www.9now.com.au” “https://manifest.prod.boltdns.net/manifest/v1/hls/v5/clear/4460760524001/5e78af1b-9ab5-4bde-b9ec-10912859a663/10s/master.m3u8?fastly_token=NjdkZDA2MGVfOTA1NWVmODk1ZDI1ZmUxOTRkMGM4OTdiOWUwNjc3NjBjZTFmYTMxZWM3NmE3ZGY3NTcwNGViZThlYjFjNTFhZQ%3D%3D”
The referer line isn’t needed for 9Now but that’s the same command I use for websites where the videos are private and harder coded to the website (like educational course websites)
As others have said, you still need ffmpeg installed.
And MAFS Episode 32 if you can’t be bothered yt-dlp -S “ext,height:720” --referer “https://www.9now.com.au” “https://manifest.prod.boltdns.net/manifest/v1/hls/v5/clear/4460760524001/2b2f9893-893b-4202-8d94-6609849f966b/10s/master.m3u8?fastly_token=NjdkYzhlYzBfYTZjZWFjMWM4MDNjOWQ3ZjJmZTRlZjBmZDU1YWZlMDFlNzc2MmYxYTkzZWU4ZmQwYzY2NzRiYjFmNzI3YWM1Mg%3D%3D”
For those outside of Aus If you haven’t already noticed, use a VPN server in Aus and it’ll download significantly quicker
https://github.com/user-attachments/assets/a2f015b5-91ab-41ee-a4ac-1fc3a80d148a
Its not so bad. As a test run I was able to search for the “brightcoveId” and then replace the bright cove template variable with the id directly and it proceeded to download just fine. The “brightcoveId” appears to be intact within the NextJS framework, meaning full reassembly is probably not a requirement for a useable extractor.
Just to clarify the problem, which is not specific to this site:
So, not straightforward.
It was happening with me, then I used
-f bestvideo+bestaudio --merge-output-format mp4tags and it worked. Note, make sure you have ffmpeg installed, otherwise you will see this warning and then you will have 2 files (one for audio and one for video):MY MISTAKE
I apologise,
I was using youtube-dl instead of yt-dlp
Please disregard my error reports of the past day.
I apologise for wasting anyone’s time, I made a mistake.
I am
striking throughthe posts I made in error.I am happy to report this has been resolved. Thank you.
@FlaminHat I’ll show how it is possible to install the latest version
Test current version
Uninstall: brew uninstall yt-dlp
Install latest: brew install --HEAD yt-dlp
Test latest version
Since the page is just the result of the satanic tooling, any technically valid structure could be generated, regardless of common sense. Some show with an enormous number of episodes might reveal problems, but by then (looks down towering WIP/TODO stack) we might have an extractor routine that recombines the new-style Next.js chunks.
I’ll update the upstream extractor but anyone is welcome to pick up the patch and PR it here, the more the merrier (use the patch code under Unlicense).
I’m still confused about the geo-restriction situation, though, but that’s obviously a second-order issue.
Was working for MAFS with an automatic bash script that I have used for years with all these kinds of auto-fetch from TV websites:
~/bin/yt-dlp -f ‘bestvideo[ext=mp4]+bestaudio[ext=m4a]/bestvideo+bestaudio’ --merge-output-format mp4 https://www.9now.com.au/married-at-first-sight/season-12/episode-32
up until last night. Now even if I go back to the episodes that worked before (so 31 back to 28), all give this same error.
~/bin/yt-dlp -vU https://www.9now.com.au/married-at-first-sight/season-12/episode-32 looks happy to start with but then I get:
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest Latest version: nightly@2025.03.16.232921 from yt-dlp/yt-dlp-nightly-builds yt-dlp is up to date (nightly@2025.03.16.232921 from yt-dlp/yt-dlp-nightly-builds) [debug] Using fake IP 1.144.162.110 (AU) as X-Forwarded-For [9now.com.au] Extracting URL: https://www.9now.com.au/married-at-first-sight/season-12/episode-32 [9now.com.au] episode-32: Downloading webpage ERROR: [9now.com.au] episode-32: Unable to extract page data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U File “yt_dlp/extractor/common.py”, line 747, in extract File “yt_dlp/extractor/ninenow.py”, line 68, in _real_extract File “yt_dlp/extractor/common.py”, line 1338, in _search_regex
I also tried my username and password options last night but it still gave the same error. It doesn’t look like a DRM issue but feels kind of remote end changed something-ish.
Also tried another show in my autv download setup which gives the same error:
https://www.9now.com.au/space-invaders/season-5/episode-6
I am a Linux bash based user - been at the UNIX/Linux command line since 1987 so am an old dog at all of this but not a python expert by any stretch.
Note for devs in case this needs to be reopened:
I took a look at the page source last night before they rolled back the change, and it looked like the site was using the new version of NextJS with all the push calls. Fun stuff
Network tab
…master.m3u8…
yt-dlp.exe “m3u8 url”