yt-dlp: goplay extractor broken
DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
- I understand that I will be blocked if I remove or skip any mandatory* field
Checklist
- I’m reporting a broken site
- I’ve verified that I’m running yt-dlp version 2023.01.06 (update instructions) or later (specify commit)
- I’ve checked that all provided URLs are playable in a browser with the same IP and same login details
- I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
- I’ve searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
- I’ve read the guidelines for opening an issue
- I’ve read about sharing account credentials and I’m willing to share it if required
Region
Belgium
Provide a description that is worded well enough to be understood
goplay.be now loads newer videos in a new format. (DASH MPD) which means the current extractor is broken.
Extracted the following information so far;
curl -X POST https://pubads.g.doubleclick.net/ondemand/dash/content/2615619/vid/10150078534976527/streams
{
"stream_id": "091f9127-c615-4354-9006-af3fb09ac261",
"duration": 2687.64,
"stream_type": "on_demand",
"stream_manifest": "https://pubads.g.doubleclick.net/ondemand/dash/content/2615619/vid/10150078534976527/GRQ/streams/091f9127-c615-4354-9006-af3fb09ac261/manifest.mpd",
"manifest_format": "dash",
"time_events_url": "https://pubads.g.doubleclick.net/ondemand/dash/content/2615619/vid/10150078534976527/GRQ/streams/091f9127-c615-4354-9006-af3fb09ac261/time-events.json"
}
Provide verbose output that clearly demonstrates the problem
- Run your yt-dlp command with -vU flag added (
yt-dlp -vU <your command line>) - Copy the WHOLE output (starting with
[debug] Command-line config) and insert it below
Complete Verbose Output
[debug] Command-line config: ['--verbose', '-o', '/media/data2/series/Big Brother/Season 3/%(series)s - S0%(season_number)sE%(episode_number)s.%(ext)s', '--no-hls-use-mpegts', '-u', 'PRIVATE', '-p', 'PRIVATE', '-N', '8', '-f', 'bestvideo+bestaudio', '--merge-output-format', 'mp4', 'https://www.goplay.be/video/big-brother/big-brother-s3/big-brother-s3-aflevering-32']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8 (No ANSI), error utf-8 (No ANSI), screen utf-8 (No ANSI)
[debug] yt-dlp version 2023.01.06 [6becd2508] (linux_exe)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-5.15.0-60-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, secretstorage-3.3.1, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1767 extractors
[GoPlay] Logging in
[GoPlay] Authenticating username
[GoPlay] Authenticating password
[GoPlay] Extracting URL: https://www.goplay.be/video/big-brother/big-brother-s3/big-brother-s3-aflevering-32
[GoPlay] big-brother-s3-aflevering-32: Downloading webpage
[GoPlay] 5bd06f78-6e20-439b-8d67-cfe152c0cd68: Downloading JSON metadata
ERROR: An extractor error has occurred. (caused by KeyError('manifestUrls')); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "yt_dlp/extractor/common.py", line 693, in extract
File "yt_dlp/extractor/goplay.py", line 83, in _real_extract
KeyError: 'manifestUrls'
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 28 (5 by maintainers)
Commits related to this issue
- [extractor/goplay] Fix extractor (#6654) Authored by: alard Closes #6235 — committed to yt-dlp/yt-dlp by alard 4 months ago
I figured out how to merge the files successfully with the help of this stackoverflow article: first make a document with all names of the files that looks like this
and then execute the following ffmpeg commando
$ ffmpeg -f concat -safe 0 -i mylist.txt -c copy output.mp4I downloaded the manifest file and checked it
if we remove the preroll + ads
Every part has 1080p 720p … and 1 audio link
Then I downloaded them using
part1: yt-dlp -f qualityID+audioID <url to manifest> part2: yt-dlp -f qualityID+audioID <url to manifest> part3: yt-dlp -f qualityID+audioID <url to manifest>
As an alternative, you could concatenate the
fragmentslist for all periods and have them downloaded byyt-dlpas a single file.I made an update to the goplay extractor that does this. It works on the recent De Mol streams.
It’s not ideal to do this at the goplay extractor level, I guess. I assume other sources with multi-period MPD streams will have the same problem, so it would be better if the MPD code could separate the periods (and perhaps merge them).
Thanks! Currently at work so I’ll check myself later, much appreciated! How did you figure out what audio goes with what video?
EDIT nvm doesn’t seem to be relevant if you actually download the manifest. Using the -F flag gives me everything at once, so 3 audio files, and 3 video files per resolution
I haven’t looked into this yet either. Since the GoPlay app now supports AirPlay I’m using that instead of downloading it first.
Don’t count on me too much, all I did last time was apply patches from different people together in a new branch, I didn’t actually go into the details of how the extractor works.
Hopefully someone can fix it in time!
Doesn’t look like it’s just newer videos - for example, I could download De Rechtbank S12E05 just fine last sunday using the “old” URL format, but S12E02 right now has the new format too.
Thanks for your investigation though, I thought I was going crazy. I just verified your findings for another video 😊
the api of goplay.be returns following:
GET https://api.goplay.be/web/v1/videos/long-form/26e19146-57ef-4ea4-ab41-e247061d8407It then does a POST to
https://pubads.g.doubleclick.net/ondemand/dash/content/2615619/vid/10149680614692527/streams(2615619being thecontentSourceIDand10149680614692527thevideoIdDoubleclick returns the
stream_manifestwhich contains the video including ads and prerolls.Downloading the MPD file directly via yt-dlp {stream_manifest} gives me very mixed results - it seems the audio and video are now split up in 3 parts…