yt-dlp: Comedy Central Broken - New URL for m3u8?
DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
- I understand that I will be blocked if I intentionally remove or skip any mandatory* field
Checklist
- I’m reporting that yt-dlp is broken on a supported site
- I’ve verified that I have updated yt-dlp to nightly or master (update instructions)
- I’ve checked that all provided URLs are playable in a browser with the same IP and same login details
- I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
- I’ve searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
- I’ve read the guidelines for opening an issue
- I’ve read about sharing account credentials and I’m willing to share it if required
Region
United States/Wisconsin
Provide a description that is worded well enough to be understood
I believe the URL for the m3u8 files may have changed for Comedy Central (any they may be single files instead of a playlist) Below is last night’s episode of The Daily Show (Monday night’s worked fine). Opening the Network tab in the Developer tools does give me an m3u8 file, and that downloads successfully (I haven’t tested all 4, but the first one is the length of the video provided from the web browser player).
https://www.cc.com/episodes/1pc7nr/the-daily-show-december-5-2023-season-28-ep-116 https://topaz.viacomcbs.digital/h/a/dG9wYXoxYDQ3MjYwNmMwLWNjY2QtMTFlZC1hZDhlLTBlNDBjZjJmYzI4NWA2OTJmZDFkZmNjOTdmN2UzMjE5MzY3MGJjZmE5Y2JlYWY5OGZjYmY0YHI2akVvUHJINFNaSHFxNnJKOUFkRUZFeGxtTXRNel82VDBJN0xVWUVOcTA/master.m3u8?mgid=mgid:arc:episode:comedycentral.com:472606c0-cccd-11ed-ad8e-0e40cf2fc285&ts=1701854196-0&pthash=692fd1dfcc97f7e32193670bcfa9cbeaf98fcbf4&hdnts=exp=1701872554~acl=/h/a/dG9wYXoxYDQ3MjYwNmMwLWNjY2QtMTFlZC1hZDhlLTBlNDBjZjJmYzI4NWA2OTJmZDFkZmNjOTdmN2UzMjE5MzY3MGJjZmE5Y2JlYWY5OGZjYmY0YHI2akVvUHJINFNaSHFxNnJKOUFkRUZFeGxtTXRNel82VDBJN0xVWUVOcTA/*~hmac=793604cbfb66fd4ffa6b62ef1da823403d72eb3ee241ad31318ffbf2f2423b85&CMCD=mtp%3D500%2Cot%3Dm%2Csf%3Dh%2Csid%3D"3511f061-09ec-4b7c-9dc5-784e455683e0"%2Csu https://topaz.viacomcbs.digital/h/a/dG9wYXoxYDQ3MjYwNmMwLWNjY2QtMTFlZC1hZDhlLTBlNDBjZjJmYzI4NWA2OTJmZDFkZmNjOTdmN2UzMjE5MzY3MGJjZmE5Y2JlYWY5OGZjYmY0YHI2akVvUHJINFNaSHFxNnJKOUFkRUZFeGxtTXRNel82VDBJN0xVWUVOcTA/hdntl=exp=1701958655~acl=%2F*~data=hdntl~hmac=4f41f1548f0379efea466f4bba895180340c1d80fae2a1dd17541feb12f7b9f6/index-f1-v1.m3u8?CMCD=mtp%3D500%2Cot%3Dm%2Csf%3Dh%2Csid%3D"3511f061-09ec-4b7c-9dc5-784e455683e0"%2Csu https://topaz.viacomcbs.digital/h/a/dG9wYXoxYDQ3MjYwNmMwLWNjY2QtMTFlZC1hZDhlLTBlNDBjZjJmYzI4NWA2OTJmZDFkZmNjOTdmN2UzMjE5MzY3MGJjZmE5Y2JlYWY5OGZjYmY0YHI2akVvUHJINFNaSHFxNnJKOUFkRUZFeGxtTXRNel82VDBJN0xVWUVOcTA/hdntl=exp=1701958655~acl=%2F*~data=hdntl~hmac=4f41f1548f0379efea466f4bba895180340c1d80fae2a1dd17541feb12f7b9f6/index-f8-a1.m3u8?CMCD=mtp%3D500%2Cot%3Dm%2Csf%3Dh%2Csid%3D"3511f061-09ec-4b7c-9dc5-784e455683e0"%2Csu https://topaz.viacomcbs.digital/h/a/dG9wYXoxYDQ3MjYwNmMwLWNjY2QtMTFlZC1hZDhlLTBlNDBjZjJmYzI4NWA2OTJmZDFkZmNjOTdmN2UzMjE5MzY3MGJjZmE5Y2JlYWY5OGZjYmY0YHI2akVvUHJINFNaSHFxNnJKOUFkRUZFeGxtTXRNel82VDBJN0xVWUVOcTA/hdntl=exp=1701958655~acl=%2F*~data=hdntl~hmac=4f41f1548f0379efea466f4bba895180340c1d80fae2a1dd17541feb12f7b9f6/index-f7-v1.m3u8?CMCD=mtp%3D61400%2Cot%3Dm%2Csf%3Dh%2Csid%3D"3511f061-09ec-4b7c-9dc5-784e455683e0"%2Csu
Provide verbose output that clearly demonstrates the problem
- Run your yt-dlp command with -vU flag added (
yt-dlp -vU <your command line>) - If using API, add
'verbose': TruetoYoutubeDLparams instead - Copy the WHOLE output (starting with
[debug] Command-line config) and insert it below
Complete Verbose Output
>$ yt-dlp -vU https://www.cc.com/episodes/tpg24h/star-trek-very-short-treks-skin-a-cat-season-1-ep-1
[debug] Command-line config: ['-vU', 'https://www.cc.com/episodes/tpg24h/star-trek-very-short-treks-skin-a-cat-season-1-ep-1']
[debug] User config "/home/stephen/.config/yt-dlp/config": ['-f', '[ext=mp4][height<=720]/[ext=mp4][height<=1080]/[ext=mp4][height<=720]+bestaudio/[ext=mp4][height<=1080]+bestaudio/best', '--write-sub', '--sub-lang', 'en', '--extractor-args', 'twitch:subs=False']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.11.16 from yt-dlp/yt-dlp [24f827875]
[debug] Python 3.11.6 (CPython x86_64 64bit) - Linux-6.1.64-1-lts-x86_64-with-glibc2.38 (OpenSSL 3.1.4 24 Oct 2023, glibc 2.38)
[debug] exe versions: ffmpeg 6.1 (setts), ffprobe 6.1, phantomjs broken, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.12.0, certifi-2023.11.17, sqlite3-3.44.2, websockets-10.4
[debug] Proxy map: {}
[debug] Request Handlers: urllib
[debug] Loaded 1901 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2023.11.16 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2023.11.16 from yt-dlp/yt-dlp)
[ComedyCentral] Extracting URL: https://www.cc.com/episodes/tpg24h/star-trek-very-short-treks-skin-a-cat-season-1-ep-1
[ComedyCentral] star-trek-very-short-treks-skin-a-cat-season-1-ep-1: Downloading webpage
[ComedyCentral] 41738d55-83e9-11ee-ad8e-0e40cf2fc285: Downloading info
[ComedyCentral] 41738d55-83e9-11ee-ad8e-0e40cf2fc285: Extracting information
[ComedyCentral] 41738d55-83e9-11ee-ad8e-0e40cf2fc285: Downloading video urls
ERROR: [ComedyCentral] tpg24h: ComedyCentral returned error: not_found -
Sorry, this video is not found or no longer available due to date or rights restrictions.
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 715, in extract
ie_result = self._real_extract(url)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/mtv.py", line 323, in _real_extract
videos_info = self._get_videos_info(mgid, url=url)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/mtv.py", line 227, in _get_videos_info
return self._get_videos_info_from_url(info_url, video_id, use_hls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/mtv.py", line 239, in _get_videos_info_from_url
info = self._get_video_info(item, use_hls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/mtv.py", line 150, in _get_video_info
raise ExtractorError(message, expected=True)
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 20 (4 by maintainers)
Hello all,
In honor of Jon Stewart’s upcoming return to The Daily Show, I am happy to report that I was able to debug and fix this on my machine. I’m very poor at Python however so my guess is that anyone looking at the code will want to suggest modifications. Although the subtitles are in the HLS manifest so far I haven’t figured out how to extract them yet. Right now I get the following warning by the tool and subtitle base64 junk at the bottom of the video:
Ignoring subtitle tracks found in the HLS manifest; if any subtitle tracks are missing, please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -UThere are a lot of overloaded and modified versions of the same functions so I might have broken other MTV properties like South Park or Nickelodeon. There also might be a better way to extract and promote more information in the video information — right now the title is the end of the episode url and I guess we could promote the episode and season from that as well.
If someone in the community could help me get this over the finish line, I’d be much obliged.
Basically,
The reference URL I used was:
https://www.cc.com/episodes/fppd9b/the-daily-show-december-14-2023-taraji-p-henson-season-28-ep-122
To apply the patch and help test:
_FEED_URL = 'https://topaz.viacomcbs.digital/topaz/api/'This appears to work for newer content — it does not work for things published before about 12/8/2023.
@pishposhmcgee , thanks for the subtitles and testing legwork. I’ve been meaning to spend some time to understand the testing framework a bit more before proceeding with this.
Oddly the videos that are being extracted do still have a junk subtitle stream in the mp4. Is anyone else seeing this? It appears to be extracted somehow downstream and the m3u8 doesn’t appear to have multiple references to it so I’m not sure yet where it’s coming from.
Regarding playlists, I don’t know how it worked before, but functionally from what I can see there’s nothing that separates pages like /episodes, /videos, and /playlists, both serve up the current video and then on completion redirects to the next one on the queue one is just presented more explicitly as a theme whereas other one serves up additional pre-selected related content. Typically I would think a traditional playlist would have multiple media references in the feed (here, mica.json) and that doesn’t seem to be the case. I welcome any corrections if my understanding is incorrect.
Previously, CC’s extractor used to grab each chapter/segment and I needed to stitch them together explicitly. Now the feed has a parent section that references the whole thing sans ads, but in the few cases I’ve tested the feed content section only contains 1 media file and the playlist information are urls passed by some other mechanism that in turn could be iterated and have the M3u8 extracted, but I don’t think most playlists are extracted this way.
I don’t have a timeframe yet but I can message this a bit more and move the feed URL to the base class and move forward with a PR.
Regarding backwards compatibility, I’m not sure if that’s a bar that’s been consistently cleared by other extractors as most media references tend to rotate in these streaming services (or consolidated in otther services 😉). But I can look at the error to see if forking logic could try to do a good faith attempt at older content.