yt-dlp: [Nebula] Extractor failed to obtain "id"

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

I’m reporting that a supported site is broken
I’ve verified that I’m running yt-dlp version 2023.03.04 (update instructions) or later (specify commit)
I’ve checked that all provided URLs are playable in a browser with the same IP and same login details
I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
I’ve searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
I’ve read the guidelines for opening an issue
I’ve read about sharing account credentials and I’m willing to share it if required

Region

Provide a description that is worded well enough to be understood

I can still download older videos from Nebula, but recent videos from https://nebula.tv/videos/nothingbuttech-i-cloned-my-voice-to-expose-ai-drake on give an ‘extractor failed to obtain “id”’ error.

Please let me know if you require further information/login details.

Provide verbose output that clearly demonstrates the problem

Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
If using API, add 'verbose': True to YoutubeDL params instead
Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', '-f', 'bestvideo[height<=1440]+bestaudio/best[height<=1440]', '-o', 'H:/Videos/%(title)s-%(height)s.%(ext)s', '--embed-metadata', '--username', 'PRIVATE', '--password', 'PRIVATE', 'https://nebula.tv/videos/rmtransit-how-we-used-to-build-subways-fast-and-why-we-should-go-back']
[debug] Encodings: locale cp932, fs utf-8, pref cp932, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d] (pip)
[debug] Python 3.10.0 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1l  24 Aug 2021)
[debug] exe versions: ffmpeg 5.1-essentials_build-www.gyan.dev (setts), ffprobe 5.1-essentials_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2021.10.08, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.1
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.03.04, Current version: stable@2023.03.04
yt-dlp is up to date (stable@2023.03.04)
[Nebula] Logging in to Nebula with supplied credentials
[Nebula] Authorizing to Nebula
[Nebula] Extracting URL: https://nebula.tv/videos/rmtransit-how-we-used-to-build-subways-fast-and-why-we-should-go-back
[Nebula] rmtransit-how-we-used-to-build-subways-fast-and-why-we-should-go-back: Fetching video meta data
[Nebula] rmtransit-how-we-used-to-build-subways-fast-and-why-we-should-go-back: Fetching video stream info
[Nebula] rmtransit-how-we-used-to-build-subways-fast-and-why-we-should-go-back: Downloading m3u8 information
ERROR: [Nebula] Extractor failed to obtain "id"; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
Traceback (most recent call last):
  File "C:\Users\mrhut\AppData\Local\Programs\Python\Python310\lib\site-packages\yt_dlp\YoutubeDL.py", line 1518, in wrapper
    return func(self, *args, **kwargs)
  File "C:\Users\mrhut\AppData\Local\Programs\Python\Python310\lib\site-packages\yt_dlp\YoutubeDL.py", line 1615, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "C:\Users\mrhut\AppData\Local\Programs\Python\Python310\lib\site-packages\yt_dlp\YoutubeDL.py", line 1674, in process_ie_result
    ie_result = self.process_video_result(ie_result, download=download)
  File "C:\Users\mrhut\AppData\Local\Programs\Python\Python310\lib\site-packages\yt_dlp\YoutubeDL.py", line 2495, in process_video_result
    raise ExtractorError('Extractor failed to obtain "id"', ie=info_dict['extractor'])
yt_dlp.utils.ExtractorError: [Nebula] Extractor failed to obtain "id"; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

About this issue

Original URL
State: closed
Created a year ago
Reactions: 2
Comments: 20 (10 by maintainers)

Commits related to this issue

[nebula] change to new API URL * fix up tests * keep zype_id as archive ID if it exists Original Patch by @Lamieur on issue #7017 — committed to rohieb/yt-dlp by rohieb a year ago
[extractor/nebula] Fix extractor (#7156) Closes #7017 Authored by: Lamieur, rohieb Co-authored-by: Lam <github@Lam.pl> — committed to yt-dlp/yt-dlp by rohieb a year ago

Most upvoted comments

First problem is that there’s a new API:

current extractor uses https://content.watchnebula.com/video/…
but the website started using https://content.api.nebula.app/video/…

And the old link still works, but actually returns 404 for videos posted in the last 24h.

The new API’s reply is different, but the only important part is still there (manifest/m3u8).

Second problem: there’s no zype_id for the main video. You can pull it from another API call, but I don’t think it’s necessary (it’s not actually the real video id, it was used just to please yt-dlp).

If yt-dlp insists of having real ids, there actually are real video ids in Nebula (looks like “video_episode:8c2f6d8e-29ea-450f-bc16-51d896a2f59a” and is used in the API, like https://content.api.nebula.app/video_episodes/video_episode:8c2f6d8e-29ea-450f-bc16-51d896a2f59a/ - we might need to start using that API eventually). I’m removing the “video_episode:” part from the id in the patch, because it was getting mangled into “：” (also question marks in the titles are mangled into “？”) when actually downloading.

Trivial patch working for now:

--- nebula.py@2023.03.04	2023-03-06 01:00:00.000000000 +0100
+++ nebula.py	2023-05-09 19:02:14.063160007 +0200
@@ -65,7 +65,7 @@
         return response['token']
 
     def _fetch_video_formats(self, slug):
-        stream_info = self._call_nebula_api(f'https://content.watchnebula.com/video/{slug}/stream/',
+        stream_info = self._call_nebula_api(f'https://content.api.nebula.app/video/{slug}/stream/',
                                             video_id=slug,
                                             auth_type='bearer',
                                             note='Fetching video stream info')
@@ -77,7 +77,7 @@
         channel_slug = episode['channel_slug']
         channel_title = episode['channel_title']
         return {
-            'id': episode['zype_id'],
+            'id': episode['id'].replace('video_episode:',''),
             'display_id': episode['slug'],
             'formats': fmts,
             'subtitles': subs,
@@ -186,7 +186,7 @@
     ]
 
     def _fetch_video_metadata(self, slug):
-        return self._call_nebula_api(f'https://content.watchnebula.com/video/{slug}/',
+        return self._call_nebula_api(f'https://content.api.nebula.app/video/{slug}/',
                                      video_id=slug,
                                      auth_type='bearer',
                                      note='Fetching video meta data')

Lamieur on May 9, 2023

I’m happy to report that it not only works perfectly, it seems to download an order of magnitude faster than the last release version that worked with Nebula. (I have no data, that’s just my subjective impression.) I did not use aria2c.

Thanks go to the entire yt-dlp team, to @Lamieur, to @rohieb, and to @bashonly.

CarlFink on Jun 9, 2023

No idea what is the hold up for the patch, but for the size issue, you should be able to use the -F/--list-format option with yt-dlp to get a list of formats for the video, then put one of those formats behind the -f/--format option of yt-dlp.

rohieb on Jun 8, 2023

If my understanding of nebula.py is correct (and it likely isn’t), the issue is that that video’s zype_id (fetched from https://content.watchnebula.com/video/rmtransit-how-we-used-to-build-subways-fast-and-why-we-should-go-back/) is null

gamer191 on May 9, 2023