yt-dlp: [niconico] Free content gives "ERROR: This video is only available for registered users"

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Region

Japan

Provide a description that is worded well enough to be understood

Niconico video services came back online today, approximately 30 minutes ago. When trying to download videos from niconico using yt-dlp, it fails with the following error:

ERROR: [niconico] sm41399813: This video is only available for registered users. Use --cookies, --cookies-from-browser, --username and --password, --netrc-cmd, or --netrc (niconico) to provide account credentials

Niconico videos can be viewed in the browser without logging in, so this error should not happen. yt-dlp was able to successfully download niconico videos with the same parameters prior to the cyberattack incident.

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://www.nicovideo.jp/watch/sm41399813', '-f', 'mp4']
[debug] Encodings: locale cp932, fs utf-8, pref cp932, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.08.01 from yt-dlp/yt-dlp [ffd7781d6] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.22631-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 6.1-full_build-www.gyan.dev (setts), ffprobe 6.1-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.07.04, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.35.5, urllib3-2.2.2, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1830 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.08.01 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.08.01 from yt-dlp/yt-dlp)
[niconico] Extracting URL: https://www.nicovideo.jp/watch/sm41399813
[niconico] sm41399813: Downloading webpage
ERROR: [niconico] sm41399813: This video is only available for registered users. Use --cookies, --cookies-from-browser, --username and --password, --netrc-cmd, or --netrc (niconico) to provide account credentials
  File "yt_dlp\extractor\common.py", line 740, in extract
  File "yt_dlp\extractor\niconico.py", line 500, in _real_extract
  File "yt_dlp\extractor\common.py", line 1245, in raise_login_required

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 20 (12 by maintainers)

Commits related to this issue

Most upvoted comments

diff --git a/yt_dlp/extractor/niconico.py b/yt_dlp/extractor/niconico.py
index 9d7b010c5..fba05ed6f 100644
--- a/yt_dlp/extractor/niconico.py
+++ b/yt_dlp/extractor/niconico.py
@@ -458,9 +458,8 @@ def _real_extract(self, url):
             if video_id.startswith('so'):
                 video_id = self._match_id(handle.url)
 
-            api_data = self._parse_json(self._html_search_regex(
-                'data-api-data="([^"]+)"', webpage,
-                'API data', default='{}'), video_id)
+            api_data = self._parse_json(
+                self._html_search_meta('server-response', webpage), video_id)['data']['response']
         except ExtractorError as e:
             try:
                 api_data = self._download_json(

I pushed a quick fix for this in 4d9231208332d4c32364b8cd814bff8b20232cae just to get the extractor working at all, but anyone can feel free to open a PR to further fix the API fallback (I couldn’t get v3 or v3_guest to work)

As they completely remake streaming server, this make sense.

Obviously, these 3 lines of logic can be improved where yt-dlp will use either v3_guest or v3 endpoint based on auth status https://github.com/yt-dlp/yt-dlp/blob/e7d73bc4531ee3f91a46b15e218dcc1fbeb6226c/yt_dlp/extractor/niconico.py#L466-L469

However, I would prefer to first fix the regex expression I mentioned before, then we can think about how to improve the fallback method

I am having no issue accessing the API providing the necessary cookie. Here is the postman screenshot

image

However, debugging shows that api_data is empty image

So I am pretty sure the issue is on the yt-dlp side that should have handled the cookie value properly.

Meanwhile, the regex for fetching the api_data JSON (data-api-data="([^"]+)") requires change. I am glad that Niconico still preserve the “feature” where they still put the long JSON data on the webpage which makes us easier to fetch all necessary data. However it is renamed to server-response.

image

So I guess an easy regex change on line https://github.com/yt-dlp/yt-dlp/blob/e7d73bc4531ee3f91a46b15e218dcc1fbeb6226c/yt_dlp/extractor/niconico.py#L462 can unblock this parsing issue

I think it’s some other changes, from what I can see Niconico web is still getting the same m3u8 playlist from the previous endpoint as before. The streaming server has changed since last year, see: https://blog.nicovideo.jp/niconews/205042.html, and the extractor of yt-dlp had been updated for this, see: #9282. The response body of the web matches the code of yt-dlp, although I don’t know where the error occurs.

I have a potential conjecture: It appears that yt-dlp might need to extract video information (such as video titles) beforehand. Could it be because Niconico has altered its webpage structure (previously, the video information was hidden within a script named Ld-Json, but its location has now changed), resulting in yt-dlp’s inability to extract the necessary information properly, thus causing it to malfunction, rather than an issue with the streaming server?