youtube-dl: youtube_dl.jsinterp.JSInterpreter.Exception: slice must be applied on a list in: 'a.slice(0,0)

Checklist

  • I’m reporting a broken site support issue
  • I’ve verified that I’m running youtube-dl version 2024.08.02
  • I’ve checked that all provided URLs are alive and playable in a browser
  • I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
  • I’ve searched the bugtracker for similar bug reports including closed ones
  • I’ve read bugs section in FAQ

Verbose log

Latest version: 2024.08.02, Current version: 2024.08.02
youtube-dl is up to date (2024.08.02)
[youtube] LG9G4aA28rU: Downloading webpage
[youtube] LG9G4aA28rU: Downloading player b12cc44b
WARNING: [youtube] Falling back to generic n function search
WARNING: [youtube] LG9G4aA28rU: Unable to decode n-parameter: expect download to be blocked or throttled (slice must be applied on a list in: 'a.slice(0,0)'; please report this issue on https://github.com/ytdl-org/youtube-dl/issues , using the appropriate issue template. Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose option and include the complete output. Traceback (most recent call last):
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/extractor/youtube.py", line 1646, in _decrypt_nsig
    ret = extract_nsig(jsi, func_code)(n)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/extractor/youtube.py", line 1722, in extract_nsig
    ret = func([s])
          ^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1122, in resf
    ret, should_abort = self.interpret_statement(code.replace('\n', ' '), var_stack, allow_recursion - 1)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/extractor/youtube.py", line 1646, in _decrypt_nsig
    ret = extract_nsig(jsi, func_code)(n)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/extractor/youtube.py", line 1722, in extract_nsig
    ret = func([s])
          ^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1122, in resf
    ret, should_abort = self.interpret_statement(code.replace('\n', ' '), var_stack, allow_recursion - 1)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 495, in interpret_statement
    ret, should_return = self.interpret_statement(sub_stmt, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 711, in interpret_statement
    ret, should_abort = self.interpret_statement(sub_expr, local_vars, allow_recursion)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 750, in interpret_statement
    local_vars[m.group('out')] = self._operator(
                                 ^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 441, in _operator
    right_val = self.interpret_expression(right_expr, local_vars, allow_recursion)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1017, in interpret_expression
    ret, should_return = self.interpret_statement(expr, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1001, in interpret_statement
    return eval_method(variable, member), should_return
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 883, in eval_method
    argvals = [
              ^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 884, in <listcomp>
    self.interpret_expression(v, local_vars, allow_recursion)
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1017, in interpret_expression
    ret, should_return = self.interpret_statement(expr, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1001, in interpret_statement
    return eval_method(variable, member), should_return
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 928, in eval_method
    assertion(isinstance(obj, list), 'must be applied on a list')
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line [85](https://github.com/xihajun/speech2text/actions/runs/10256086421/job/28374451507#step:4:86)1, in assertion
    raise self.Exception('{memb} {msg}'.format(**locals()), expr=expr)
youtube_dl.jsinterp.JSInterpreter.Exception: slice must be applied on a list in: 'a.slice(0,0)'; please report this issue on https://github.com/ytdl-org/youtube-dl/issues , using the appropriate issue template. Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose option and include the complete output.

Description

WRITE DESCRIPTION HERE

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 15 (7 by maintainers)

Commits related to this issue

Most upvoted comments

Slicing patch for yt-dl, where we can’t say fn(*args, rest), and also handling JS .slice(n) -> Py [slice(n, None)]:

--- old/youtube-dl/youtube_dl/jsinterp.py
+++ new/youtube-dl/youtube_dl/jsinterp.py
@@ -925,9 +925,11 @@
                     obj.reverse()
                     return obj
                 elif member == 'slice':
-                    assertion(isinstance(obj, list), 'must be applied on a list')
-                    assertion(len(argvals) == 1, 'takes exactly one argument')
-                    return obj[argvals[0]:]
+                    assertion(isinstance(obj, (list, compat_str)), 'must be applied on a list or string')
+                    assertion(len(argvals) <= 2, 'takes between 0 and 2 arguments')
+                    if len(argvals) < 2:
+                        argvals += (None,)
+                    return obj[slice(*argvals)]
                 elif member == 'splice':
                     assertion(isinstance(obj, list), 'must be applied on a list')
                     assertion(argvals, 'takes one or more arguments')

There are two targets here, basically because yt-dl is by design not Chrome, nor even Firefox.

  1. We need to extract the code from the YT JS player that runs the n-parameter challenge. Essentially, like our code, the YT code has to get the n query parameter value out of the URL, pass it the challenge function, and set it back in the URL. From time to time YT changes the way that code is called, which makes it difficult to find reliably. In the commit above I reworked the search pattern, but it is always going to be easy for a YT intern or LLM action to produce another circumlocution that we don’t recognise. From the previous update, we also have a fallback that tries to find the challenge code directly rather than finding its caller. This would have worked with all (I think) the previous players and so should be more resilient.

  2. We need to execute the extracted challenge code without a browser (or external JS platform, Deno say). The program has a (HN-worthy) toy JS interpreter that relies heavily on the similarities between JS and Python. This has gradually acquired more, and more accurate, functionality, but generalisations beyond what any player has required before are more only likely to have been implemented if they’re easy.

The issue here is a good example. The interpreter has long supported the JS .slice() method (to make a subsequence), but (a) only for instances of Array (modelled by list) and (b) only with a single argument from. The problem player broke the assumptions on which the implementation relied. This is not so much sloppy coding as coding that was as good as it had to be when it wasn’t being paid for (obviously not as good as the code delivered by ClownStrike, say).

To implement the method (more) properly, we have to observe the following:

  • JS .slice() can have 0, 1 or 2 arguments
  • JS x.slice(n) means a subsequence of x from the n-th item whereas Python’s slice(n) selects to the n-th item: we need to say slice(n, None)
  • JS .slice() can apply to other sequence types, such as strings (modelled by compat_str, aka unicode in Py2, str in Py3).

In this case, more of the fix is about fixing the original assumptions than fixing the slicing.

I’m using the latest nightly build and I’m getting the 403 error on various videos. Strangely, it always seems to happen on the videos I actually WANT to download, while pretty much every other video I try just as a test, works fine.

I’ve compared the formats available, tried downloading specific formats, etc, and there doesn’t seem to be any pattern to it.

Right now, it seems completely incapable of downloading these two URLs;

https://www.youtube.com/watch?v=_qMlE4XHs4c https://www.youtube.com/watch?v=oH6Q_2f0aCI

Here’s the part I don’t get: The last time I updated the version of yt-dlp (backported to my old system by nicolaasjan, was on 7/31/24, and that version has no trouble downloading the same videos. In fact, I don’t think I’ve seen a single 403 error while using it.

I prefer youtube-dl though. Mostly because it runs faster.