yt-dlp: YouTube channel pages have infinite scrolling page count.

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

  • I’m reporting a broken site
  • I’ve verified that I’m running yt-dlp version 2022.11.11 (update instructions) or later (specify commit)
  • I’ve checked that all provided URLs are playable in a browser with the same IP and same login details
  • I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
  • I’ve searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
  • I’ve read the guidelines for opening an issue
  • I’ve read about sharing account credentials and I’m willing to share it if required

Region

United States

Provide a description that is worded well enough to be understood

YouTube has some automatically generated channels and playlists for musicians via a partner named TuneCore. The playlist page for these channels is infinite scrolling (as of today). This causes yt-dlp to grab infinite number of playlist pages. There is no option to tell yt-dlp to stop grabbing playlist pages upon repetition.

yt-dlp will need to add playlist repetition detection to abort continued grabbing of playlist pages.

Example:

https://www.youtube.com/channel/UCH0_ywht5OYCdRYWZUjBQwQ/playlists

This channel and all its playlists and videos are auto-generated by YouTube via partner TuneCore.

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

>yt-dlp -vU "https://www.youtube.com/channel/UCH0_ywht5OYCdRYWZUjBQwQ/playlists"
[debug] Command-line config: ['-vU', 'https://www.youtube.com/channel/UCH0_ywht5OYCdRYWZUjBQwQ/playlists']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-7-6.1.7601-SP1 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg n4.4-6-g7e9b9f24df (setts), ffprobe n4.4-6-g7e9b9f24df
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [youtube:tab] Extracting URL: https://www.youtube.com/channel/UCH0_ywht5OYCdRYWZUjBQwQ/playlists
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ/playlists: Downloading webpage
[debug] [youtube:tab] Selected tab: 'playlists' (playlists), Requested tab: 'playlists'
[download] Downloading playlist: Pogo - Topic - Playlists
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 1: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 2: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 3: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 4: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 5: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 6: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 7: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 8: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 9: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 10: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 11: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 12: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 13: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 14: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 15: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 16: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 17: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 18: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 19: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 20: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 21: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 22: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 23: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 24: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 25: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 26: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 27: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 28: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 29: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 30: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 31: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 32: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 33: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 34: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 35: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 36: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 37: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 38: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 39: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 40: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 41: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 42: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 43: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 44: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 45: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 46: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 47: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 48: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 49: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 50: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 51: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 52: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 53: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 54: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 55: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 56: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 57: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 58: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 59: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 60: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 61: Downloading API JSON
[youtube:tab] UCH0_ywht5OYCdRYWZUjBQwQ page 62: Downloading API JSON

ERROR: Interrupted by user

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (8 by maintainers)

Commits related to this issue

Most upvoted comments

Has this gone away now? Or should yt-dl be acquiring some variant of #6621?

Have not seen or heard of any reports recently. Must have been a short-lived bug on YouTube’s side.

The issue still exists. We may want to try your suggestion