yt-dlp: Question: direct media links give back 403 in my program, yt-dlp still works, despite sometimes reporting "extracting nsig failed"

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Please make sure the question is worded well enough to be understood

I’m actually looking for some help, to identify a problem within my software, not yt-dlp, that works fine. This video: https://www.youtube.com/watch?v=hzA5AhH1mbw can be perfectly downloaded via yt-dlp. However if i try to get the direct links with my software, the resulting links will give 403 from every software (VLC, MPV, wget, etc.)

First i suspected the nsig, and since sometimes yt-dlp reported that it could not extract the nsig (and yet, it was able to download the video), i thought the decipher simply changed again. However, here is a package of the extracted decipher from that video: http://oscomp.hu/depot/yt.hzA5AhH1mbw.2024_8_8_22_27_25.zip Inside, the extracted_decipher.js is running without errors. Of course the result still could be bad, but i had copied the entire base.js into Ungoogled Chromium 116 and added Oma('tWnZ0PahN_DEUy59W') at the end and the result was identical to the command line nodejs’s result. (90IKuYq5hTIutQ) Since yt-dlp told me when it supposedly failed to extract the nsig (i repeat, it was still able to download the video despite that warning), that i should install phantomjs to workaround that problem. I tried it with my JS file and it gave identical results.

At this point i’ve added two extra lines at the end of extracted_decipher.js to compare the extracted nsig-s from yt-dlp which are visible in the log below:

[debug] Loading youtube-nsig.28fd7348 from cache
[debug] [youtube] Decrypted nsig BKUh2hmhwyy9hlmpe => CP0MY8VwOo1oRA
[debug] Loading youtube-nsig.28fd7348 from cache
[debug] [youtube] Decrypted nsig dfsrXGLw2QZyIuraU => EyaFlxAUbY18Cg

And the extracted decipher from my program gave the very same results. I think the nsig deciphering is okay at my end, yet the links my program extracted from YT are giving 403 more and more times. Sometimes after one or two reload it starts to work, but now more and more videos are giving 403 permanently.

Can anyone tell me what is wrong? Maybe it is not the nsig, but the signatureCipher -> sig is done erroneously? Two years ago @pukkandan helped me fix that (#5011), but perhaps YT changed something and it broke again? Or now YT uses cookies to protect the videos and the external videoplayer don’t have them? I tried to pass the agent and the referer to MPV and it still got a 403, so i think it is not those.

Can anyone help me in this? Thanks in advance.

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://www.youtube.com/watch?v=hzA5AhH1mbw']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.08.06 from yt-dlp/yt-dlp [4d9231208] (zip)
[debug] Python 3.9.2 (CPython x86_64 64bit) - Linux-6.1.0-0.deb11.13-amd64-x86_64-with-glibc2.31 (OpenSSL 1.1.1w  11 Sep 2023, glibc 2.31)
[debug] exe versions: ffmpeg 5.1.2 (setts), ffprobe 5.1.2, rtmpdump 2.4
[debug] Optional libraries: certifi-2020.06.20, sqlite3-3.34.1
[debug] Proxy map: {}
[debug] Request Handlers: urllib
[debug] Loaded 1830 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.08.06 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.08.06 from yt-dlp/yt-dlp)
[youtube] Extracting URL: https://www.youtube.com/watch?v=hzA5AhH1mbw
[youtube] hzA5AhH1mbw: Downloading webpage
[youtube] hzA5AhH1mbw: Downloading ios player API JSON
[youtube] hzA5AhH1mbw: Downloading web creator player API JSON
[debug] Loading youtube-nsig.28fd7348 from cache
[debug] [youtube] Decrypted nsig BKUh2hmhwyy9hlmpe => CP0MY8VwOo1oRA
[debug] Loading youtube-nsig.28fd7348 from cache
[debug] [youtube] Decrypted nsig dfsrXGLw2QZyIuraU => EyaFlxAUbY18Cg
[youtube] hzA5AhH1mbw: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] hzA5AhH1mbw: Downloading 1 format(s): 248+251
[debug] Invoking http downloader on "https://rr2---sn-qpbpu8-c0qs.googlevideo.com/videoplayback?expire=1723174215&ei=5ji1ZqmoPIzb6dsPqai_-Q4&ip=84.236.31.132&id=o-AC2jid7oO2sZQkR1wodYgXj5jVB-QCg9-z7PiJ-u7zSs&itag=248&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&mh=xH&mm=31%2C29&mn=sn-qpbpu8-c0qs%2Csn-c0q7lnsl&ms=au%2Crdu&mv=m&mvi=2&pl=24&initcwndbps=1867500&vprv=1&svpuc=1&mime=video%2Fwebm&rqh=1&gir=yes&clen=152750169&dur=1303.835&lmt=1663512024087339&mt=1723152314&fvip=4&keepalive=yes&c=IOS&txp=4437434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cvprv%2Csvpuc%2Cmime%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRQIhAPLDPjqhHiSetrnyJC2CmaXsIuBP1KNFCRH2i-bnKzhsAiA5nXE8BX77ibjQoG8uta7WI8hEbd-Vy-NGtdu3xKFWpg%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AGtxev0wRAIgaVeaJvOBS8jtEoOR4b1nRhol2dvfK-ZvKo1xz170ZJYCIC2bMeaNPKcHT8lb_LJXpyIQsauSDebhqn1kBLmqJSxU"
[download] Destination: Amiga 3000 - The Best Amiga Ever? [hzA5AhH1mbw].f248.webm
[download] 100% of  145.67MiB in 00:00:15 at 9.24MiB/s
[debug] Invoking http downloader on "https://rr2---sn-qpbpu8-c0qs.googlevideo.com/videoplayback?expire=1723174215&ei=5zi1ZufNCcC-i9oP9My38A4&ip=84.236.31.132&id=o-AEhRaNEKw-gvX7k1LZ5YToLfOfaR0QrLzzJSKki-wQrj&itag=251&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&mh=xH&mm=31%2C29&mn=sn-qpbpu8-c0qs%2Csn-c0q7lnsl&ms=au%2Crdu&mv=m&mvi=2&pl=24&initcwndbps=1892500&bui=AXc671IHIirCFvCUvZulBj8v2Fz85BTl3H62jsWol2pQnHXQgAAzx5bz61clTTaUHtjNqJbIW9MVaKdW&spc=Mv1m9qBkcXzLMJg7YinI8N26LfVgjAHy1AJ3YvfqL_zCRyDrwXUu&vprv=1&svpuc=1&mime=audio%2Fwebm&ns=xQbFdFC_o9hd82lVO3dJUaEQ&rqh=1&gir=yes&clen=19821711&dur=1303.861&lmt=1663510689199780&mt=1723152553&fvip=4&keepalive=yes&c=WEB_CREATOR&sefc=1&txp=4432434&n=EyaFlxAUbY18Cg&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRQIgHKj8Lq9ShzDmcvgbRmq4hzvBLTWCGi2y_QpdYx4fBfUCIQDG6jmKwoDBs3vL9DFjIxug_aJdX2zTwOknxi6hshzwBg%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AGtxev0wRQIgfCslSfNmY4K0hqLxiybtpcu6E28eImVPdKG7USSEe24CIQDNhk7EIWiuV2Lqjbh5nOxKEU5eM_UHrE3yoWFMf4Kf6A%3D%3D"
[download] Destination: Amiga 3000 - The Best Amiga Ever? [hzA5AhH1mbw].f251.webm
[download] 100% of   18.90MiB in 00:00:02 at 9.32MiB/s
[Merger] Merging formats into "Amiga 3000 - The Best Amiga Ever? [hzA5AhH1mbw].webm"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:Amiga 3000 - The Best Amiga Ever? [hzA5AhH1mbw].f248.webm' -i 'file:Amiga 3000 - The Best Amiga Ever? [hzA5AhH1mbw].f251.webm' -c copy -map 0:v:0 -map 1:a:0 -movflags +faststart 'file:Amiga 3000 - The Best Amiga Ever? [hzA5AhH1mbw].temp.webm'
Deleting original file Amiga 3000 - The Best Amiga Ever? [hzA5AhH1mbw].f251.webm (pass -k to keep)
Deleting original file Amiga 3000 - The Best Amiga Ever? [hzA5AhH1mbw].f248.webm (pass -k to keep)

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 28 (7 by maintainers)

Most upvoted comments

POToken is generated by BotGuard, which attests the browser to ensure requests are coming from a real browser. So you need to generate it using a browser. It is tied to the visitor data/ID.

Some work is being done in https://github.com/yt-dlp/yt-dlp/pull/10648 so you can pass the po token from your browser to yt-dlp, which surprisingly works.

I see, so a full browser engine would be needed to be integrated into the client to be able to extract this poToken. (Or extract the poToken from a browser each time when it starts to fail again…)

To extract from IOS you need to make a /player request with the IOS client context, which you’ve linked. Perhaps run yt-dlp with --print-traffic or mitmproxy to see the final request being made.

Thanks! yt-dlp is really useful! So, i need to POST the following data

{"context": {"client": {"clientName": "IOS", "clientVersion": "19.29.1", "deviceMake": "Apple", "deviceModel": "iPhone16,2", "userAgent": "com.google.ios.youtube/19.29.1 (iPhone16,2; U; CPU iOS 17_5_1 like Mac OS X;)", "osName": "iPhone", "osVersion": "17.5.1.21F90", "hl": "en", "timeZone": "UTC", "utcOffsetMinutes": 0}}, "videoId": "hzA5AhH1mbw", "playbackContext": {"contentPlaybackContext": {"html5Preference": "HTML5_PREF_WANTS"}}, "contentCheckOk": true, "racyCheckOk": true}

to /youtubei/v1/player?prettyPrint=false and then build an EDL from the fragment links and pass that to MPV or VLC. Thank you again!