yt-dlp: RaiPlay Extractor now returning 403: Forbidden

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Region

Italy

Provide a description that is worded well enough to be understood

The RAI extractor was working as of at least a couple of days ago, but appears to have broken and returns 403: Forbidden for every stream. I am still able to get a 720p format by inputting the M3U8 or JSON manually, so I think the URL linking or something has changed and it can’t parse it any more. I can’t find the 1080p https-5000 format in the M3U8 or JSON, but it shows up with doing --list-formats so it should still be there.

I’ve tested this on a bunch of videos that worked before, and they’re all broken now, so it seems this change is site-wide. You previously didn’t need to be logged in for the extractor to work, but I tested being logged out and logged in and both give the same error, so that’s not the problem.

@nixxo has previously fixed this extractor in the past, so hopefully they can look at it again.

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

yt-dlp.exe -vU https://www.raiplay.it/programmi/solecuoreamore
[debug] Command-line config: ['-vU', 'https://www.raiplay.it/programmi/solecuoreamore']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.12.30 from yt-dlp/yt-dlp [f10589e34] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg N-109444-geef763c705-20221222 (setts), ffprobe N-109444-geef763c705-20221222, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.19.1, brotli-1.1.0, certifi-2023.11.17, mutagen-1.47.0, requests-2.31.0, sqlite3-3.35.5, urllib3-2.1.0, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1798 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2023.12.30 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2023.12.30 from yt-dlp/yt-dlp)
[RaiPlayPlaylist] Extracting URL: https://www.raiplay.it/programmi/solecuoreamore
[RaiPlayPlaylist] solecuoreamore: Downloading program JSON
[RaiPlayPlaylist] ContentSet-da756fb7-0eb0-4c58-95fe-d0e49440d8a6: Downloading content set JSON
[download] Downloading playlist: Sole Cuore Amore
[RaiPlayPlaylist] Playlist Sole Cuore Amore: Downloading 1 items of 1
[download] Downloading item 1 of 1
[RaiPlay] Extracting URL: https://www.raiplay.it/video/2019/08/Sole-Cuore-Amore-20ea42ca-d856-44de-8eac-05ecddd67192.json
[RaiPlay] 20ea42ca-d856-44de-8eac-05ecddd67192: Downloading video JSON
[RaiPlay] 20ea42ca-d856-44de-8eac-05ecddd67192: Downloading XML metadata
[RaiPlay] 20ea42ca-d856-44de-8eac-05ecddd67192: Downloading m3u8 information
[debug] [RaiPlay] Creating https format for quality 1200
[debug] [RaiPlay] Creating https format for quality 1800
[debug] [RaiPlay] Creating https format for quality 2400
[debug] [RaiPlay] Creating https format for quality 3600
[debug] [RaiPlay] Creating https format for quality 5000
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 20ea42ca-d856-44de-8eac-05ecddd67192: Downloading 1 format(s): https-5000
[debug] Invoking http downloader on "https://mediapolisvod.rai.it/relinker/relinkerServlet.htm?cont=8Pz8oQnrQmQ5NsuDPtWg9weeqqEEqualeeqqEEqual&overrideUserAgentRule=mp4-5000"
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 4082, in urlopen
  File "yt_dlp\networking\common.py", line 114, in send
  File "yt_dlp\networking\_helper.py", line 204, in wrapper
  File "yt_dlp\networking\common.py", line 325, in send
  File "yt_dlp\networking\_requests.py", line 343, in _send
yt_dlp.networking.exceptions.HTTPError: HTTP Error 403: Forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 3418, in process_info
  File "yt_dlp\YoutubeDL.py", line 3139, in dl
  File "yt_dlp\downloader\common.py", line 455, in download
  File "yt_dlp\downloader\http.py", line 364, in real_download
  File "yt_dlp\downloader\http.py", line 120, in establish_connection
  File "yt_dlp\YoutubeDL.py", line 4114, in urlopen
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 403: Forbidden

[download] Finished downloading playlist: Sole Cuore Amore

About this issue

  • Original URL
  • State: closed
  • Created 5 months ago
  • Comments: 30 (6 by maintainers)

Commits related to this issue

Most upvoted comments

I’ll look into it in the weekend.

+4
nixxo on Feb 8, 2024

the fix resolves the 403 error but it remains impossible to download(merge bv+ba) or play(mpv) a complete stream of video and audio for this “new?” type of contents. my hackish workaround to get the url of the main hls manifest:

#!/bin/sh
_help() {
        echo "${0##*/}" 'https://www.raiplay.it/video/*.html'
        exit
}
test "$#" -ne 1 && _help
test "${1##*.}" = "html" || _help
if which jq >/dev/null;then
        _url="$(curl -sk "${1%.*}.json"|jq -r '.video |.content_url')"
else
        _url="$(curl -sk "${1%.*}.json"|sed 's/.*content_url": "//;s/", ".*//')"
fi
test -n "$_url" && curl -sk -m 2 -L --max-redirs 1 --user-agent "Mozilla" -w %{url_effective} -o /dev/null "$_url" && echo
+2
bann0 on Feb 13, 2024

Merger is failing because yt-dlp thinks the video-only format has audio, and is passing arguments to ffmpeg accordingly. This is #887

+1
bashonly on Feb 16, 2024

@claviola AFAICT you are experiencing separate issue that is being tracked as #887

try manually selecting the formats you want, and adding --audio-multistreams and --compat-options no-direct-merge to your command, e.g.:

yt-dlp --audio-multistreams --compat-options no-direct-merge -f "hls-2400+hls-aac-Italiano" "https://www.raiplay.it/video/2024/02/Un-posto-al-sole---Puntata-del-06022024-EP6377-c160ce25-14ff...e5-3d24d04a8dac.html"

Notes:

  • You didn’t post a verbose log so the URL in my example is the abbreviated one from your log, obviously replace it with the real URL
  • I don’t know that hls-aac-Italiano is actually the id of the audio format since I don’t have the full URL. Check with yt-dlp -F
  • The video format id will vary between videos, and the audio format id may as well. Again, check with yt-dlp -F
  • You can also try -f "bv*+ba" instead of specific format IDs. I just can’t test any of this because I am geo-blocked
+1
bashonly on Feb 16, 2024

@superclarkk

ffmpeg -i video.mp4 -i audio.mp4 -c:v copy -c:a aac output.mp4

to merge video+audio without reencode use: ffmpeg -i video.mp4 -i audio.mp4 -map 0 -map 1 -c copy output.mp4

+1
bann0 on Feb 15, 2024

@bann0 Unfortunately, your code doesn’t work for videos with separate visual and audio - it doesn’t download the audio. For example, try Mare Fuori E07 - E12

@superclarkk it seems that yt-dlp has, for now, some problems with this type of hls manifests, the script is to bypass yt-dlp, use a recent version of ffmpeg to download: best(usually 1080):ffmpeg -i “$url” -c copy file.mkv 576: ffmpeg -i “$url” -map p:0 -c copy file.mkv 720: ffmpeg -i “$url” -map p:1 -c copy file.mkv etc

+1
bann0 on Feb 15, 2024

@bann0 Grazie infinite, gentilissimo, per il tuo suggerimento (link) che mi ha permesso di scaricare bene (720). Alla fine compare il messaggio [tcp @ 062b58c0] ffurl_write returned 0xffffd8ba [tcp @ 08271d00] ffurl_write returned 0xffffd8ba ma il file scaricato è perfetto! grazie ancora.

+1
anunasica on Feb 15, 2024

@nixxo Grazie nixxo; scusa potresti suggerirmi come fare per scaricare (es. link postato sopra) ? rai3 giorno 6/02: programmi del mattino perfetti invece Un posto al sole forbidden. Altri downloader scaricano solo video. Sarebbe per me un dono immenso se mi scrivessi qui i comandi (o le operazioni da eseguire) per scaricare quel video (di solito è sempre 720, mai 1080). Grazie per la pazienza. Un saluto.

se non hai risolto ancora prova così: ffmpeg -hide_banner -v error -stats -i “https://mediapolisvod.rai.it/relinker/relinkerServlet.htm?cont=FYY9VJwfRu44xgXiNSspPAeeqqEEqualeeqqEEqual” -map 0:3 -map 0:4 -c copy file.mkv

+1
bann0 on Feb 11, 2024

@nixxo se si omette "&overrideUserAgentRule=mp4-" dagli ID hls- restituisce l’elenco completo anziche errore 403.

yt-dlp -F https://www.raiplay.it/video/2024/02/Sanremo-2024---74-Festival-della-Canzone-Italiana---Serata-Finale-del-10022024-c31c3075-13ab-4c3b-9133-91391eab7efb.html ID EXT RESOLUTION │ FILESIZE TBR PROTO │ VCODEC ACODEC MORE INFO ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── hls-aac-Audiodescrizione mp4 audio only │ m3u8 │ audio only unknown [Audiodescrizione] Audiodescrizione hls-aac-Italiano mp4 audio only │ m3u8 │ audio only unknown [Italiano] Italiano hls-1800 mp4 1024x576 │ ~ 3.85GiB 1758k m3u8 │ unknown unknown https-1800 mp4 1024x576 │ ≈ 3.85GiB 1758k https │ unknown unknown hls-2400 mp4 1280x720 │ ~ 5.13GiB 2344k m3u8 │ unknown unknown https-2400 mp4 1280x720 │ ≈ 5.13GiB 2344k https │ unknown unknown hls-3600 mp4 1600x900 │ ~ 7.70GiB 3516k m3u8 │ unknown unknown https-3600 mp4 1600x900 │ ≈ 7.70GiB 3516k https │ unknown unknown hls-5000 mp4 1920x1080 │ ~11.41GiB 5210k m3u8 │ unknown unknown https-5000 mp4 1920x1080 │ ≈10.95GiB 5000k https │ unknown unknown

yt-dlp -f https-1800 -g https://www.raiplay.it/video/2024/02/Sanremo-2024---74-Festival-della-Canzone-Italiana---Serata-Finale-del-10022024-c31c3075-13ab-4c3b-9133-91391eab7efb.html https://mediapolisvod.rai.it/relinker/relinkerServlet.htm?cont=9z9OdWSoNZuALk12PWClwweeqqEEqualeeqqEEqual&overrideUserAgentRule=mp4-1800

ffmpeg -i “https://mediapolisvod.rai.it/relinker/relinkerServlet.htm?cont=9z9OdWSoNZuALk12PWClwweeqqEEqualeeqqEEqual&overrideUserAgentRule=mp4-1800” [https @ 0x55c8536d2180] HTTP error 403 Forbidden https://mediapolisvod.rai.it/relinker/relinkerServlet.htm?cont=9z9OdWSoNZuALk12PWClwweeqqEEqualeeqqEEqual&overrideUserAgentRule=mp4-1800: Server returned 403 Forbidden (access denied)

ffmpeg -i “https://mediapolisvod.rai.it/relinker/relinkerServlet.htm?cont=9z9OdWSoNZuALk12PWClwweeqqEEqualeeqqEEqual” [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/chunklist_b1758000_vo_slita_t64MTgwMA==.m3u8’ for reading [https @ 0x56390d372380] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/chunklist_b2344000_vo_slita_t64MjQwMA==.m3u8’ for reading [https @ 0x56390d372380] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/chunklist_b3516000_vo_slita_t64MzYwMA==.m3u8’ for reading [https @ 0x56390d372380] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/chunklist_b5210000_vo_slita_t64NTAwMA==.m3u8’ for reading [https @ 0x56390d372380] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/chunklist_b192400_ao_slItaliano_t64SXRhbGlhbm9fYXVkaW8=.m3u8’ for reading [https @ 0x56390d372380] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/chunklist_b192400_ao_slAudiodescrizione_t64QXVkaW9kZXNjcml6aW9uZV9hdWRpbw==.m3u8’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b1758000_vo_slita_t64MTgwMA==_0.ts’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b1758000_vo_slita_t64MTgwMA==_1.ts’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b2344000_vo_slita_t64MjQwMA==_0.ts’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b2344000_vo_slita_t64MjQwMA==_1.ts’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b3516000_vo_slita_t64MzYwMA==_0.ts’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b3516000_vo_slita_t64MzYwMA==_1.ts’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b5210000_vo_slita_t64NTAwMA==_0.ts’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b5210000_vo_slita_t64NTAwMA==_1.ts’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b192400_ao_slItaliano_t64SXRhbGlhbm9fYXVkaW8=_0.aac’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b192400_ao_slItaliano_t64SXRhbGlhbm9fYXVkaW8=_1.aac’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b192400_ao_slAudiodescrizione_t64QXVkaW9kZXNjcml6aW9uZV9hdWRpbw==_0.aac’ for reading [hls,applehttp @ 0x56390d0d96c0] Opening ‘https://vod8-rai-it.akamaized.net/podcastcdn/raiuno/raiuno_multiaudio_nogeo/22348652_,1800,2400,3600,5000/media_b192400_ao_slAudiodescrizione_t64QXVkaW9kZXNjcml6aW9uZV9hdWRpbw==_1.aac’ for reading Input #0, hls,applehttp, from ‘https://mediapolisvod.rai.it/relinker/relinkerServlet.htm?cont=9z9OdWSoNZuALk12PWClwweeqqEEqualeeqqEEqual’: Duration: 05:06:08.27, start: 0.000000, bitrate: N/A Program 0 Metadata: variant_bitrate : 1758000 Stream #0:0: Data: timed_id3 (ID3 / 0x20334449) Metadata: variant_bitrate : 1758000 Stream #0:1: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1024x576 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Metadata: variant_bitrate : 1758000 Stream #0:8(Italiano): Audio: aac (LC), 44100 Hz, stereo, fltp, 242 kb/s (default) Metadata: id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x00\x00\x00\x00 comment : Italiano Stream #0:9(Audiodescrizione): Audio: aac (LC), 44100 Hz, stereo, fltp, 221 kb/s Metadata: id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x00\x00\x00\x00 comment : Audiodescrizione Program 1 Metadata: variant_bitrate : 2344000 Stream #0:2: Data: timed_id3 (ID3 / 0x20334449) Metadata: variant_bitrate : 2344000 Stream #0:3: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Metadata: variant_bitrate : 2344000 Stream #0:8(Italiano): Audio: aac (LC), 44100 Hz, stereo, fltp, 242 kb/s (default) Metadata: id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x00\x00\x00\x00 comment : Italiano Stream #0:9(Audiodescrizione): Audio: aac (LC), 44100 Hz, stereo, fltp, 221 kb/s Metadata: id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x00\x00\x00\x00 comment : Audiodescrizione Program 2 Metadata: variant_bitrate : 3516000 Stream #0:4: Data: timed_id3 (ID3 / 0x20334449) Metadata: variant_bitrate : 3516000 Stream #0:5: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 1600x900 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Metadata: variant_bitrate : 3516000 Stream #0:8(Italiano): Audio: aac (LC), 44100 Hz, stereo, fltp, 242 kb/s (default) Metadata: id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x00\x00\x00\x00 comment : Italiano Stream #0:9(Audiodescrizione): Audio: aac (LC), 44100 Hz, stereo, fltp, 221 kb/s Metadata: id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x00\x00\x00\x00 comment : Audiodescrizione Program 3 Metadata: variant_bitrate : 5210000 Stream #0:6: Data: timed_id3 (ID3 / 0x20334449) Metadata: variant_bitrate : 5210000 Stream #0:7: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Metadata: variant_bitrate : 5210000 Stream #0:8(Italiano): Audio: aac (LC), 44100 Hz, stereo, fltp, 242 kb/s (default) Metadata: id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x00\x00\x00\x00 comment : Italiano Stream #0:9(Audiodescrizione): Audio: aac (LC), 44100 Hz, stereo, fltp, 221 kb/s Metadata: id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x00\x00\x00\x00 comment : Audiodescrizione At least one output file must be specified

+1
bann0 on Feb 11, 2024

Da una analisi veloce sembra che rai abbia disabilitato il download del file mp4 sui nuovi contenuti, rimane attivo il download tramite manifest m3u8.

+1
nixxo on Feb 9, 2024
Read more comments on GitHub
← yt-dlp: [prosiebensat1] Unable to extract clip id
yt-dlp: [RTBF] "ERROR: An extractor error has occurred." using "--cookies-from-browser" or --cookies →