youtube-dl: 【FranceTV】Download no longer possible

Here is quick early patch I wrote that seems to repair quite a few links. youtube-dl-patch.txt

Here is quick early patch I wrote that seems to repair quite a few links. youtube-dl-patch.txt

Thank you @sarnoud ! So is there an on-the-ground solution for end users with little or no programming expertise to apply this patch? If yes could you provide some guidance on the steps? Thank you.

Warm regards, NiesmialyGosc

In your Python directory (i.e. /usr/lib/python3/dist-packages for Linux) enter the following command:

patch youtube_dl/extractor/francetv.py < youtube-dl-patch.txt

Et voilà 😃

Workaround for lazy people like me:

• use this (non open source) extension to sniff manifest.mpd file
  • or manually find the manifest.mpd file using dev tools
• give it as command line argument to youtube-dl

Example:

youtube-dl "https://cloudingest.ftven.fr/ce699b31d0416/95cd8106-8e5e-4f58-bffe-78e73b7c8193_france-domtom_TA.ism/ZXhwPTE2NDI4MjIxMDJ+YWNsPSUyZmNlNjk5YjMxZDA0MTYlMmY5NWNkODEwNi04ZTVlLTRmNTgtYmZmZS03OGU3M2I3YzgxOTNfZnJhbmNlLWRvbXRvbV9UQS5pc20qfmhtYWM9MDU0OGZjMWQ1ODMyYzliYWYwMzFjMDhlZjg4ZGI4ZGRhMTUwYWFhMTU4NDA2NzExMzllZjM4MGRiMTUyOGUxNg==/manifest.mpd"

For newbies who don’t understand how to follow the comprehensive information repeated several times in this issue thread, if as me, you’re using compiled version in /usr/local/bin/youtube-dl, here’s a simple procedure to replace it:

(French version)

Pour les débutants qui ne comprennent pas comment appliquer les instructions complètes indiquées ci-dessus dans ce suivi de problème (surtout destiné à corriger le code pour que ce soit intégré à la prochaine version), si comme moi vous utilisez la version compilée dans /usr/local/bin/youtube-dl, voici une procédure simple pour la remplacer. Ces commandes sont à exécuter en utilisateur non privilégié, d’où le sudo devant cp. Il vous faut au préalable les programmes wget, make, zip, pas forcément installés.

mkdir /tmp/ytdl_francetv
cd /tmp/ytdl_francetv
wget https://yt-dl.org/downloads/2021.06.06/youtube-dl-2021.06.06.tar.gz https://github.com/ytdl-org/youtube-dl/files/7156590/youtube-dl-patch.txt
tar xzf youtube-dl-2021.06.06.tar.gz
cd youtube-dl
patch -p1 < ../youtube-dl-patch.txt 
make youtube-dl
sudo cp youtube-dl /usr/local/bin/youtube-dl
rm -r /tmp/ytdl_francetv

Hello @NiesmialyGosc, below I upload the complete francetv.py file patched with @sarnoud fix. Suppress .txt extension and replace extractor/francetv.py in your youtube-dl local installation by this modified version.

Hope that helps.

francetv.py.txt

I just downloaded the last update of Captvty (version 2.9.4.1) which fixes the extraction of France Télévisions’ contents and I tried to download Les Enquêtes de Murdoch replays (which are DRM protected contents) and Captvty was able to download the replays without DRMs! So I looked into the app requests to see how the extraction works and here is how the app is able to download non-protected M3U8s playlists:

(Note: the example I am using is based on this program.)

1. Captvty downloads the player JSON of France Télévisions (the one linked above).
2. The app takes the identifier after the server host of the URL of the DASH encrypted manifest (the app uses the iPad player response (so it gets an URL of an HLS encrypted manifest), but using the DASH manifest URL returned for Android/Web clients works too) (for my example: we are getting the following URL https://cloudreplay.ftven.fr/7ecf38ef53416/1036062453_france-domtom_DRM.ism/manifest.mpd, so the id is 7ecf38ef53416) and reverses it (61435fe83fce7). It tooks also the string of the plurimedia_id (1036066892).
3. The app construct a replayftv-vh URL based on the following structure:

https://replayftv-vh.akamaihd.net/i/streaming-adaptatif_media-secure_france-dom-tom/YEAR/SWEEK/JDAY/PLURIMEDIA_ID-MANIFEST_ID_REVERSED-,standard1,standard2,standard3,standard4,qaa,.mp4.csmil/master.m3u8

Where YEAR, WEEK (which represents the week number of the broadcast) and DAY (which represents the day number of the broadcast) should be extracted somewhere (maybe from the broadcasted_at string in the JSON player response): for my example, the values are respectively 2021, 37 and 1. The PLURIMEDIA_ID and MANIFEST_ID values are the corresponding values we are getting from the step 2…

1. The app requests then the token-akamai/hdfauthftv-a endpoint with the URL created (https://hdfauthftv-a.akamaihd.net/esi/TA?format=json&url=https://replayftv-vh.akamaihd.net/i/streaming-adaptatif_media-secure_france-dom-tom/2021/S37/J1/1036062453-61435fe83fce7-,standard1,standard2,standard3,standard4,qaa,.mp4.csmil/master.m3u8).

2. It takes the URL returned in the JSON response (from the url string) and fetches it. In this case, we are getting a 404 error (An error occurred while processing your request.).

But Captvty increases the DAY number to 1 (and of course the WEEK number, if DAY is equal to 7) and repeats steps 4. and 5. a few times:

• https://hdfauthftv-a.akamaihd.net/esi/TA?format=json&url=https://replayftv-vh.akamaihd.net/i/streaming-adaptatif_media-secure_france-dom-tom/2021/S37/J2/1036062453-61435fe83fce7-,standard1,standard2,standard3,standard4,qaa,.mp4.csmil/master.m3u8 -> URL with token returns 404.
• https://hdfauthftv-a.akamaihd.net/esi/TA?format=json&url=https://replayftv-vh.akamaihd.net/i/streaming-adaptatif_media-secure_france-dom-tom/2021/S37/J3/1036062453-61435fe83fce7-,standard1,standard2,standard3,standard4,qaa,.mp4.csmil/master.m3u8 -> URL with token returns 404.

And when requesting an URL with a token with 4 as the value of DAY (J4) (https://hdfauthftv-a.akamaihd.net/esi/TA?format=json&url=https://replayftv-vh.akamaihd.net/i/streaming-adaptatif_media-secure_france-dom-tom/2021/S37/J4/1036062453-61435fe83fce7-,standard1,standard2,standard3,standard4,qaa,.mp4.csmil/master.m3u8), the URL we are getting from token-akamai/hdfauthftv-a works (response code 200) and is not protected by DRMs!

I don’t know if this can be implemented in youtube-dl (and/or in its forks), but at least we have now a fix for DRM protected videos on the website/apps!

(Note: by searching for what is qaa, I found it means for original version, maybe this “param” is not always present on some streams.)

Merci @alexisfrjp ça fonctionne nickel désormais !

Et merci @sarnoud pour le patch !

Patch works. Thanks.

Need to download video and audio separately :

youtube-dl -i -f “bestvideo,bestaudio” https://www.france.tv/france-3/carnets-de-vol/2783631-emission-du-samedi-11-septembre-2021.html

Merging video and audio :

ffmpeg -i Carnets\ de\ vol-a386d839-8337-4975-bc98-767299aafb82.mp4 -i Carnets\ de\ vol-a386d839-8337-4975-bc98-767299aafb82.m4a -c copy -map 0:0 -map 1:0 Carnets\ de\ vol.mp4

Même problème.

@Bocio First, try with a non-mobile URL (https://www.france.tv/france-2/mediterranee-l-odyssee-pour-la-vie/3257512-les-aventuriers-de-la-mediterranee.html). Second, since the France.tv extractor was not patched in youtube-dl since it broke last September, use yt-dlp instead: it will work fine, and use the same parameters than yt-dl.

The logic described is definitely within the scope of what yt-dl can do. There is an extractor method _is_valid_url() that could be used to check the URLs returned by Akamai in step 5. Presumably increase the DAY by 1 rather than ~to~ 1?

Python has datetime.isocalendar() to get the week and day number once broadcasted_at has been parsed into a timestamp and then passed to datetime.fromtimestamp().

Is there any rule as to how many times the Akamai API has to be challenged before it reveals a valid URL?

In the yt-dlp discussion it was mentioned that qxx is a private-use ISO 639 language code. An audio description may be tagged qad. See also here.

For what it’s worth… I discovered that you can still have access to videos which are no longer “officially” available. What you need is the “videoid” code. Search engines (Google, Bing) proposing “cached” pages are of little help since their “cached” version of the web page is usually very recent (2-3 days at most), and they don’t keep archives (each new version replaces the former) ; archive.org does keep multiple versions of a same page but doesn’t browse that kind of website. In this case that works though : right now (2021/09/22 07:00 AM) Google’s cached page, which was saved on 2021/09/20 08:06 PM, does contain the code for the program mentioned above. If that doesn’t work, I found out that it is possible to dump a JSON file containing data relative to all programs that were available on the “replay” platform for a given channel over the past few weeks. I now do such dumps regularly with a script :

for /F "tokens=1,2,3 delims=/" %%d in ("%date%") do set YMJ=%%f%%e%%d
curl "http://api-front.yatta.francetv.fr/standard/publish/taxonomies/france-3/contents/?size=1000&page=0&sort=begin_date:desc" -o "G:\http _api-front.yatta.francetv.fr_standard_publish_taxonomies_france-3_contents_ size=1000&page=0&sort=begin_date desc %YMJ%.json"

(Change the name of the channel to get a similar dump for that channel, for instance “france-5” instead of “france-3”. I tried various values for “size”, it would seem like the limit is 9999, but 1000 should be more than enough for such purposes, it covers about 2 weeks of broadcasts.) Open the resulting file with MiTeC JSONView, search the “product_id” code of the page you’re looking for (the one appearing in the web page URL, in this case “736443” – the interface is a bit capricious, sometimes you have to press the “TAB” key to highlight the search box as it won’t work with the mouse pointer), then go to “content has media”, then “<element #0>”, then “media”, and there you should find the “videoid” code. With this code you can then download the specific JSON page for this program through this URL. (For another program, replace the “videoid” code and the “product_id” code, although in practice the second one doesn’t seem to matter.) And the video link is still there (which wasn’t the case before, I could still reconstruct it but it involved a few more tricks). I don’t know how long it works beyond the regular end of availibility ; in the past I’ve been able to download videos from M3U8 links up to about a month later, now with the complete overhaul of the platform I’ll have to test again. (For those interested, I asked about this here, got little insight but that was enough to get on the right track to finding what I was looking for on my own.)

Now, could someone reply to my questions about the most recommended methods to download and convert those streams ? Should I be concerned with those warnings about “packet discontinuity” and whatnot, when converting TS files downloaded by Captvty to MP4 with either ffmpeg or mp4box, or are there extra options that could solve the issue altogether ? Does anyone know what exact command Captvty uses to record those streams (with video and audio separated) to a single TS file, and why it proceeds like this ? (I mean : why does it record to a single TS file considering that it then goes through a rather complicated process to demux those streams before remuxing them as MP4 ?) I tried doing it manually with ffmpeg but I get a 403 error ; apparently youtube-dl follows a redirection whereas ffmpeg gets stuck.

G:\>youtube-dl -F "https://cloudreplay.ftven.fr/d0a47704c0416/1036025569_monde_TA.ism/manifest.mpd?hdnea=exp=1632240849~acl=%2fd0a47704c0416%2f1036025569_monde_TA.ism*~hmac=9b7f9e147c16026e7ae723bfd31b3b79a8322eb32dc25492f305f3addcb43b49"
[generic] manifest.mpd?hdnea=exp=1632240849~acl=/d0a47704c0416/1036025569_monde_TA: Requesting header
[redirect] Following redirect to https://cloudreplay.ftven.fr/d0a47704c0416/1036025569_monde_TA.ism/ZXhwPTE2MzIyNjE4ODF+YWNsPSUyZmQwYTQ3NzA0YzA0MTYlMmYxMDM2MDI1NTY5X21vbmRlX1RBLmlzbSp+aG1hYz1mY2MxODg5ZmNmZmZhYjhlYzRiNmEwNmVhMDA5NGJiYzMxYjc2YjI3ZjhhYmU1MDQwMGRjYTRhYjE0MjJkOTZi/manifest.mpd
[generic] manifest: Requesting header
WARNING: Falling back on generic information extractor.
[generic] manifest: Downloading webpage
[generic] manifest: Extracting information
WARNING: [generic] Unknown MIME type application/mp4 in DASH manifest
[info] Available formats for manifest:
format code      extension  resolution note
audio_fre=96000  m4a        audio only [fr] DASH audio   96k , m4a_dash container, mp4a.40.2 (48000Hz)
video=400000     mp4        384x216    DASH video  400k , mp4_dash container, avc1.42C01E, 25fps, video only
video=950000     mp4        640x360    DASH video  950k , mp4_dash container, avc1.4D401F, 25fps, video only
video=1400000    mp4        960x540    DASH video 1400k , mp4_dash container, avc1.4D401F, 25fps, video only
video=2000000    mp4        1280x720   DASH video 2000k , mp4_dash container, avc1.64001F, 25fps, video only (best)

G:\>ffmpeg -i "https://cloudreplay.ftven.fr/d0a47704c0416/1036025569_monde_TA.ism/ZXhwPTE2MzIyNjE4ODF+YWNsPSUyZmQwYTQ3NzA0YzA0MTYlMmYxMDM2MDI1NTY5X21vbmRlX1RBLmlzbSp+aG1hYz1mY2MxODg5ZmNmZmZhYjhlYzRiNmEwNmVhMDA5NGJiYzMxYjc2YjI3ZjhhYmU1MDQwMGRjYTRhYjE0MjJkOTZi/manifest.mpd" -c copy "20210914 CAD.ts"
ffmpeg version 4.2.3 Copyright (c) 2000-2020 the FFmpeg developers
...
[https @ 0000000000552500] HTTP error 403 Forbidden
https://cloudreplay.ftven.fr/d0a47704c0416/1036025569_monde_TA.ism/ZXhwPTE2MzIyNjE4ODF+YWNsPSUyZmQwYTQ3NzA0YzA0MTYlMmYxMDM2MDI1NTY5X21vbmRlX1RBLmlzbSp+aG1hYz1mY2MxODg5ZmNmZmZhYjhlYzRiNmEwNmVhMDA5NGJiYzMxYjc2YjI3ZjhhYmU1MDQwMGRjYTRhYjE0MjJkOTZi/manifest.mpd: Server returned 403 Forbidden (access denied)

When downloading with youtube-dl, a small annoyance is that the resulting files are identified by MediaInfo as having a “constant” audio bitrate, whereas it should be “variable”. When converting a TS file downloaded by Captvty to MP4, either through Captvty’s own convoluted 3 steps method, or with ffmpeg, or with mp4box, the audio bitrate is correctly identified as “variable” in the resulting file. I tried downlading to TS with youtube-dl (to see if downloading to TS and then converting to MP4 would fix the erroneous audio bitrate identification), but it insists on generating a MP4 file. (I asked about these minor issues on the VideoHelp forum, got very little insight, quite frustrating.)

Another question : now there’s only link available, which if I’m not mistaken corresponds to the “DASH” protocol (I’ll admit that I have a very cursory and empirical knowledge of all that stuff). How do you get the “HLS” versions (which are apparently made available with this new patch), and what is the difference ?

@NiesmialyGosc

To get subtitles for:

https://www.france.tv/france-3/plus-belle-la-vie/plus-belle-la-vie-saison-18/2743347-episode-du-mardi-14-septembre-2021.html

The manifest.mpd is:

https://cloudreplay.ftven.fr/6f1d1b301a316/1036062661_france-domtom_TA.ism/ZXhwPTE2MzE2NTQ3MDh+YWNsPSUyZjZmMWQxYjMwMWEzMTYlMmYxMDM2MDYyNjYxX2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTNlNjU1N2QxNTYwYWEwZjQ1NjcyYTIwMmRmYTgyYWRkYjg5ZDU5YThmYmMwODEwNmI5Nzc2NGViNDZjNTg1NzE=/manifest.mpd

In Firefox (any browser) replace manifest.mpd with textstream_fre=1000.webvtt and save file as vtt

https://cloudreplay.ftven.fr/6f1d1b301a316/1036062661_france-domtom_TA.ism/ZXhwPTE2MzE2NTQ3MDh+YWNsPSUyZjZmMWQxYjMwMWEzMTYlMmYxMDM2MDYyNjYxX2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTNlNjU1N2QxNTYwYWEwZjQ1NjcyYTIwMmRmYTgyYWRkYjg5ZDU5YThmYmMwODEwNmI5Nzc2NGViNDZjNTg1NzE=/textstream_fre=1000.webvtt

Useful info for a future back-port/update.

From the UK with 2023.03.04, the AV media all give 403 and only spritesheets are available.

Can confirm this on latest yt-dlp-nightly, from a non-French IP address; however, using a French VPN, the programme referenced by @amigainc appears well supported:

yt-dlp -vF "https://www.france.tv/france-3/les-poules/5066034-les-poules-au-supermarche.html" =>

debug] Command-line config: ['--ffmpeg-location', '..\\FFmpeg', '--downloader-args', 'ffmpeg:-v 8 -stats', '-vF', 'https://www.france.tv/france-3/les-poules/5066034-les-poules-au-supermarche.html']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version nightly@2023.07.02.193114 [8776349ef] (win_x86_exe)
[debug] Python 3.7.9 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 (OpenSSL 1.1.1g  21 Apr 2020)
[debug] exe versions: ffmpeg n6.1-dev-1397-N-111281-g285c7f6 (setts), ffprobe n6.1-dev-1397-N-111281-g285c7f6, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1856 extractors
[FranceTVSite] Extracting URL: https://www.france.tv/france-3/les-poules/5066034-les-poules-au-supermarche.html
[FranceTVSite] 5066034-les-poules-au-supermarche: Downloading webpage
[FranceTV] Extracting URL: francetv:bb637040-1729-11ee-9664-85e1f187131f
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading desktop video JSON
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading mobile video JSON
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading signed dash manifest URL
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading MPD manifest
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading signed hls manifest URL
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for bb637040-1729-11ee-9664-85e1f187131f:
ID                               EXT   RESOLUTION FPS |   FILESIZE   TBR PROTO | VCODEC        VBR ACODEC     ABR ASR MORE INFO
-----------------------------------------------------------------------------------------------------------------------------------------------
spritesheets                     mhtml unknown        |                  mhtml | images                               storyboard
hls-audio-aacl-96-Audio_Fran?ais mp4   audio only     |                  m3u8  | audio only        unknown            [fr] Audio Fran?ais
dash-audio_fre=96000             m4a   audio only     | ~144.00KiB   96k dash  | audio only        mp4a.40.2  96k 48k [fr] DASH audio, m4a_dash
dash-video=400000                mp4   384x216     25 | ~600.00KiB  400k dash  | avc1.42C01E  400k video only         DASH video, mp4_dash
hls-521                          mp4   384x216     25 | ~781.50KiB  521k m3u8  | avc1.42C01E  521k video only
dash-video=950000                mp4   640x360     25 | ~  1.39MiB  950k dash  | avc1.4D401F  950k video only         DASH video, mp4_dash
hls-1104                         mp4   640x360     25 | ~  1.62MiB 1104k m3u8  | avc1.4D401F 1104k video only
dash-video=1400000               mp4   960x540     25 | ~  2.05MiB 1400k dash  | avc1.4D401F 1400k video only         DASH video, mp4_dash
hls-1581                         mp4   960x540     25 | ~  2.32MiB 1581k m3u8  | avc1.4D401F 1581k video only
dash-video=2000000               mp4   1280x720    25 | ~  2.93MiB 2000k dash  | avc1.64001F 2000k video only         DASH video, mp4_dash
hls-2217                         mp4   1280x720    25 | ~  3.25MiB 2217k m3u8  | avc1.64001F 2217k video only
dash-video=5000000               mp4   1920x1080   25 | ~  7.32MiB 5000k dash  | avc1.640029 5000k video only         DASH video, mp4_dash
hls-5397                         mp4   1920x1080   25 | ~  7.91MiB 5397k m3u8  | avc1.640029 5397k video only

Be that as it may, this is still the yt-dl issue tracker and can confirm yt-dl itself remains in a broken state:

yt-dl -vF "https://www.france.tv/france-3/les-poules/5066034-les-poules-au-supermarche.html" => 

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ffmpeg-location', '.\\FFmpeg', '--external-downloader-args', '-v 8 -stats', '-vF', 'https://www.france.tv/france-3/les-poules/5066034-les-poules-au-supermarche.html']
[debug] Encodings: locale cp1253, fs mbcs, out cp737, pref cp1253
[debug] youtube-dl version 2023.07.02.19419 (single file build)
[debug] Python 3.4.4 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 - OpenSSL 1.0.2d 9 Jul 2015
[debug] exe versions: ffmpeg n6.1-dev-1397-N-111281-g285c7f6, ffprobe n6.1-dev-1397-N-111281-g285c7f6, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[FranceTVSite] 5066034-les-poules-au-supermarche: Downloading webpage
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading video JSON
ERROR: Unable to download JSON metadata: <urlopen error [Errno 11001] getaddrinfo failed> (caused by URLError(gaierror(11001, 'getaddrinfo failed'),))
  File "common.py", line 635, in _request_webpage
  File "YoutubeDL.py", line 2356, in urlopen
  File "C:\hostedtoolcache\windows\Python\3.4.4\x86\lib\urllib\request.py", line 464, in open
  File "C:\hostedtoolcache\windows\Python\3.4.4\x86\lib\urllib\request.py", line 482, in _open
  File "C:\hostedtoolcache\windows\Python\3.4.4\x86\lib\urllib\request.py", line 442, in _call_chain
  File "D:\a\youtube-dl\youtube-dl\youtube_dl\utils.py", line 2791, in https_open
  File "C:\hostedtoolcache\windows\Python\3.4.4\x86\lib\urllib\request.py", line 1185, in do_open

(test above performed under the same French VPN…).

It works well with yt-dlp 2023.06.22:

[FranceTVSite] Extracting URL: https://www.france.tv/france-3/les-poules/5066034-les-poules-au-supermarche.html
[FranceTVSite] 5066034-les-poules-au-supermarche: Downloading webpage
[FranceTV] Extracting URL: francetv:bb637040-1729-11ee-9664-85e1f187131f
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading desktop video JSON
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading mobile video JSON
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading signed dash manifest URL
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading MPD manifest
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading signed hls manifest URL
[FranceTV] bb637040-1729-11ee-9664-85e1f187131f: Downloading m3u8 information
[info] bb637040-1729-11ee-9664-85e1f187131f: Downloading 1 format(s): hls-5397+dash-audio_fre=96000
[download] Destination: /home/guy/Downloads/Les poules au supermarché.fhls-5397.mp4
(... ffmpeg logs ...)
[download] 100% of    7.58MiB in 00:00:00 at 9.23MiB/s
[dashsegments] Total fragments: 7
[download] Destination: /home/guy/Downloads/Les poules au supermarché.fdash-audio_fre=96000.m4a
[download] 100% of  145.57KiB in 00:00:00 at 287.33KiB/s
[Merger] Merging formats into "/home/guy/Downloads/Les poules au supermarché.mp4"
Deleting original file /home/guy/Downloads/Les poules au supermarché.fhls-5397.mp4 (pass -k to keep)
Deleting original file /home/guy/Downloads/Les poules au supermarché.fdash-audio_fre=96000.m4a (pass -k to keep)

yt-dlp finds manifests but they are 403 from the UK, regardless of --geo-bypass-country FR. I guess that they could be OK from FR.

@TiA4f8R Yes, it works!!! Thank you!!!

J2? It would mean that they have re-uploaded the video (J2 = Tuesday, so today), because they had uploaded it first yesterday (Monday, so J1). The date to find is the date when the video was uploaded in the server (not always the broadcast day, especially for the “avant-première” version, when it’s available before it has aired on the linear TV; once it’s done, they replace it by the brodcasted version).

And no .qaa in the request? Well, we’ll see for the next episode, this Sunday.

@Trit34 I think you still don’t understand what I am saying. You can’t use this URL, you need to get a temporary URL with the hdauth server for this URL (so an URL with an authorization in its params). That’s why you need to request the hdauth endpoint with the URL of the playlist (https://hdfauthftv-a.akamaihd.net/esi/TA?format=json&url=https://replayftv-vh.akamaihd.net/i/streaming-adaptatif_media-secure_france-dom-tom/2021/S39/J2/1040654889-6152c2dceecf4-,standard1,standard2,standard3,standard4,qaa,.mp4.csmil/master.m3u8).

Then look at the JSON response, copy the URL you get from the hdauth server and use it to download the stream. The URL should look like this:

https://replayftv-vh.akamaihd.net/i/streaming-adaptatif_media-secure_france-dom-tom/2021/S39/J2/1040654889-6152c2dceecf4-,standard1,standard2,standard3,standard4,qaa,.mp4.csmil/master.m3u8?hdnea=exp=1633209940~acl=%2fi%2fstreaming-adaptatif_media-secure_france-dom-tom%2f2021%2fS39%2fJ2%2f1040654889-6152c2dceecf4-,standard1,standard2,standard3,standard4,qaa,.mp4.csmil*~hmac=7c3e3afe0b9592ec429084d09af41149aab7649ac7bf31df3a7ef1bcc1b6db3e

Hope you can now understand what I wanted to say.

Merci pour le patch rapide. Je ne critique jamais ceux qui travaillent (sauf si je paye). Ce n’est pas la panacée, mais ça permet de continuer à récupérer des vidéos avant qu’elles ne disparaissent, modulo un “post-processing” manuel. S’il y a des choses qu’on peut faire sans être un spécialiste, n’hésitez pas à demander. Cordialement

@guy-teube you should try to understand what the commands you run do. You have a global version (sudo pip…) AND a user version (–user without sudo). Remove one of them. From your whereis, the global version is run. To delete one of them: sudo pip uninstall youtube-dl to remove the global verison and pip uninstall --user youtube-dl to remove the user’s version (without sudo). And then, search where is your francetv.py: cd / && sudo find . -type f -name francetv.py

Thank you @alexisfrjp, I’ve never used pip before…

pip uninstall --user youtube-dl returns: no such option: --user

And the search of francetv.py gives:

./home/myuser/.local/lib/python2.7/site-packages/youtube_dl/extractor/francetv.py
./usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/francetv.py

Which one to you advise to keep or remove?

@guy-teube Sorry, remove --user and use just sudo to remove the globally installed app. If your user is the only one using youtube-dl, remove the global package: sudo pip uninstall youtube-dl and keep the user’s one. Then apply the patch patch /home/myuser/.local/lib/python2.7/site-packages/youtube_dl/extractor/francetv.py < youtube-dl-patch.txt.

06/06/2021 alors que nous sommes le 15/09/2021…???

@D84web Open the console (F12) then Network tab. When the video is launched, make a search “manifest” (under this Network tab), and you will get this url.

@guy_teube : Chapeau ! It works. 😃

That’s can help me for some documentaries which will end tomorrow… I hope the Team will find a solution.

Whatever, thanks all for your help

PS : Sorry for English… I get rusty.

En français pour toi : je recommande de passer d’abord par youtube-dl -F [URL] pour récupérer les identifiants des bons flux vidéo et audio, car j’ai déjà constaté chez France TV que laisser récupérer automatiquement bestvideo et bestaudio pouvait récupérer le flux audio d’audiodescription pour les malvoyants ou la version originale d’une série étrangère. C’est pas forcément ce qu’on veut.

Par exemple : pour Slam d’aujourd’hui, tu fais ensuite youtube-dl -f dash-video=2000000+dash-audio_fre=96000 https://www.france.tv/france-3/slam/2743369-emission-du-mardi-14-septembre-2021.html, et si tu as ffmpeg installé, ce dernier sera automatiquement sollicité pour réunir les deux flux en un fichier MP4 unique. Tu utiliseras ensuite la méthode donnée plus haut pour les sous-titres.

@D84web Open the console (F12) then Network tab. When the video is launched, make a search “manifest” (under this Network tab), and you will get this url.

@D84web @LateLament

Thank you both team for the support! This is greatly appreciated & your tips really useful for future reference. THANK U. Can we have the subtitles downloaded as well?

I actually had to resort to “Lj video downloader” for the video download, and the only & key issue is actually with the subtitles for which I had to use the yt-dlp downloader:

https://www.france.tv/france-3/plus-belle-la-vie/plus-belle-la-vie-saison-18/2743347-episode-du-mardi-14-septembre-2021.html

yt-dlp --write-subs --skip-download --external-downloader aria2c --allow-u "https://cloudreplay.ftven.fr/6f1d1b301a316/1036062661_france-domtom_TA.ism/ZXhwPTE2MzE2MzcyMDJ+YWNsPSUyZjZmMWQxYjMwMWEzMTYlMmYxMDM2MDYyNjYxX2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTg5OGFhNGFhODVkYWZjMmQ3N2MyOTg5M2M2MmMyOTYyOTRlNGE5MzM2NzE2ODhlNjMxOTA5MDc3MTgyMWUyZTA=/manifest.mpd" --output 4367.webvtt

However, the generated subtitle file is in a mess which looks like this:

            trun  
   
   p  '  € @  mdat<?xml version="1.0" encoding="utf-8"?><tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttm="http://www.w3.org/ns/ttml#metadata" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" xmlns:tts="http://www.w3.org/ns/ttml#styling" xml:lang="fr"><head><metadata><ttm:title></ttm:title><ttm:desc></ttm:desc><ttm:copyright></ttm:copyright></metadata><styling><style xml:id="captionStyle" tts:backgroundColor="transparent" tts:displayAlign="center" tts:extent="100% 20%" tts:fontFamily="proportionalSansSerif" tts:fontSize="30px" tts:origin="0% 75%" tts:textAlign="center" tts:textOutline="black 1px 0px" /></styling><layout><region style="captionStyle" xml:id="region2" /></layout></head><body><div region="region2"><p begin="00:00:00.000" end="00:00:02.233" region="region2" xml:id="caption1" ttm:role="caption">
        <span tts:textAlign="center" tts:color="white">-L&apos;animal n&apos;est pas commode.</span>
      </p><p begin="00:00:02.500" end="00:00:04.166" region="region2" xml:id="caption2" ttm:role="caption">
        <span tts:textAlign="center" tts:color="yellow">-Je vais le dresser.</span>

As a result, I have to remove those tags & recode the encoding which would take a lot of work… Is there a better way to generate the subtitles than using the yt-dlp tool??? Or could any of you provide some tips on how to mass download, merge & reformat the webvtt files from France.TV, or a quick fix to remove the tags above? (If possible & if you could elaborate each step in detail, that’ll be great)

Thanks & Regards NiesmialyGosc

Hello @NiesmialyGosc, below I upload the complete francetv.py file patched with @sarnoud fix. Suppress .txt extension and replace extractor/francetv.py in your youtube-dl local installation by this modified version.

Hope that helps.

francetv.py.txt

Sorry I don’t understand. I’m not an professional IT guy and I can’t understand some technical terms such as “patched with” & “youtube-dl local installation”.

Is youtube-dl intended for users with IT development knowledge (e.g. python) only?

Could anyone please help to translate the above into an on-the-ground language without me as a normal user that wishes to watch online videos having to learn python grammar & vocabulary from scratch?

Thank you for your patience & time, NiesmialyGosc

I’m an Ubuntu 18.04 user. I have no youtube_dl directory in my /usr/lib/python3 directory (even in python3.6/3.7/3.8). a whereis youtube-dl says : /usr/local/bin/youtube-dl I have searched a file name francetv.py in my whole /usr directory, nothing found. Do you know how (and where) run your patch script?

Hello @guy-teube , your local youtube-dl installation should be in /usr/local/lib/python3.9/dist-packages/youtube_dl-... (modify python3.9 path if you have another version).

The file to patch is extractor/francetv.py

I confirm this issue with FranceTV site. Specific code for extractor is in youtube_dl/extractor/francetv.py

To download JSON metadata, extractor requests https://sivideo.webservices.francetelevisions.fr/tools/getInfosOeuvre/v2/ with video ID (line 94). But it seems that this API is deprecated 😦

With datas used for tests in FranceTV extractor (line 51) : request for https://sivideo.webservices.francetelevisions.fr/tools/getInfosOeuvre/v2/?idDiffusion=162311093 => JSON message {"status":"NOK","code":404,"message":"This api is deprecated, contact France.tv video-factory"}

After a few debug, I didn’t find the new URL/service in my HTTP logs to get metadatas via API.