youtube-dl: RoosterTeeth.com support is broken following site update
Please follow the guide below
- You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
- Put an
xinto all the boxes [ ] relevant to your issue (like this:[x]) - Use the Preview tab to see what your issue will actually look like
Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.04.03. If it’s not, read this FAQ entry and update. Issues with outdated version will be rejected.
- I’ve verified and I assure that I’m running youtube-dl 2018.04.03
Before submitting an issue make sure you have:
- At least skimmed through the README, most notably the FAQ and BUGS sections
- Searched the bugtracker for similar issues including closed ones
- Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser
What is the purpose of your issue?
- Bug report (encountered problems with youtube-dl)
- Site support request (request for adding support for a new site)
- Feature request (request for a new functionality)
- Question
- Other
youtube-dl : [debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose',
'https://roosterteeth.com/episode/rooster-teeth-podcast-2018-486']
[debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[debug] youtube-dl version 2018.04.03
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.16299
[debug] exe versions: ffmpeg N-86482-gbc40674, ffprobe N-86482-gbc40674
[debug] Proxy map: {}
ERROR: Unable to extract m3u8 URL; please report this issue on
https://yt-dl.org/bug . Make sure you are using the latest version; type
youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag
and include its complete output.
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build
archive\youtube-dl\rg3\tmp_it5pa_f\build\youtube_dl\YoutubeDL.py", line 785,
in extract_info
File "C:\Users\dst\AppData\Roaming\Build
archive\youtube-dl\rg3\tmp_it5pa_f\build\youtube_dl\extractor\common.py", line
440, in extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp_it5pa_f\b
uild\youtube_dl\extractor\roosterteeth.py", line 115, in _real_extract
youtube_dl.utils.ExtractorError: Unable to extract m3u8 URL; please report
this issue on https://yt-dl.org/bug . Make sure you are using the latest
version; type youtube-dl -U to update. Be sure to call youtube-dl with the
--verbose flag and include its complete output.
Single video: https://rtv3-roosterteeth.akamaized.net/store/7281d24a2d5af5cba832b5335ced69ea-1e35796e/ts/index.m3u8
Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
Description of your issue, suggested solution and other information
Following the upgrade to RoosterTeeth.com, youtube-dl is no longer able to download videos, nor authenticate with the login page.
The m3u8 files have previously been hard to find, but now they are nearly impossible - it seems that the pages are dynamically generated on the client side as the <body> tag is completely empty, aside from some required <script> tags. Reverse engineering the API may be necessary, as simply snooping on network traffic doesn’t result in getting the m3u8 value.
See below content, the m3u8 file is available through an API call
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 3
- Comments: 53
Commits related to this issue
- [roosterteeth] fix free episode extraction(#16094) — committed to ytdl-org/youtube-dl by remitamine 5 years ago
- [roosterteeth] fix free episode extraction(#16094) — committed to Lamieur/youtube-dl by remitamine 5 years ago
- [roosterteeth] fix free episode extraction(#16094) — committed to meunierd/youtube-dl by remitamine 5 years ago
- [roosterteeth] fix login request(closes #16094)(closes #22689) — committed to meunierd/youtube-dl by remitamine 5 years ago
- Revert "[roosterteeth] fix free episode extraction(#16094)" This reverts commit fefe28e18201fe417702be1b26d5043ba34fbd78. — committed to Lamieur/youtube-dl by Lamieur 4 years ago
- Revert "[roosterteeth] fix free episode extraction(#16094)" This reverts commit fefe28e18201fe417702be1b26d5043ba34fbd78. — committed to Lamieur/youtube-dl by Lamieur 4 years ago
- [roosterteeth] fix free episode extraction(#16094) — committed to pareronia/youtube-dl by remitamine 5 years ago
- [roosterteeth] fix login request(closes #16094)(closes #22689) — committed to pareronia/youtube-dl by remitamine 5 years ago
I’ve just created a pull request with a working FIRST only video downloader. #22689 I’ve tested it with a few FIRST videos and it seems to work, please try it with any others and tell me if you have any problems.
If you put the episode you want to download in to this url:
eg:
becomes:
and then from that webpage copy the .m3u8 link, you can feed it in to youtube-dl, and it’ll download OK.
First exclusive shows are broken as c72dc20d099bbe1dc4ede83e8f94a7bc42d81532 only extracts free episodes. It needs to be modified to support authentication tokens like #16105 does. @remitamine I can give you a valid first login to test with if you are willing to write a patch.
I got it to work now. So for anyone having the same problem.
ffmpeg -protocol_whitelist file,http,https,tcp,tls,crypto -i "URL_TO_M3U8_FILE_FROM_2" -c copy video.mp4I suspect that the crypto parameters might expire, so you probably can’t wait too long between doing the API request and then downloading with ffmpeg.
I could not get it to work with youtube-dl as I kept getting a 403 error, but this is a solution until youtube-dl works.
Can confirm, using the api trick and slapping the m3u8 link on to youtube-dl does indeed download Rooster Teeth videos. (I don’t have a subscription so I can’t say anything for those exclusive ones.)
Any update on this problem? It’s been almost three months since last comment for both the pull request and issue thread.
For now, a quick and dirty fix could be to run a python equivalent of this, which gets exactly what we want:
curl https://svod-be.roosterteeth.com/api/v1/episodes/<<videoID>>/videos | sed 's/"/\n/g' | grep .m3u8Even if it’s not implemented into youtube-dl like this, it’s how I’m getting the URLs at the moment.I have no idea why that worked, but it did. I cloned your repo directly through github desktop. Not sure what the issue was.
@ipat8, it looks like you are not using RichardHancock’s version. I’ll admit I haven’t tried building an exe from source, but that log line indicates you are using the wrong version. Did you
git checkout patch-1? I made that mistake when verifying the pull request.@Matt14451 Did you specifically download the Version from my pull request? The main youtube-dl doesn’t have my fixes until the pull request is approved and merged.
Yep was using Python 3. Couldn’t get either way to work, but thanks for taking your time to try and help me. I ended up just deciding to spend half a day recording the episodes manually from my screen using OBS. Took a while but I have have the episodes now.
@sasjafor Thanks for this workaround!
I noticed this is only necessary when the link starts with
https://rtv3-roosterteeth.akamaized.net/store/.When it starts with
https://rtv2-roosterteeth.akamaized.net/uploads/videos/you don’t even need to copy the entire link, just up to the .m3u8 part.@Candunc Yeah, that client_id is not unique to your account, I see it in my request headers.
I found that authorization URL earlier and I’m able to log in through youtube-dl now. I’m trying to get the m3u8 URL now, to see if downloading FIRST videos will work. Thanks for the tip about the Authorization header; I haven’t done anything with OAuth in years, so it didn’t occur to me how to use the access_token.
Edit: Downloading FIRST-only videos now works. I’ll clean it up and push my changes, and hopefully someone will take a look at merging my pull request soon.
I can confirm this, the m3u8 urls don’t seem to require authentication to access. However we still face the issue of getting the initial m3u8 file.
After a bit of network analysis, I’ve found what I believe should allow a username/password combo to access FIRST content.
A POST request to ‘https://auth.roosterteeth.com/oauth/token’ with the below JSON file returns some parameters.
Of these parameters, “access_token” seems to be most valuable. By calling the api url with the ‘authorization’ header set to “Bearer <<access_token>>” returns the m3u8 file.
I have no idea what client_id is. It might be related to their various apps (Xbox, Apple TV, iOS, Browser?) I’m sharing it as I presume it isn’t unique to my username, as it was set prior to the login.
I opened a pull request to use the new API, so downloading public videos works now, but anything that requires you to be logged in fails. I’m trying to find out how to authenticate with the API, but I’m not making much headway.
Nevermind, the API provides it directly. Example:
https://svod-be.roosterteeth.com/api/v1/episodes/rooster-teeth-podcast-2018-486/videos
From input url “https://roosterteeth.com/episode/rooster-teeth-podcast-2018-486”, grab the last item and call the API endpoint. This doesn’t work for FIRST videos, as they likely want authentication tokens passed along.