yt-dlp: [teachable] ERROR: Unable to find Video URL https://edu.ecomsuccess.pk/courses/enrolled/903296
Checklist
- I’m reporting a broken site
- I’ve verified that I’m running yt-dlp version 2022.04.08 (update instructions) or later (specify commit)
- I’ve checked that all provided URLs are alive and playable in a browser
- I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
- I’ve searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
- I’ve read the guidelines for opening an issue
- I’ve read about sharing account credentials and I’m willing to share it if required
Region
No response
Description
Just a few days ago I was able to download videos from the ‘member-only’ site, but not now. For every course link, yt-dlp issues the error ‘unable to find video URL’. Please help.
Verbose log
[debug] Command-line config: ['-f', '(bestvideo[ext=mp4]+bestaudio/best[ext=mp4]/best)[protocol^=http]', '--cookies', '/root/ecomsuccess.pk_cookies.txt', '--verbose', '--output', '/home/Tutorial/EComSuccess/%(playlist_title)s.%(ext)s', 'https://edu.ecomsuccess.pk/courses/enrolled/903296']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8 (No ANSI), err utf-8 (No ANSI), pref UTF-8
[debug] yt-dlp version 2022.04.08 [7884ade65] (zip)
[debug] Python version 3.8.10 (CPython 64bit) - Linux-5.4.0-107-generic-x86_64-with-glibc2.29
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4
[debug] Optional libraries: certifi, secretstorage, sqlite
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://edu.ecomsuccess.pk/courses/enrolled/903296
[generic] 903296: Requesting header
WARNING: [generic] Falling back on generic information extractor.
[generic] 903296: Downloading webpage
[generic] 903296: Extracting information
[debug] Looking for video embeds
[debug] [TeachableCourse] Extracting URL: teachable:https://edu.ecomsuccess.pk/courses/enrolled/903296
[TeachableCourse] 903296: Downloading webpage
[download] Downloading playlist: Affiliate Marketing
[TeachableCourse] playlist Affiliate Marketing: Collected 9 videos; downloading 9 of them
[download] Downloading video 1 of 9
[debug] [Teachable] Extracting URL: teachable:https://edu.ecomsuccess.pk/courses/903296/lectures/16630922
[Teachable] 16630922: Downloading webpage
ERROR: [Teachable] 16630922: Unable to find video URL; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 641, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/teachable.py", line 175, in _real_extract
raise ExtractorError('Unable to find video URL')
[download] Downloading video 2 of 9
[debug] [Teachable] Extracting URL: teachable:https://edu.ecomsuccess.pk/courses/903296/lectures/19396131
[Teachable] 19396131: Downloading webpage
ERROR: [Teachable] 19396131: Unable to find video URL; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 641, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/teachable.py", line 175, in _real_extract
raise ExtractorError('Unable to find video URL')
[download] Downloading video 3 of 9
[debug] [Teachable] Extracting URL: teachable:https://edu.ecomsuccess.pk/courses/903296/lectures/19397049
[Teachable] 19397049: Downloading webpage
ERROR: [Teachable] 19397049: Unable to find video URL; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 641, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/teachable.py", line 175, in _real_extract
raise ExtractorError('Unable to find video URL')
[download] Downloading video 4 of 9
[debug] [Teachable] Extracting URL: teachable:https://edu.ecomsuccess.pk/courses/903296/lectures/19397114
[Teachable] 19397114: Downloading webpage
ERROR: [Teachable] 19397114: Unable to find video URL; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 641, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/teachable.py", line 175, in _real_extract
raise ExtractorError('Unable to find video URL')
[download] Downloading video 5 of 9
[debug] [Teachable] Extracting URL: teachable:https://edu.ecomsuccess.pk/courses/903296/lectures/19397129
[Teachable] 19397129: Downloading webpage
ERROR: [Teachable] 19397129: Unable to find video URL; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 641, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/teachable.py", line 175, in _real_extract
raise ExtractorError('Unable to find video URL')
[download] Downloading video 6 of 9
[debug] [Teachable] Extracting URL: teachable:https://edu.ecomsuccess.pk/courses/903296/lectures/22181679
[Teachable] 22181679: Downloading webpage
ERROR: [Teachable] 22181679: Unable to find video URL; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 641, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/teachable.py", line 175, in _real_extract
raise ExtractorError('Unable to find video URL')
[download] Downloading video 7 of 9
[debug] [Teachable] Extracting URL: teachable:https://edu.ecomsuccess.pk/courses/903296/lectures/22181680
[Teachable] 22181680: Downloading webpage
ERROR: [Teachable] 22181680: Unable to find video URL; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 641, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/teachable.py", line 175, in _real_extract
raise ExtractorError('Unable to find video URL')
[download] Downloading video 8 of 9
[debug] [Teachable] Extracting URL: teachable:https://edu.ecomsuccess.pk/courses/903296/lectures/22868643
[Teachable] 22868643: Downloading webpage
ERROR: [Teachable] 22868643: Unable to find video URL; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 641, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/teachable.py", line 175, in _real_extract
raise ExtractorError('Unable to find video URL')
[download] Downloading video 9 of 9
[debug] [Teachable] Extracting URL: teachable:https://edu.ecomsuccess.pk/courses/903296/lectures/22868644
[Teachable] 22868644: Downloading webpage
ERROR: [Teachable] 22868644: Unable to find video URL; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 641, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/yt-dlp/yt_dlp/extractor/teachable.py", line 175, in _real_extract
raise ExtractorError('Unable to find video URL')
[download] Finished downloading playlist: Affiliate Marketing
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 88 (19 by maintainers)
Thanks for the work @Green0Photon. Took myself a bit of time to get everything ready, therefore I thought other users might like a little summary of the steps above:
./yt-dlp.sh --cookies cookies.txt --verbose -N 32 https://course.com/courses/enrolled/1234567
optional: add the following parameter to get your video files organized into directories according to the course chapters.-o "./%(chapter_number)s-%(chapter)s/%(autonumber)03d-%(title)s.%(ext)s"
Edit: Though it “only” seemed to help on the “unable to exctract course title” error for me. In the browser I saw one course with a “newer” design, which still gave me the “Unsupported URL” error.
Edit 2: For the “newer” design I still could download the lectures one by one. The ID of the lectures (as shown in the URL) were just counting up in my case. Which allowed me running a loop over the numbers do download all. So you have to check the first ID and the last ID and run a for loop to download. Same commands as used above (will organize them as above):
I’ve been using yt-dlp to consume some Teachable content (cantrill for AWS certs is great) on my phone at high speed (with skip silence feature, I love lectures now, I rec the AntennaPod app), and I’ve come back to discover it not working. So I’m giving my shot at seeing if I can fix it.
Right now, I’ve been able to do what @svenop5 says. That is, downloading something manually by grabbing stuff via Web Dev Tools after whatever of their necessary JS runs in the browser.
Firstly, some of his content is public on his site, that you can view (as a trial I guess) without buying the course. I’ll use it as an example and for tests. This is a good example link.
Teachable Site
Now, when you visit that link in e.g. curl, it won’t provide the hotmart iframe. Instead, you get this:
Yes, the html download via curl did have that shoddy indentation.
No clue where that
attachment-id
comes from or if it ever changes – but you can seecourse-id
andlecture-id
from the url. Theuser-id
is-1
when logged out, and it’s some other id when I’m logged in.It looks like those two outer divs are remnants from Teachable embedding Wistia, but they just replaced the player itself ergo the inner hotmart div. Now, when I visit the page in an actual browser, it injects an iframe into that
hotmart_video_player
div. Who knows how that iframe gets generated from the JS, but getting its src attribute is one thing yt-dlp will probably need to do now.IFrame on the Teachable Site
That inner iframe looks like this:
That link should be expired when you use it, so you’ll have to generate a new one yourself by visiting the Teachable page. Things to note:
/embed/
.signature
query parameter, looks to be base64 with the -_ url safe variant.token
query parameter, looks to be a uuid.user
number from the containing div got passed in, interestingly enough. It’s strange for this to be the case, I expected theuser-id
to be unique to either Teachable or the specific Teachable site (I can’t remember which I’ve logged in with), and don’t know why it would get passed to their backend.autoplay
query parameter.autoplay
anduser
parameters and it’ll still work just fine. A wrong length or changedtoken
gives a 400, and removing it entirely is a 401. A changed or removedsignature
is a 401. When the page expires, it’ll give a 410.One interesting side note: there is only one flavor of link, but I’ve encountered two different video content sources, which I’ll get to. One has an HLS playlist link of qualities you can plug directly into streamlink, and another has the seperate key link you need to need to give to streamlink separately. The latter actually causes Firefox to think it’s a tracker and so it actually gets blocked, but both work when you visit the iframe src directly. I’ve been able to download both via streamlink. The former was what I initially encountered, but now I’m only getting the latter. I’m not sure if we’ll want to support both or just the latter (i.e. the separate key link one).
Hotmart Player Site
So let’s visit that src link now. Again, you’ll need to grab your own, so I’m not going to put it in here again.
Now open the Network section of the Web Dev tools – preferably open it before you go to the src link, so you can see all requests without needing to refresh the page. There’s some html, js, css, a favicon, and a thumbnail jpg (maybe it’ll be fun to keep this later, too, like the youtube extractor). Also for me, several blocked POST calls for data collection, though unfortunately one goes through. Besides that, the one we care about.
The easy type has the following interesting requests (I was only able to get this for a different paid video, titled “Public Introduction (Release v1)”, which sounds like it’s mistakenly private):
The same video when loaded under the hard type looks like this:
All of these when queried are expired, so it should be safe to put them here. Though that token is the same between the two – hopefully that doesn’t leak anything from my account. I’ll try relogging later. Or perhaps it’s actually an app id. I’ve also saved all their contents, but I won’t upload them now just in case – though the response from HLS Segment Key URI is the same between the two, though not between videos.
Encrypted Keys
Now, I was able to download these using streamlink, as I said before. It’s a different sha1sum from the old video that used to be there (which I believe just served whatever video was uploaded, since those even had year old timestamps from when they were made), but now are just HLS streams. As @svenop5 said, the hard version needs
--hls-segment-key-uri
. This overwrites a line in the 1080p stream, though a similar line exists in every format. It looks like this:The
chave
means it’s custom. yt-dlp does support AES-128 here, but it needs to be a normal url, not that encrypted one. I downloaded that 1080p playlist, put it into a text editor, replaced that url with the decoded one (HLS Segment Key URI above), and yt-dlp and streamlink were both able to then immediately download it. Easy mode’s looked like this:Which is the url of the key which yt-dlp and streamlink are able to get and decode the videos. Again, that result of that key uri is different for different sizes of video, but is the same for easy/hard mode for the same size of the video.
Also, I noticed that the files downloaded are the same between streamlink and yt-dlp, regardless of easy/hard, for the same video size, as long as you have
--fixup never
. Though the fixup is consistent and will provide the same sha1sum afterwards. I haven’t tested resuming, but I assume it works.Conclusion
That should be enough information for anybody to easily write e.g. a selenium bot to get all this information to automate a download (I quickly did something similar to this in the past for something unsupported by yt-dlp, but with easy download mp4 source files, this was for logging in and getting a list of videos, instead of not needing to write a decryptor). Otherwise, this is what we need to do:
chave
link to a normal url if necessary.I might try and do these things myself. Only thing is, I’ve already spent a few hours fiddling around, figuring this out, and documenting this. And I don’t know much about the yt-dlp codebase and contributing to it the best – though my day job is Python. I’m mostly just trying to get a semi-quick and dirty way of fixing this, then maybe one of you can finish and fix it. IDK. It’s been cool learning about all this, though. (And I do need these videos downloaded to do training for my job efficiently.)
Last thought: I wonder if Hotmart is just using someone else’s HLS implementation, in particular, their encryption code, so I wonder if there is another extractor already written which shared the
chave
decryption code, to make all of this easier. Maybe even the URL stuff too?Until this gets fixed I have written my own script for downloading courses https://github.com/FallingLights/Teachable-dl
@Abdess 's repo doesn’t work for me. Is this the Cloudflare issue mentioned above?
@bruhcephalus the changes made by @Green0Photon are being integrated here: https://github.com/yt-dlp/yt-dlp/pull/7650
Could you please test if this fork works and give some feedback? : https://github.com/Abdess/yt-dlp/tree/teachable-fix-add-hotmart
Any feedback is welcome to find out if the changes are working properly. The only remaining issue is the login which causes a 403 error, which is due to Cloudflare’s security. 😃
I just sent a message to reddit
@kzorfy I’ve got a beta version for you now. https://github.com/Green0Photon/yt-dlp/tree/teachable-fix-add-hotmart – or download directly.
I forced pushed the branch since I had a bad commit I didn’t want included in the final PR, just in case you already cloned it.
I’ve been able to run my previous command where I can download a whole course. Just slower unfortunately because this move to HLS means it’s downloading lots of small files instead of one bigger one – even then, still seems oddly slow. But it has chapter number and stuff.
That said, I want to investigate more and make sure more metadata stuff is good, plus add tests later. Make sure I’m fully following this project’s best practices.
But it would be super great to see people trying this out on different sites. I’ve even added description hopefully downloading too.
Might be a little while before I add more progress, but I’m curious of people’s thoughts of my code here, too.