pytube: HTTP Error 404: Not Found when trying get stream
Describe the bug I need to get URL to upload the video on AWS. The code is simplest as possible
yt = YouTube(video_url)
streams = yt.streams
The video that I want to download: http://youtube.com/watch?v=2lAe1cqCOXo
On the line where we get streams, it returns the error urllib.error.HTTPError: HTTP Error 404: Not Found
It’s happening not always around once at 2-3 requests.
Full error list:
Internal Server Error: /api/external-video-upload/
Traceback (most recent call last):
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/core/handlers/base.py", line 179, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
return callback(request, *args, **kwargs)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/home/alex/Developer/blossom-backend/blossom/recordings/views.py", line 819, in get
youtube_video_upload(video_url, recording)
File "/home/alex/Developer/blossom-backend/blossom/recordings/utils.py", line 791, in youtube_video_upload
streams = yt.streams
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 321, in streams
return StreamQuery(self.fmt_streams)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 214, in fmt_streams
if "adaptive_fmts" in self.player_config_args:
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 188, in player_config_args
self._player_config_args = self.vid_info
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 291, in vid_info
return dict(parse_qsl(self.vid_info_raw))
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 109, in vid_info_raw
self._vid_info_raw = request.get(self.vid_info_url)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/request.py", line 53, in get
response = _execute_request(url, headers=extra_headers, timeout=timeout)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/request.py", line 37, in _execute_request
return urlopen(request, timeout=timeout) # nosec
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 523, in open
response = meth(req, response)
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 632, in http_response
response = self.parent.error(
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 561, in error
return self._call_chain(*args)
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 641, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
Internal Server Error: /api/external-video-upload/
Traceback (most recent call last):
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/core/handlers/base.py", line 179, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
return callback(request, *args, **kwargs)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/home/alex/Developer/blossom-backend/blossom/recordings/views.py", line 819, in get
youtube_video_upload(video_url, recording)
File "/home/alex/Developer/blossom-backend/blossom/recordings/utils.py", line 791, in youtube_video_upload
streams = yt.streams
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 321, in streams
return StreamQuery(self.fmt_streams)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 214, in fmt_streams
if "adaptive_fmts" in self.player_config_args:
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 188, in player_config_args
self._player_config_args = self.vid_info
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 291, in vid_info
return dict(parse_qsl(self.vid_info_raw))
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/__main__.py", line 109, in vid_info_raw
self._vid_info_raw = request.get(self.vid_info_url)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/request.py", line 53, in get
response = _execute_request(url, headers=extra_headers, timeout=timeout)
File "/home/alex/.local/share/virtualenvs/blossom-backend-FLIGhEMr/lib/python3.9/site-packages/pytube/request.py", line 37, in _execute_request
return urlopen(request, timeout=timeout) # nosec
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 523, in open
response = meth(req, response)
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 632, in http_response
response = self.parent.error(
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 561, in error
return self._call_chain(*args)
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/home/alex/.pyenv/versions/3.9.0/lib/python3.9/urllib/request.py", line 641, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
System information
- Python version 3.9
- Pytube version 10.9.3
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 41
- Comments: 82 (4 by maintainers)
Links to this issue
Commits related to this issue
- Fix #1060 (#1067) * User InnerTube in place of `get_video_info` url * Added some additional base parameters for innertube requests. * Added Oauth support for innertube client * Add exception... — committed to pytube/pytube by Zeecka 3 years ago
- Fix pytube#1060 (pytube#1067) — committed to JNYH/pytube by JNYH 2 years ago
I can’t fix this tonight, I will try to work on it tomorrow
Heads up for those following this thread:
The upcoming fix will require authentication for age-restricted videos. For end users, this will prompt the user to go to a webpage and enter a code to allow access to their account. There does not appear to be a way around this, as YouTube is actively killing off the endpoint we used to rely on. I don’t think that you will need to make changes to your code, unless you’re accessing unusual parts of the YouTube object.
For those of you who are commenting, “I have the same error” or “same error” and then showing it’s only the HTTP Error 404: Not Found, can you please not?
I’m not trying to be rude but it’s already clear that we’re all having the same issue. Adding such comment to this thread is not adding any value to those who are following this thread and trying to get updates and also to the contributors who are trying to solve this problem.
I don’t want to speak for the main contributor of this project, but this type of passive-aggressive demand is not welcome in any libre and open-source project. The contributors do not owe you anything, as specified in the license.
That being said, your actual contributions to the project are certainly always welcome.
Sorry I came out a bit rude, but libre and open-source projects contributors expect a bit more maturity from their users.
The fix for this has been pushed to pypi as v11.0.0, so you should be able to install it normally with
python -m pip install --upgrade pytube
Do we have an update on this, since this being an awesome library and with a critical issue like this one, I am having an impact on an already running app. Request to please kindly provide an update on this issue. Also, thanks for the awesome work you guys are doing!
Update: I made a couple slight changes to Zeecka’s solution that should help with the age restrictions. I believe the current version will now work correctly for most age-restricted videos without needing to authenticate. There are certain age-restricted videos that are even more restricted than others that will require authentication, but these videos are very uncommon.
If you are able to, please install the experimental version, test it out, and let us know if there are any unexpected errors. If you run into an
AgeRestrictedError
exception, then you are probably testing against a video that has the higher level of age-restriction, and will need to authenticate. You can do that as follows:The first time that you run this code with the
use_oauth
argument, it will prompt you to visit a URL and input a code to authorize access to your account. This generates an authorization and refresh OAuth token, which are then stored on the machine running the code. Doing this introduces a potential security risk. Specifically, those tokens are scoped to YouTube, which means that if they get exposed, somebody could perform certain actions as though they were your account. Although this does not give direct access to your account, it does allow access to certain features. I’m a little fuzzy on exactly what this scope allows, but YouTube’s public API states that the scope allows the token toManage your YouTube account
. If you do NOT want to store these locally, you can pass another argument,allow_oauth_cache=False
to prevent them from being stored on disk.You can install the experimental fix to test it with
python -m pip install git+https://github.com/Zeecka/pytube@fix_1060
, as @shohamtal says above.This is just proving my point.
We all know this is a popular library. We all want the same goal of having it fixed. We all have projects that depend on this library. The contributors are doing what they can to help us all, but just as @billsioros had referenced to a previous comment made by @glubsy, the contributors also don’t owe us anything.
Please stop with this commentary, as it adds absolutely no value to those who are trying to receives updates and to the contributors who are trying to come up with a fix for everyone.
You are absolutely right Sir. Please accept humble apologies. I know we people(dev) working hard to improve open-source projects to enhance technology and I admire that. Thanks for letting me know, I’ll keep that in mind.
My apologies. Thanks for pointing this out. Being exhausted by trying solutions is no reason to send a request that sounds like a polite demand. I’ll surely learn from this!
Best.
@rishabh3354 actually this is expected since YouTube always block apis like these and make updates on them from time to time.
let us know the updates asap. our project is delaying due to this.
@clo0wn this solution will work for unrestricted videos without authentication, we’re just seeing weird errors on some videos.
You can test Zeecka’s experimental solution. Worked great for me:
pip install git+https://github.com/Zeecka/pytube@fix_1060
Had to change my code to explicit download mp4 format, otherwise it downloaded 3gp format:
YouTube("https://www.youtube.com/watch?v=7kZqas35FTA").streams.filter(progressive=True, file_extension='mp4').first().download()
I have the same error too!! urllib.error.HTTPError: HTTP Error 404: Not Found
If yesterday one of the 2-3 requests was a success, today all requests failed. I tried to handle it in the cycle, and if the request fails then repeat the request. Else return stream, but now all requests fail.
Even more thread with same 404 will come… That’s because its a popular library. And people wants it to be fixed soon. Their project depends on this library. All we can do is fix it for all soon.
Hello everyone, I have same problem!
Its been more than a week. No sign of hope… 🥺😟😒😒😒. Pytube was one of my favorite libraries. It can’t be Dead. 😒😒😒😒
Same problem here aswell when using
Traceback:
It’s not dead, just having problems trying to fix it, and I only have a few hours each night to work on it. Most of the updates so far can be found in #1067. Please do not post in that pull request, however, unless you are contributing to the fix.
Streamlink is pretty reliable for downloading Youtube videos, it’s just missing many PyTube features like checking video file size, checking video length, getting progress callback, etc. Maybe add it as a backup to your own project?
You can even get back most of PyTube’s features by using Youtube’s real API with an API key.
This error already happened in the past, basically youtube updated the URLs, so PyTube couldn’t download the video, at the time the solution was a new Pytube update, at the moment I’m not having problems in my application with this error, maybe that’s just it is happening to some people.
Hey guys, I’m getting HTTP 410 Gone but the video is definitely not gone. Maybe a bug?
To clarify, this is with
pip install git+https://github.com/Zeecka/pytube@fix_1060
https://github.com/ytdl-org/youtube-dl are able to do it by parsing the webpage.
@must-defend-500 I’m hoping to get to it tomorrow. I need to fix a bunch of unit tests, and it’s taking a while (most of the old ones relied on data generated by the old get_vid_info url, so now that objects are being created in a different way, its causing issues with our mock objects)
Just so you know, even if I did the initial PR, @tfdahlin was already aware of the workaround and worked on the biggest part of the fix (oauth / Age restriction bypass). Thank you for your work.
@rishabh3354 as @glubsy previously stated.
Hi, just confirming: I have the same error (as it seems with any youtube link).
urllib.error.HTTPError: HTTP Error 404: Not Found
I hope the solution to that problem will be found soon, as it’s great lib, fast and easy to use.
I tried this link : https://www.youtube.com/watch?v=vdFJ9lNdpwk
code i used below :
Error after running above code : Traceback (most recent call last): File “c:/Users/username/nlp_repo/AzureOCR/TestPython.py”, line 12, in <module> yt.streams.filter(file_extension = “mp4”) File “C:\Users\username\Anaconda3\lib\site-packages\pytube_main_.py”, line 286, in streams return StreamQuery(self.fmt_streams) File “C:\Users\username\Anaconda3\lib\site-packages\pytube_main_.py”, line 192, in fmt_streams video = Stream( File “C:\Users\username\Anaconda3\lib\site-packages\pytube\streams.py”, line 62, in init self._filesize: Optional[int] = stream[‘contentLength’] # filesize in bytes KeyError: ‘contentLength’
shared code was working earlier 20 days ago
Yes, you are Absoutly right, but you didn’t answer his question.?
@tfdahlin Please post an updated fix. This is stalling my work. I’m not able to understand why this is happening.
I am getting the same error, tried with python 3.9. 3.8. 3.7. No solution.
Pytube version:
pytube-10.9.3
code:error:
Hoping to recieve a fix soon!
@rishabh3354 This behavior changed in v10.9.3, and should only happen if you’re using the
filename
argument when downloading. When you specify a custom filename, you should now include the extension as part of the filename (which you can access withstream.subtype
). If you do not use the filename argument, a filename will be automatically generated that correctly includes the extension.Yes, it will, but it needs to be tested more thoroughly before I push it there, hence why I’m asking for help testing.
I’m glad you had a solution for the error 🙌🙌, I just have a doubt that is: If in my code I don’t put the use_oauth, and put it to download normally nothing will be stored on my machine right?
Same here, actually impacting my project too. But anyways i am sure they will make it. thanks guys
@tfdahlin good catch, Ive got python and python3 issues on my computer. I don’t know why I can’t intsall it from Github. I’ll just wait for you guys to merge with master. Any sense when that might happen?
python -m pip install git+https://github.com/Zeecka/pytube@fix_1060
worked for me, thank you SO MUCH @tfdahlin. I will keep testing it out and let you know if i encounter any issues.That is correct. Nothing is stored on your machine if you don’t include the
use_oauth
argument.I added a fix for this in #1067. Still trying to figure out how the cipher calculation stuff broke though.
Here’s the Dockerfile which built this software:
We really do appreciate your hard work, Sir! Hopefully, the next update will be released soon.
For me Zeeckas fix_1060 works: https://github.com/Zeecka/pytube/tree/fix_1060