requests: Python 3.7.6: urllib3.exceptions.ProxySchemeUnknown
Proxy setup without scheme is not working on Python 3.7.6
While it works on Python 3.7.5:
Python 3.7.5 (default, Nov 23 2019, 05:59:34)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get("http://httpbin.org/get", proxies={"http": "103.250.166.4:6666"})
<Response [200]>
Is not under Python 3.7.6
Python 3.7.6 (default, Dec 21 2019, 08:28:11)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get("http://httpbin.org/get", proxies={"http": "103.250.166.4:6666"})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 412, in send
conn = self.get_connection(request.url, proxies)
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 309, in get_connection
proxy_manager = self.proxy_manager_for(proxy)
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 199, in proxy_manager_for
**proxy_kwargs)
File "/usr/local/lib/python3.7/site-packages/urllib3/poolmanager.py", line 470, in proxy_from_url
return ProxyManager(proxy_url=url, **kw)
File "/usr/local/lib/python3.7/site-packages/urllib3/poolmanager.py", line 420, in __init__
raise ProxySchemeUnknown(proxy.scheme)
urllib3.exceptions.ProxySchemeUnknown: Not supported proxy scheme None
It seems that some changes on Python urllib.parse.urlparse brake requests.utils.prepend_scheme_if_needed
>>> utils.prepend_scheme_if_needed('103.250.166.4:6666', 'http')
'103.250.166.4://6666'
So it looks like this utils has to update to match Python changes (I think the change that produces this is https://github.com/python/cpython/commit/5a88d50ff013a64fbdb25b877c87644a9034c969). In both cases I’m using requests 2.22.0.
Reporting here since urllib.parse.urlparse behaviour is wrong in both Python versions and I think the problem is requests.utils.prepend_scheme_if_needed problem:
Python 3.7.5
>>> from urllib import parse
>>> parse.urlparse("103.250.166.4:6666")
ParseResult(scheme='', netloc='', path='103.250.166.4:6666', params='', query='', fragment='')
Python 3.7.6
>>> from urllib import parse
>>> parse.urlparse("103.250.166.4:6666")
ParseResult(scheme='103.250.166.4', netloc='', path='6666', params='', query='', fragment='')
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 18 (4 by maintainers)
Commits related to this issue
- patch around https://github.com/psf/requests/issues/5297 by prefixing http:// for proxies — committed to WIPACrepo/iceprod by dsschult 4 years ago
yes, this fixes the issue for 3.7.6 or 3.8.x either by
--proxy http://<ip>:<port>or in (Windows’) %APPDATA%\pip\pip.ini,[global]section:http://<ip>:<port>proxy={'http': 'http://91.98.102.84:1080', 'https': 'http://91.98.102.84:1080'}you can solve the problem?? i have the same issue with python 3.8
you can solve the problem?? i have same issue with python 3.8
Thanks for reporting this issue and doing investigation!
You can fix this by using
http://<ip>:<port>instead of just an IP.I had to re-edit my answer.
urllib3 and requests proxy implementations are distinctly different.
From requests: Based on the requests Proxies section; https://docs.python-requests.org/en/master/user/advanced/#proxies
Notice there is no “s” in https property value.
From urllib3: Based on the the urllib3 Proxies section; https://urllib3.readthedocs.io/en/stable/advanced-usage.html?highlight=proxy#proxies
I sove this by modify pip_vendor\requests\adapters.py def get_connection(self, url, proxies=None): “”"Returns a urllib3 connection for the given URL. This should not be called from user code, and is only exposed for use when subclassing the :class:
HTTPAdapter <requests.adapters.HTTPAdapter>.