pip: Pip --proxy parameter is not been passed to the request.

pip version

21.0.1

Python version

3.8

OS

Windows 10 / Linux

Additional information

The problem is easier to see if you are behind a proxy.

Description

Running pip install package --proxy http://my_proxy DO NOT pass the proxies to requests.Session.request(…).

As far as I could check this problem is there since version 20.2.2 at least. It was unnoticed until now because, before urllib update, it didn’t check ssl_certificate so it didn’t matter if your scheme was either http or https. Therefore, this bug is partially a root cause for the SSLError in recent versions of pip. Since users don’t have the ability to pass the proxy to pip directly.

Expected behavior

I would expect that running pip install with --proxy, the proxy set here would be passed to the request, overriding any system configuration.

How to Reproduce

Start a fresh venv with pip and setuptools only
Then set a working proxy using system var. i.e: set https_proxy: http://your_proxy or export https_proxy: http://your_proxy for Linux.
Now run pip passing a fake proxy, i.e: pip install numpy --proxy http://127.0.0.2:80 or pip.main([‘install’, ‘–proxy=http://127.0.0.2:80’, ‘numpy’]) for esier debug.
You should expect an error when installing, but if you have your proxy set correctly in the system it will NOT fail.

PS: You can also do the inverse process. Set a fake proxy on the system and run pip --proxy using a working one. Now you should get an error.

Output

Code of Conduct

I agree to follow the PSF Code of Conduct

I have already created a post about it here, including a working solution.

About this issue

Original URL
State: open
Created 3 years ago
Comments: 21 (14 by maintainers)

Commits related to this issue

Update sessions.py This change will make the hierarchy request **kwargs -> Session args -> environment to be respected. This also solves problems with pip as discussed [here](https://github.com/pypa/... — committed to junqfisica/requests by junqfisica 3 years ago
Pass the session's proxies property to request After waiting a long time for a fix on the side of the request that could fix pip install issues with proxy as already discussed in #9691. It seems that... — committed to junqfisica/pip by junqfisica 3 years ago

Most upvoted comments

I would encourage putting more effort pushing requests to merge and release a fix. Arguing about semantics here is not very helpful for anything.

uranusjr on Mar 8, 2021

request.Session … is more related to how they fetch proxies from the system env.

parse system env proxies wrongly is another bug of cpython, to be precise, it is in cpython’s urllib library.

PipSession is not given its proxy to the parent request.Session.

as i post above, pip build pipSession with options, and the options contain proxies. and pipSession inherits requests.Session. so pipSession is actually a requests.Session with some more functions.
req_command build pipSession with options, means it build a requests.Session with some more funtions and passes options to the constructor of requests.Session.

so, the ultimate problem is that requests.Session does not handle these parameters correctly.

besides, self.setdefault() is actually overriding. default value is set while pipSession/requests.Session is constructing. but requests.Session use system env value instead of parameter (this is a bug). then if you use self.setdefault(), it will override the default value that has been set yet in the construct.
so, overriding is reasonable only if requests do not fix it.

CrazyBoyFeng on Mar 6, 2021

For me is clear that this specific bug is caused because PipSession is not given its proxy to the parent request.Session.

pip sets the proxies here:

https://github.com/pypa/pip/blob/031c34e94a95c6de9236b26f7b0710588260bbb5/src/pip/_internal/cli/req_command.py#L108-L113

where session.proxies is a documented requests.Session attribute that should be preferred over environment varialbes. If that’s true, PipSession would not need to pass the proxy values on every Session.request() call. Turns out that’s not actually the case, which is a requests bug. The bug can be avoided by PipSession explicitly passing the proxies argument, but calling this a bug is like saying it’s your fault you’re using an API that behaves incorrectly, which is not a correct characterisation IMO.

uranusjr on Mar 6, 2021