pip: New resolver: pip seems to not do anything and stuck during resolution

Environment

pip 20.2 (also tried master)
Python 3.6

On a clean environment, running pip install --use-feature=2020-resolver -r requirements.txt

boto3>=1.14.33
elasticsearch-dsl>=7.1,<7.2
elasticsearch>=7.0,<8.0
user-agents>=1.1.0
requests_mock>=1.6
awsebcli>=3.18

The installation is stuck indefinitely, repeating the last output, see log. I kept it running for 5+ minutes but it keeps going (unreasonable for such a short list of requirements). I cut the log short since it repeats itself.

Collecting boto3>=1.14.33
  Using cached boto3-1.14.33-py2.py3-none-any.whl (129 kB)
Collecting elasticsearch-dsl<7.2,>=7.1
  Using cached elasticsearch_dsl-7.1.0-py2.py3-none-any.whl (53 kB)
Collecting botocore<1.18.0,>=1.17.33
  Using cached botocore-1.17.33-py2.py3-none-any.whl (6.5 MB)
Processing /home/noam/.cache/pip/wheels/99/f3/df/12f8953cef0e59c251c6e9cf7816d202e8e778a3a928a645a6/user_agents-2.1-cp36-none-any.whl
Collecting ua-parser>=0.9.0
  Using cached ua_parser-0.10.0-py2.py3-none-any.whl (35 kB)
Collecting requests_mock>=1.6
  Using cached requests_mock-1.8.0-py2.py3-none-any.whl (23 kB)
Processing /home/noam/.cache/pip/wheels/c2/3a/2f/3c725041250659af00a69f4fd3e211f69a1638e4ad79701fc2/awsebcli-3.18.2-py3-none-any.whl
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Processing /home/noam/.cache/pip/wheels/22/23/ff/a2333c06c7cee314937b0e24e46386d070082213ca940e71c6/awsebcli-3.18.1-cp36-none-any.whl
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Collecting awsebcli>=3.18
  Using cached awsebcli-3.18.0.tar.gz (249 kB)
Collecting requests_mock>=1.6
  Using cached requests_mock-1.7.0-py2.py3-none-any.whl (23 kB)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Collecting requests_mock>=1.6
  Using cached requests_mock-1.6.0-py2.py3-none-any.whl (22 kB)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
Requirement already satisfied: setuptools>=20.0 in /home/noam/.virtualenvs/tmp-a6fdd377fc9345b0/lib/python3.6/site-packages (from awsebcli>=3.18->-r requirements.txt (line 6)) (49.2.0)
...

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 4
Comments: 20 (15 by maintainers)

Most upvoted comments

Yeah, I ran the full requirements set with a few print statements locally, and it seems like the resolver is just backtracking like hell. The problem is that the output pip generates is not doing a good job reporting the situation to the user, and it seems stuck doing repeated things. (They are not the same thing, but different things generating the same output.)

Edit: I’m marking this as an enhancement since pip is arguably doing the correct thing here, but needs to do it better.

Edit 2: For maintainers wondering, the repeated setuptools line is printed because the resolver is trying to build different versions of awsebcli from source.

uranusjr on Aug 2, 2020

An additional observation. Just using two requirements out of the ones mentions causes a ResolutionImpossible error.

$ cat requirements.txt 
boto3>=1.14.33
awsebcli>=3.18

$ pip install --use-feature=2020-resolver -r requirements.txt 
Collecting boto3>=1.14.33
  Using cached boto3-1.14.33-py2.py3-none-any.whl (129 kB)
Collecting botocore<1.18.0,>=1.17.33
  Using cached botocore-1.17.33-py2.py3-none-any.whl (6.5 MB)
Collecting awsebcli>=3.18
  Using cached awsebcli-3.18.2.tar.gz (246 kB)
Collecting awsebcli>=3.18
  Using cached awsebcli-3.18.1.tar.gz (246 kB)
Collecting awsebcli>=3.18
  Using cached awsebcli-3.18.0.tar.gz (249 kB)
ERROR: Cannot install -r requirements.txt (line 1), -r requirements.txt (line 2), -r requirements.txt (line 1), -r requirements.txt (line 2), -r requirements.txt (line 1) and -r requirements.txt (line 2) because these package versions have conflicting dependencies.

The conflict is caused by:
    boto3 1.14.33 depends on botocore<1.18.0 and >=1.17.33
    awsebcli 3.18.2 depends on botocore<1.16 and >=1.15
    boto3 1.14.33 depends on botocore<1.18.0 and >=1.17.33
    awsebcli 3.18.1 depends on botocore<1.16 and >=1.15
    boto3 1.14.33 depends on botocore<1.18.0 and >=1.17.33
    awsebcli 3.18.0 depends on botocore<1.16 and >=1.15

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible For help visit: https://pip.pypa.io/en/stable/user_guide/#fixing-conflicting-dependencies

deveshks on Aug 2, 2020