pip: 23.2: Legacy resolver triggers error "while checking for conflicts"

Description

I got that error, requesting that I post an issue:

ERROR: Error while checking for conflicts. Please file an issue on pip's issue tracker: https://github.com/pypa/pip/issues/new
Traceback (most recent call last):
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 575, in _determine_conflicts
    return check_install_conflicts(to_install)
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/operations/check.py", line 108, in check_install_conflicts
    would_be_installed = _simulate_installation_of(to_install, package_set)
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/operations/check.py", line 131, in _simulate_installation_of
    dist = abstract_dist.get_metadata_distribution()
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/distributions/wheel.py", line 23, in get_metadata_distribution
    assert self.req.local_file_path, "Set as part of preparation during download"
AssertionError: Set as part of preparation during download

Expected behavior

I expected no crash.

pip version

pip-23.2-py3-none-any.whl

Python version

Python3.7

OS

Ubuntu

How to Reproduce

The CI of Nevergrad (repeatedly) leads to this crash: https://app.circleci.com/pipelines/github/facebookresearch/nevergrad/8643/workflows/c916c6c6-edcb-4ca6-bed9-ca7b9c98599c/jobs/27932

Output

 Stored in directory: /home/circleci/.cache/pip/wheels/fa/cd/1f/c6b7b50b564983bf3011e8fc75d06047ddc50c07f6e3660b00
  Building wheel for skrebate (pyproject.toml) ... - done
  Created wheel for skrebate: filename=skrebate-0.62-py3-none-any.whl size=29253 sha256=4ea15243891445733bd193094aeddd6ba1d3067eb751756c21bfdc787963043c
  Stored in directory: /home/circleci/.cache/pip/wheels/4f/3b/07/22a9a01f3f25cd376122b5ef526737a44993b4c312e50916ed
  Building wheel for GPUtil (pyproject.toml) ... - done
  Created wheel for GPUtil: filename=GPUtil-1.4.0-py3-none-any.whl size=7393 sha256=118f8016f44323a7e9f193bb5f39b8531975de242d6e374ace47b5fd1d34a834
  Stored in directory: /home/circleci/.cache/pip/wheels/6e/f8/83/534c52482d6da64622ddbf72cd93c35d2ef2881b78fd08ff0c
  Building wheel for libsvm (pyproject.toml) ... - \ | done
  Created wheel for libsvm: filename=libsvm-3.23.0.4-cp37-cp37m-linux_x86_64.whl size=262588 sha256=e45bce3692537df310c15f35a1c169cc66fe8f431cbe33997a97bf26c9e32b4a
  Stored in directory: /home/circleci/.cache/pip/wheels/cd/e8/1e/bf95cf256e4d3ffc94289ab508c49d48e34c98220af63e3513
  Building wheel for support-developer (pyproject.toml) ... - done
  Created wheel for support-developer: filename=support_developer-1.0.5-py3-none-any.whl size=5630 sha256=bd7bdab0086ec6c9d26d7dc644bb467db129ea14d576959ab5597d04ff96258d
  Stored in directory: /home/circleci/.cache/pip/wheels/e4/d3/9b/d4f296a0183daacaf202d8b69df27edca88326a47da66cd624
  Building wheel for pyDOE (pyproject.toml) ... - done
  Created wheel for pyDOE: filename=pyDOE-0.3.8-py3-none-any.whl size=18167 sha256=f3e8655c666dc3d648c7872436cec090b2782731cd4e0a05bb3f67a780f75dab
  Stored in directory: /home/circleci/.cache/pip/wheels/83/ce/8a/87b25c685bfeca1872d13b8dc101e087a9c6e3fb5ebb47022a
  Building wheel for sklearn (pyproject.toml) ... - done
  Created wheel for sklearn: filename=sklearn-0.0-py2.py3-none-any.whl size=1302 sha256=bc09e67e8e39fa3f926a583ac3dc5f1df182465fb0a13453d4e23f4da8fdf834
  Stored in directory: /home/circleci/.cache/pip/wheels/46/ef/c3/157e41f5ee1372d1be90b09f74f82b10e391eaacca8f22d33e
Successfully built nevergrad gym gym-anm silence-tensorflow websocket-server future skrebate GPUtil libsvm support-developer pyDOE sklearn
ERROR: Error while checking for conflicts. Please file an issue on pip's issue tracker: https://github.com/pypa/pip/issues/new
Traceback (most recent call last):
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 575, in _determine_conflicts
    return check_install_conflicts(to_install)
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/operations/check.py", line 108, in check_install_conflicts
    would_be_installed = _simulate_installation_of(to_install, package_set)
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/operations/check.py", line 131, in _simulate_installation_of
    dist = abstract_dist.get_metadata_distribution()
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/distributions/wheel.py", line 23, in get_metadata_distribution
    assert self.req.local_file_path, "Set as part of preparation during download"
AssertionError: Set as part of preparation during download
Installing collected packages: numpy, cma, scipy, joblib, threadpoolctl, scikit-learn, colorama, bayesian-optimization, typing-extensions, six, python-dateutil, pytz, pandas, zipp, importlib-metadata, click, platformdirs, pathspec, mypy-extensions, tomli, typed-ast, black, mypy, iniconfig, packaging, pluggy, exceptiongroup, pytest, coverage, pytest-cov, lazy-object-proxy, wrapt, astroid, isort, mccabe, tomlkit, dill, pylint, wheel, sphinxcontrib-applehelp, sphinxcontrib-devhelp, sphinxcontrib-jsmath, sphinxcontrib-htmlhelp, sphinxcontrib-serializinghtml, sphinxcontrib-qthelp, MarkupSafe, Jinja2, Pygments, docutils, snowballstemmer, babel, alabaster, imagesize, charset-normalizer, idna, urllib3, certifi, requests, sphinx, sphinxcontrib-jquery, sphinx-rtd-theme, commonmark, recommonmark, pkginfo, webencodings, bleach, readme-renderer, requests-toolbelt, more-itertools, jaraco.classes, importlib-resources, pycparser, cffi, cryptography, jeepney, SecretStorage, keyring, rfc3986, mdurl, markdown-it-py, rich, twine, autodocsumm, pyparsing, xlwt, xlrd, opencv-python, cycler, fonttools, kiwisolver, Pillow, matplotlib, cloudpickle, gym-notices, gym, qdldl, osqp, ecos, scs, cvxpy, websocket-client, websocket-server, gym-anm, pygame, nvidia-cuda-runtime-cu11, nvidia-cublas-cu11, nvidia-cudnn-cu11, nvidia-cuda-nvrtc-cu11, torch, parso, jedi, decorator, pickleshare, traitlets, wcwidth, prompt-toolkit, backcall, matplotlib-inline, ptyprocess, pexpect, ipython, Werkzeug, itsdangerous, flask, brotli, flask-compress, soupsieve, beautifulsoup4, hiplot, fcmaes, et-xmlfile, openpyxl, pyproj, tqdm, torchvision, nose, PyUtilib, ply, pyomo, mixsimulator, networkx, future, py4j, hyperopt, IOHexperimenter, patsy, statsmodels, skrebate, GPUtil, cdt, tensorflow-estimator, imageio, tifffile, PyWavelets, scikit-image, absl-py, astunparse, flatbuffers, gast, google-pasta, grpcio, h5py, keras, libclang, opt-einsum, protobuf, cachetools, pyasn1, pyasn1-modules, rsa, google-auth, oauthlib, requests-oauthlib, google-auth-oauthlib, markdown, tensorboard-data-server, tensorboard-plugin-wit, tensorboard, termcolor, tensorflow-io-gcs-filesystem, tensorflow, libsvm, image-quality, autograd, pymoo, Keras-Preprocessing, support-developer, silence-tensorflow, dm-tree, tensorflow-probability, pyDOE, sklearn, tabulate, sobol-seq, py-expression-eval, bayes-optim, nlopt, pybullet, box2d-py, glfw, pyopengl, mujoco, olymp, nevergrad
ERROR: Exception:
Traceback (most recent call last):
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 460, in run
    pycompile=options.compile,
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/req/__init__.py", line 79, in install_given_reqs
    pycompile=pycompile,
  File "/home/circleci/repo/venv/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 805, in install
    assert self.local_file_path
AssertionError

Exited with code exit status 2 CircleCI received exit code 2

Code of Conduct

I agree to follow the PSF Code of Conduct.

About this issue

Original URL
State: closed
Created a year ago
Reactions: 5
Comments: 39 (13 by maintainers)

Commits related to this issue

chore(deps): Bump pip from 23.1.2 to 23.2.1 (#1426) Bumps [pip](https://github.com/pypa/pip) from 23.1.2 to 23.2.1. <details> <summary>Changelog</summary> Sourced from <a href="https://git... — committed to kai687/sphinxawesome-theme by dependabot[bot] a year ago
Bump pip from 23.2 to 23.2.1 (PR #6298) Bumps [pip](https://github.com/pypa/pip) from 23.2 to 23.2.1. <details> <summary>Changelog</summary> Sourced from <a href="https://github.com/pypa/pip/b... — committed to inmanta/inmanta-core by dependabot[bot] a year ago
Bump pip from 23.2 to 23.2.1 (PR #6300) Bumps [pip](https://github.com/pypa/pip) from 23.2 to 23.2.1. <details> <summary>Changelog</summary> Sourced from <a href="https://github.com/pypa/pip/b... — committed to inmanta/inmanta-core by dependabot[bot] a year ago
chore(deps): bump pip from 23.2 to 23.2.1 in /.github/workflows (#74) Bumps [pip](https://github.com/pypa/pip) from 23.2 to 23.2.1. <details> <summary>Changelog</summary> Sourced from <a h... — committed to klarna-incubator/mleko by ErikBavenstrand a year ago
chore(deps-dev): Bump pip from 23.2 to 23.2.1 in /packages/jsii-pacmak/test/generated-code (#4192) Bumps [pip](https://github.com/pypa/pip) from 23.2 to 23.2.1. <details> <summary>Changelog</summary>... — committed to aws/jsii by dependabot[bot] a year ago
Update pip requirement from <23.2,>=21.0 to >=21.0,<23.3 (#38) Updates the requirements on [pip](https://github.com/pypa/pip) to permit the latest version. <details> <summary>Changelog</summary> ... — committed to torbennehmer/hacs-e3dc by torbennehmer a year ago
Bump pip from 23.1.2 to 23.2.1 (#165) Bumps [pip](https://github.com/pypa/pip) from 23.1.2 to 23.2.1. <details> <summary>Changelog</summary> Sourced from <a href="https://github.com/pypa/p... — committed to gignsky/tdarr-node-switcher by gignsky 10 months ago

Most upvoted comments

I would strongly advise that regardless of when the fix is released, if removing the use of the legacy resolver works for you, you should do that. At some point we will remove the legacy resolver altogether, so you should not continue using it unless you have a very good reason to do so.

Having said that, the fix is likely to be included in a bugfix release of pip 23.2, which I hope to have done in the next few days, unless further issues arise. I’d like a few more confirmations that the fix PR works, so if you can test it that would be great.

pfmoore on Jul 19, 2023

OK, if this is only a problem with the legacy resolver, I would strongly recommend switching to the “new” resolver (which is hardly new, at this point it’s been round for 3 years).

pfmoore on Jul 17, 2023

^That also fixes my repro

ckw017 on Jul 18, 2023

^In a totally unscientific manner, changing if self.require_hashes to if True also fixes the repro, so probably.

ckw017 on Jul 18, 2023

The reason I’m asking is that there are some known restrictions on the PEP 658/714 code, because of some weird interactions with the internals of the requirement processing. In particular, we deferred some problems with sdist metadata, because it can never occur right now (PyPI doesn’t serve separate metadata for sdists, and until metadata 2.2 is supported, sdist metadata isn’t reliable anyway).

I wouldn’t be surprised if the legacy resolver is behaving weirdly when faced with partially-downloaded requirement objects. The requirement preparation code is pretty complex and fragile, largely because there’s a huge amount of code that’s mainly there for backward compatibility 🙁

It’s definitely a bug, but if switching to the new resolver addresses it, it’s not as critical.

pfmoore on Jul 18, 2023

I’ve got the same problem here. Using python 3.9 and pip 23.2, during building a docker image a got this error:

10.08 ERROR: Error while checking for conflicts. Please file an issue on pip's issue tracker: https://github.com/pypa/pip/issues/new
10.08 Traceback (most recent call last):
10.08   File "/usr/local/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 575, in _determine_conflicts
10.08     return check_install_conflicts(to_install)
10.08   File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/check.py", line 108, in check_install_conflicts
10.08     would_be_installed = _simulate_installation_of(to_install, package_set)
10.08   File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/check.py", line 131, in _simulate_installation_of
10.08     dist = abstract_dist.get_metadata_distribution()
10.08   File "/usr/local/lib/python3.9/site-packages/pip/_internal/distributions/wheel.py", line 23, in get_metadata_distribution
10.08     assert self.req.local_file_path, "Set as part of preparation during download"
10.08 AssertionError: Set as part of preparation during download
10.08 Installing collected packages: asgiref, bcrypt, charset-normalizer, click, coreschema, urllib3, requests, itypes, uritemplate, coreapi, cryptography, pytz, sqlparse, django, django-cors-headers, django-extensions, django-filter, django-pgtrigger, django-postgres-extensions, django-safedelete, django-utils-six, djangorestframework, djangorestframework-jwt, inflection, drf-yasg, hstspreload, httpcore, httpx, markupsafe, msal, openpyxl, phone-iso3166, psycopg2-binary, pyasn1, qrcode, rsa, twilio, unittest-xml-reporting, uwsgi, whitenoise
10.08 ERROR: Exception:
10.08 Traceback (most recent call last):
10.08   File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
10.08     status = run_func(*args)
10.08   File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
10.08     return func(self, options, args)
10.08   File "/usr/local/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 452, in run
10.08     installed = install_given_reqs(
10.08   File "/usr/local/lib/python3.9/site-packages/pip/_internal/req/__init__.py", line 72, in install_given_reqs
10.08     requirement.install(
10.08   File "/usr/local/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 805, in install
10.08     assert self.local_file_path
10.08 AssertionError

MichalTkac on Jul 17, 2023