pip: New resolver reports version conflict on github repo URL variations

The resolver appears to treat projectname @ git+https://git@github.com/org/repo.git as different from projectname @ git+https://github.com/org/repo.git even though they’re the same thing and the only difference is the “git@” in the url. If something is required directly and indirectly (via another requirement) using slightly different but totally valid URL variations, pip fails with a version conflict error.

pip install -r requirements.txt on a requirements.txt containing:

project-abc @ git+https://github.com/org/project-abc.git
project-def @ git+https://github.com/org/project-def.git

Where project-def has a requirements.txt containing the “git@” URL variant:

project-abc @ git+https://git@github.com/org/project-abc.git

Produces

ERROR: Cannot install -r requirements.txt (line 2) ... because these package
versions have conflicting dependencies.

The conflict is caused by:
    The user requested project-abc 0.1.0 (from git+https://github.com/org/project-abc.git)
    project-def 0.1.2 depends on project-abc 0.1.0 (from git+https://****@github.com/org/project-abc.git)

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

Removing the “user@” from the second requirements.txt avoids this, but it worked before the resolver upgrade and may require changes to packages upstream from the user.

Expected Behavior

Working URL variations for the same package should not affect version resolution.

Environment details

pip 20.3.1 macOS Mojave 10.14.6 Python 3.8.6

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 21 (8 by maintainers)

Commits related to this issue

Use a github url without .git extension Having some issues with pip conflicts where other ckan extensions have the same dependencies but without the .git part. This causes a version conflict. Joy! Se... — committed to NaturalHistoryMuseum/ckanext-statistics by jrdh 3 years ago

Most upvoted comments

It is possible, but would require a feature request and substantial discussion, since the current behaviour (use the revision pointed by the default branch) is well-established and likely depended by many.

Variants of a Python package are ultimately identified by their versions, and this mechanism is built into many areas of Python. PEP 508 URL is a mean to specify where pip should download things to satisfy a package version request. By giving semantics to a Git URL’s revision information, we’d essentially introduce an alternative versioning scheme parallel to the package’s version metadata, which would not play well with other parts of packaging. Not that the revision can’t be used as package version—Go’s packaging solution does basically that—but the scheme does not work well in Python packaging without substential designing to fit into the current version system.

uranusjr on Dec 10, 2020

Wouldn’t it possible to modify pip so that it at least considers that when the urls just differ by presence / absence of a commit hash, the one without the commit hash should be considered as satisfied by the one with the commit hash?

blaiseli on Dec 10, 2020

From pip’s perspective, any URL spec is pinning the requirement. So it does not really matter; by using a URL is setup.py, you’re already pinning.

uranusjr on Dec 10, 2020