pip: Migrate legacy #egg= requirements to PEP 508 direct URL specifier

Update by maintainer: See https://github.com/pypa/pip/issues/9429#issuecomment-756452705 for the plan to fix this.

Original issue report below:

Environment

pip version: pip 20.3.3
Python version: 3.7
OS: docker (python:3.7 image)

Description

Our Application uses a “requirements.txt” file that contains a reference to a public github repository-

git+ssh://git@github.com/radaisystems/awsscrubber.git

By itself this works fine.

We recently added another library to it-

git+ssh://git@github.com/radaisystems/awsscrubber.git
git+ssh://git@github.com/radaisystems/private.git

This library has a setup.py file in it that also references the “awsscrubber” library (truncated version)-

setup(
    install_requires = [
        'awsscrubber @ git+https://github.com/radaisystems/awsscrubber.git'
    ]
)

By itself both of these have no problem running. As of pip 2.3.3 (and possibly earlier) this results in an error-

Cannot install awsscrubber 0.0.1 (from git+ssh://****@github.com/radaisystems/awsscrubber.git#egg=awsscrubber) and threeproc because these package versions have conflicting dependencies.
ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

The conflict is caused by:
    The user requested awsscrubber 0.0.1 (from git+https://****@github.com/radaisystems/awsscrubber.git#egg=awsscrubber)
    amalthealib 0.1.0 depends on awsscrubber 0.0.1 (from git+https://github.com/radaisystems/awsscrubber.git)

Despite these both pointing at the same repository, and being recognized by pip as being part of the same project.

To resolve this I’ve tried a few things-

Add “egg” query values to both repos.
Made sure to use “https” or “git” in both places.

Expected behavior

I expect it to install the awsscrubber library.

How to Reproduce

Steps to reproduce are described above, as is the specific error.

Work Arounds

Downgrading pip to an older version resolved the problem without causing any conflicts.
Having pip use --use-deprecated=legacy-resolver also removed the problem, again without causing any conflicts.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 17 (14 by maintainers)

Most upvoted comments

Special handling of #egg= sounds to me like the most reasonable thing pip can do. The #egg= fragment has traditionally been treated as a special case by various tools (e.g. setuptools), so any reasonable existing source wouldn’t give it meanings that breaks these legacy tools. We can add appropriate deprecating warnings to signal people to drop it and/or switch to PEP 508 syntax.

If a requirement’s does not contain an egg fragment, use it as it.
If a requirement contains an egg fragment, and is not in PEP 508 format, convert it into PEP 508 with a warning.
If a PEP 508 direct URL requirement’s URL part contains an egg fragment containing an identical package name, from the egg fragment with a warning.
If a PEP 508 direct URL requirement’s URL part contains an egg fragment containing a different package name, fail with an error (requirement invalid).

Does this sound reasonable?

uranusjr on Jan 7, 2021