pip: pip does not ignore yanked version of package in unqualified installation when it's the only non-prerelease choice

Environment

• pip version: 20.1
• Python version: 3.8.2
• OS: Arch Linux (using pyenv + virtualenv)

Description

If you have a package named mypkg with the following versions:

• 1.0 (Yanked)
• 2.0rc0

Running pip install mypkg will install 1.0 rather than 2.0rc0.

Expected behavior I would have expected 2.0rc0 to be installed, since that is what would happen if mypkg only published version 2.0rc0. This seems to be an edge case not super well-clarified in PEP 592, which states:

An installer MUST ignore yanked releases, if the selection constraints can be satisfied with a non-yanked version, and MAY refuse to use a yanked release even if it means that the request cannot be satisfied at all. An implementation SHOULD choose a policy that follows the spirit of the intention above, and that prevents “new” dependencies on yanked releases/files.

I believe 2.0rc0 satisfies the constraints specified if you do not consider “didn’t include --pre” to be a constraint, so depending on whether --pre or its absence a “constraint”, this may be a violation of the spec. I think the fact that pip install will take release candidates if nothing else matches argues in favor of it not being considered a “constraint”.

More clearly, though, it does not fit one of the two suggested implementation strategies:

1. Yanked files are always ignored, unless they are the only file that matches a version specifier that “pins” to an exact version using either == (without any modifiers that make it a range, such as .*) or ===. Matching this version specifier should otherwise be done as per PEP 440 for things like local versions, zero padding, etc.
2. Yanked files are always ignored, unless they are the only file that matches what a lock file (such as Pipfile.lock or poetry.lock) specifies to be installed. In this case, a yanked file SHOULD not be used when creating or updating a lock file from some input file or command.

Regardless of “spec lawyering”, from a user experience point of view, I am personally in favor of pip refusing to install yanked versions at all without an exact version pin (option 1). Right now it installs them but with a warning; I think the warning could be turned into an error that lists the yanked versions, and end users who want to work around the “can’t find a matching version” can add an exact version pin (though admittedly in some circumstances this can have perverse consequences if a package yanks a version because there’s something wrong with it and a consumer pins to the exact version as a workaround, thus preventing an update to a later working, non-yanked version).

How to Reproduce

At the moment the tzdata package has only yanked and release candidate packages on PyPI (and only release candidate packages on Test PyPI). Until a proper release is made of tzdata, you can reproduce it with:

• pip install tzdata: To see the bug
• pip install --index-url https://test.pypi.org/simple/ tzdata: To see pip’s behavior when only release candidates are available.