pip: pip 21.0.1 fails when run with warnings converted to errors

Environment

pip version: 21.0.1
Python version: 3.9.1
OS: Windows

Description With the latest version of packaging (vendored in 21.0.1) a DeprecationWarning is issued when parsing a “legacy version”. If pip is run with warnings converted to errors, this causes a failure.

Expected behavior No error

How to Reproduce py -wE -m pip --version

Or to pinpoint it further,

py -wE
>>> from pip._vendor import pkg_resources

This does not happen with setuptools 52.0.0, it appears to be related to the version of setuptools (44.0.0) that we vendor.

Output

Traceback (most recent call last):
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\packaging\version.py", line 57, in parse
    return Version(version)
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\packaging\version.py", line 298, in __init__
    raise InvalidVersion("Invalid version: '{0}'".format(version))
pip._vendor.packaging.version.InvalidVersion: Invalid version: 'pip'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3252, in <module>
    def _initialize_master_working_set():
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 574, in _build_master
    ws = cls()
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 567, in __init__
    self.add_entry(entry)
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 623, in add_entry
    for dist in find_distributions(entry, True):
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2061, in find_on_path
    path_item_entries = _by_version_descending(filtered)
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2034, in _by_version_descending
    return sorted(names, key=_by_version, reverse=True)
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2032, in _by_version
    return [packaging.version.parse(part) for part in parts]
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2032, in <listcomp>
    return [packaging.version.parse(part) for part in parts]
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\packaging\version.py", line 59, in parse
    return LegacyVersion(version)
  File "C:\Users\Gustav\AppData\Local\Programs\Python\Python39\lib\site-packages\pip\_vendor\packaging\version.py", line 127, in __init__
    warnings.warn(
DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 15 (14 by maintainers)

Most upvoted comments

OK, I had some time to debug. Setuptools is the issue, in _by_version_descending, the nested function _by_version parses every part of the filename, to make a sort key.

TBH, I’m not even sure how the code achieves what the docstring says it does. I think it’s relying on the fact that LegacyVersion objects sort like normal strings, but it may well also be trying to handle non-PEP 440 versions somehow.

    def _by_version(name):
        """
        Parse each component of the filename
        """
        name, ext = os.path.splitext(name)
        parts = itertools.chain(name.split('-'), [ext])
        return [packaging.version.parse(part) for part in parts]

The following fixes it:

    def _by_version(name):
        """
        Parse each component of the filename
        """
        name, ext = os.path.splitext(name)
        parts = itertools.chain(name.split('-'), [ext])
        def safe_parse(s):
            try:
                return packaging.version.parse(s)
            except (packaging.version.InvalidVersion, DeprecationWarning):
                return s
        return [safe_parse(part) for part in parts]

but I don’t like (1) that it’s not obvious to me that it does the same as the original in edge cases, and (2) that we have to catch DeprecationWarning for the “warnings as errors” case.

I’ve submitted https://github.com/pypa/setuptools/issues/2547 to let them know about the issue, but it’s not likely to hit them soon, so I think we need a local solution for now.

pfmoore on Jan 31, 2021