pip-audit: Does not work with prerelease-only dependencies that have no specific version

Is your feature request related to a problem? Please describe.

pip-audit does not seem to support pre-release packages, or packages that depend on a pre-release package, if the prerelease version isn’t explicitly defined.

Describe the solution you’d like

When a package depends on something a version like *, including pre-releases, pip-audit should be able to find that release. It might be just the case that only packages that ONLY have prereleases aren’t able to be found. Dependency resolution should work the same way as pip install in that it will grab the latest version, even if the only versions are pre-releases.

Describe alternatives you’ve considered

I haven’t figured out any viable alternatives yet, but I was thinking of excluding the problem dependency before running pip-audit. That seems less than ideal because that would end up ignoring vulnerabilities. The less-than-ideal workaround I’m using now is:

pipenv requirements | grep -v sqlmodel > requirements.txt
pip-audit --requirement requirements.txt

Additional context

I have a pipenv package that depends on sqlmodel. When I run pipenv requirements or pipfile2req Pipfile.lock I get these two lines (among others):

sqlalchemy2-stubs==0.0.2a31 ; python_version >= '3.6'
sqlmodel==0.0.8

Pipenv is fine with the explicit version of sqlalchemy, but I believe that’s actually coming from sqlmodel, which has this in their pyproject.toml (using poetry):

[tool.poetry.dependencies]
# ...
sqlalchemy2-stubs = {version = "*", allow-prereleases = true}

It seems like pip-audit can’t determine the version of a prerelease package (I’m assuming this is just the case for ones where there is no non-prerelease version yet):

❯ docker run -it python:3.10-alpine sh
/ # pip install --quiet pip-audit
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
/ # echo sqlmodel > requirements.txt
/ # pip-audit --requirement requirements.txt
Traceback (most recent call last):
  File "/usr/local/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_cli.py", line 455, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_audit.py", line 67, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_service/interface.py", line 155, in query_all
    for spec in specs:
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 116, in collect
    for _, dep in self._collect_cached_deps(filename, reqs):
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 328, in _collect_cached_deps
    for req, resolved_deps in self._resolver.resolve_all(iter(req_values)):
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_dependency_source/interface.py", line 88, in resolve_all
    yield (req, self.resolve(req))
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 77, in resolve
    result = self.resolver.resolve([req])
  File "/usr/local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 521, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 414, in resolve
    raise ResolutionImpossible(self.state.backtrack_causes)
resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=<Requirement('sqlalchemy2-stubs')>, parent=<sqlmodel==0.0.8 wheel=True>), RequirementInformation(requirement=<Requirement('sqlalchemy2-stubs')>, parent=<sqlmodel==0.0.7 wheel=True>), RequirementInformation(requirement=<Requirement('sqlalchemy2-stubs')>, parent=<sqlmodel==0.0.6 wheel=True>), RequirementInformation(requirement=<Requirement('sqlalchemy2-stubs')>, parent=<sqlmodel==0.0.5 wheel=True>), RequirementInformation(requirement=<Requirement('sqlalchemy2-stubs')>, parent=<sqlmodel==0.0.4 wheel=True>), RequirementInformation(requirement=<Requirement('sqlalchemy2-stubs')>, parent=<sqlmodel==0.0.3 wheel=True>), RequirementInformation(requirement=<Requirement('sqlalchemy2-stubs<0.0.2,>=0.0.2-alpha.5')>, parent=<sqlmodel==0.0.2 wheel=True>), RequirementInformation(requirement=<Requirement('sqlalchemy2-stubs<0.0.2,>=0.0.2-alpha.5')>, parent=<sqlmodel==0.0.1 wheel=True>)]

And an even more minified example:

/ # pip-audit --requirement requirements.txt
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 372, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/usr/local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 173, in _add_to_criteria
    raise RequirementsConflicted(criterion)
resolvelib.resolvers.RequirementsConflicted: Requirements conflict: <Requirement('sqlalchemy2-stubs')>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_cli.py", line 455, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_audit.py", line 67, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_service/interface.py", line 155, in query_all
    for spec in specs:
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 116, in collect
    for _, dep in self._collect_cached_deps(filename, reqs):
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 328, in _collect_cached_deps
    for req, resolved_deps in self._resolver.resolve_all(iter(req_values)):
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_dependency_source/interface.py", line 88, in resolve_all
    yield (req, self.resolve(req))
  File "/usr/local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 77, in resolve
    result = self.resolver.resolve([req])
  File "/usr/local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 521, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 374, in resolve
    raise ResolutionImpossible(e.criterion.information)
resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=<Requirement('sqlalchemy2-stubs')>, parent=None)]

However, I’m able to install sqlalchemy2-stubs with pip without any problems, and without specifying explicitly that I want a pre-release:

/ # pip install sqlalchemy2-stubs
Collecting sqlalchemy2-stubs
  Downloading sqlalchemy2_stubs-0.0.2a31-py3-none-any.whl (191 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 191.3/191.3 kB 1.9 MB/s eta 0:00:00
Collecting typing-extensions>=3.7.4
  Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Installing collected packages: typing-extensions, sqlalchemy2-stubs
Successfully installed sqlalchemy2-stubs-0.0.2a31 typing-extensions-4.4.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

About this issue

Original URL
State: closed
Created a year ago
Comments: 16 (16 by maintainers)

Most upvoted comments

#477 has the WIP fix.

woodruffw on Jan 20, 2023

OK, I think I tracked this down.

It looks like pip is using ss.filter((x,)) rather than x in ss, and the behavior is a bit different. The filter function will return prereleases even if you don’t have prereleases=True, but only if there are no non-prereleases. The contains function will always respect prereleases with no fallback. They behave identically if you have prereleases=True which would be the case if you use pip’s --pre option.

Note the documentation in SpecifierSet.filter:

    def filter(
        self, iterable: Iterable[UnparsedVersionVar], prereleases: Optional[bool] = None
    ) -> Iterator[UnparsedVersionVar]:
        """Filter items in the given iterable, that match the specifiers in this set.
        
        :param iterable:
            An iterable that can contain version strings and :class:`Version` instances.
            The items in the iterable will be filtered according to the specifier.
        :param prereleases:
            Whether or not to allow prereleases in the returned iterator. If set to
            ``None`` (the default), it will be intelligently decide whether to allow
            prereleases or not (based on the :attr:`prereleases` attribute, and
            whether the only versions matching are prereleases).
            
        This method is smarter than just ``filter(SpecifierSet(...).contains, [...])``
        because it implements the rule from :pep:`440` that a prerelease item
        SHOULD be accepted if no other versions match the given specifier.
        
        >>> list(SpecifierSet(">=1.2.3").filter(["1.2", "1.3", "1.5a1"]))
        ['1.3']
        >>> list(SpecifierSet(">=1.2.3").filter(["1.2", "1.3", Version("1.4")]))
        ['1.3', <Version('1.4')>]
        >>> list(SpecifierSet(">=1.2.3").filter(["1.2", "1.5a1"]))
        []
        >>> list(SpecifierSet(">=1.2.3").filter(["1.3", "1.5a1"], prereleases=True))
        ['1.3', '1.5a1']
        >>> list(SpecifierSet(">=1.2.3", prereleases=True).filter(["1.3", "1.5a1"]))
        ['1.3', '1.5a1']
        
        An "empty" SpecifierSet will filter items based on the presence of prerelease
        versions in the set.
        
        >>> list(SpecifierSet("").filter(["1.3", "1.5a1"]))
        ['1.3']
        >>> list(SpecifierSet("").filter(["1.5a1"]))
        ['1.5a1']
        >>> list(SpecifierSet("", prereleases=True).filter(["1.3", "1.5a1"]))
        ['1.3', '1.5a1']
        >>> list(SpecifierSet("").filter(["1.3", "1.5a1"], prereleases=True))
        ['1.3', '1.5a1']
        """
        ...

There’s a test in pip to check that prereleases work, but no test that I could find for this specific situation where there are only prereleases and --pre isn’t set.

redbmk on Jan 20, 2023

Minimized further:

$ pip-audit -r <(echo 'sqlmodel==0.0.8')
Traceback (most recent call last):
  File "/Users/william/devel/pip-audit/env/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/Users/william/devel/pip-audit/pip_audit/_cli.py", line 455, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/Users/william/devel/pip-audit/pip_audit/_audit.py", line 67, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/Users/william/devel/pip-audit/pip_audit/_service/interface.py", line 155, in query_all
    for spec in specs:
  File "/Users/william/devel/pip-audit/pip_audit/_dependency_source/requirement.py", line 116, in collect
    for _, dep in self._collect_cached_deps(filename, reqs):
  File "/Users/william/devel/pip-audit/pip_audit/_dependency_source/requirement.py", line 328, in _collect_cached_deps
    for req, resolved_deps in self._resolver.resolve_all(iter(req_values)):
  File "/Users/william/devel/pip-audit/pip_audit/_dependency_source/interface.py", line 88, in resolve_all
    yield (req, self.resolve(req))
  File "/Users/william/devel/pip-audit/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 77, in resolve
    result = self.resolver.resolve([req])
  File "/Users/william/devel/pip-audit/env/lib/python3.7/site-packages/resolvelib/resolvers.py", line 521, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/william/devel/pip-audit/env/lib/python3.7/site-packages/resolvelib/resolvers.py", line 414, in resolve
    raise ResolutionImpossible(self.state.backtrack_causes)
resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=<Requirement('sqlalchemy2-stubs')>, parent=<sqlmodel==0.0.8 wheel=True>)]

woodruffw on Jan 11, 2023