pipenv: Pipenv lock fails for packages with extras as of 2022.8.13
Issue description
There was a regression in the ability to lock Pipfiles containing packages with extras on the 2022.8.13 release. This was working fine on versions prior to 2022.8.13. I believe this is as a result of the changes introduced in #5234
Expected result
Given the Pipfile in the replicate section, it should lock, showing
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Updated Pipfile.lock (3e8e27)!
Actual result
[ResolutionFailure]: File "***.venv/lib/python3.10/site-packages/pipenv/resolver.py", line 824, in _main
[ResolutionFailure]: resolve_packages(
[ResolutionFailure]: File "***.venv/lib/python3.10/site-packages/pipenv/resolver.py", line 772, in resolve_packages
[ResolutionFailure]: results, resolver = resolve(
[ResolutionFailure]: File "***.venv/lib/python3.10/site-packages/pipenv/resolver.py", line 751, in resolve
[ResolutionFailure]: return resolve_deps(
[ResolutionFailure]: File "***.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 1126, in resolve_deps
[ResolutionFailure]: results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
[ResolutionFailure]: File "***.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 919, in actually_resolve_deps
[ResolutionFailure]: resolver.resolve()
[ResolutionFailure]: File "***.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 723, in resolve
[ResolutionFailure]: raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: Constraints cannot have extras
Steps to replicate
With the following Pipfile
[packages]
"uvicorn[standard]" = "~=0.18.2"
[dev-packages]
pytest = "~=7.1.1"
[requires]
python_version = "3.9"
Run pipenv lock
Note that there is a package with ‘extras’, namely uvicorn[standard]. This issue does not occur with normal packages.
The change to lock the packages first and then treat that lock as constraints for the dev packages seems to raise the exception that the extras are not allowed.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 2
- Comments: 33 (22 by maintainers)
Or maybe I’m wrong,
PipDeprecationWarning
says Constraints are only allowed to take the form of a package name and a version specifier so it should not containts url. In that case we could easily fix that by usingnew_dep.is_named
.But the documentation does not say that. So maybe it could have url as constraints? I tried using this constraints file with
pip
and it did work.@dqkqd I found this interesting and has a lot of context: https://github.com/pypa/pip/issues/8210
That was a long read, but the conclusion is that “unnamed constraints are not allowed.”
I think it’s the bug in
requirementlib
. Actually the result fromnew_dep.as_line()
at line 284. https://github.com/pypa/pipenv/blob/f04071c52f09015758cd3eab68fae0542c4325aa/pipenv/utils/dependencies.py#L274-L286For example: if I passed in the packages
django-redis = {file = "https://github.com/antoniomika/django-redis/archive/4.12.2.zip"}
the output I expected isdjango-redis @ https://github.com/antoniomika/django-redis/archive/4.12.2.zip
as mentioned in https://pip.pypa.io/en/stable/reference/requirements-file-format/#exampleBut the actual output is
https://github.com/antoniomika/django-redis/archive/4.12.2.zip
, and it doesn’t contain name anymore, it’s name isNone
now.The name is disappeared because:
as_line
callself.line_instance.get_line
https://github.com/pypa/pipenv/blob/f04071c52f09015758cd3eab68fae0542c4325aa/pipenv/vendor/requirementslib/models/requirements.py#L2801-L2805self.line_instance.get_line
check if package should have a name or not: https://github.com/pypa/pipenv/blob/f04071c52f09015758cd3eab68fae0542c4325aa/pipenv/vendor/requirementslib/models/requirements.py#L219-L220And because it has url, so it’s name is discarded from now. https://github.com/pypa/pipenv/blob/f04071c52f09015758cd3eab68fae0542c4325aa/pipenv/vendor/requirementslib/models/requirements.py#L753-L760
@dqkqd I see what you mean now, we are using that method to filter them out already, but thats causing the emitting of a deprecation warning because that method seems to expect that data was already cleaned – probably we should just have our own method for this like you were suggesting, be it a similar copy or a simplified refactor.
That begs the question of what is causing resolution failures for @antoniomika – can you elaborate on the file configuration setting you are using? Do you just mean local editable installs, or something else?
I don’t think we need to rewrite the pip function, I think its actually warning us of valid errors. I think the issue is that we need to filter out certain requirements from the
Pipfile
if they are constraints, because it ends up confusing the resolver if they get included. Probably for the extras we can do something like: