pipenv: PipEnv fails installs with TypeError ('LegacySpecifier' not iterable)

PipEnv fails some installs with an exception, TypeError: 'LegacySpecifier' object is not iterable:

$ pip3 freeze | grep pipenv
pipenv==2020.11.15

Installation should work or show an actual error related to a package. It fails on a TypeError exception without any legible problem report related to the install:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/resolver.py", line 764, in <module>
    main()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/resolver.py", line 760, in main
    dev=parsed.dev)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/resolver.py", line 741, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages, dev)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/resolver.py", line 709, in resolve_packages
    requirements_dir=requirements_dir,
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/resolver.py", line 692, in resolve
    req_dir=requirements_dir
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/utils.py", line 1403, in resolve_deps
    req_dir=req_dir,
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/utils.py", line 1110, in actually_resolve_deps
    resolver.resolve_constraints()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/utils.py", line 862, in resolve_constraints
    marker = marker_from_specifier(candidate.requires_python)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/markers.py", line 719, in marker_from_specifier
    for marker_segment in cleanup_pyspecs(spec):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/markers.py", line 288, in cleanup_pyspecs
    for op_and_version_type, versions in _group_by_op(tuple(specs)):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/markers.py", line 203, in _group_by_op
    specs = [_get_specs(x) for x in list(specs)]
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/markers.py", line 203, in <listcomp>
    specs = [_get_specs(x) for x in list(specs)]
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/markers.py", line 187, in _get_specs
    for spec in set(specset):
TypeError: 'LegacySpecifier' object is not iterable

Verbose log is attached as a file.

I believe this is related to parsing markers, e.g. importlib-metadata>=3.6 ; python_version<='3.7'. However, I have not deep dived very far into the cause.

Reproduce:

rm -r dist *.egg* ; pip3 uninstall exotic ; rm -rf Pipfile* rm -rf ~/.local/share/virtualenvs/somelocalappname*
pipenv --verbose install

You could probably reproduce it by simply installing the EXOTIC product, which is a dependency of somelocalappname:

pipenv install exotic

IT WORKS WITH AN OLDER VERSION OF PIPENV: pip3 uninstall pipenv ; pip3 install pipenv==2018.11.26

someorgs-MacBook-Pro:somelocalappname someuser$ pipenv --support

$ pipenv --support

Pipenv version: '2020.11.15'

Pipenv location: '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pipenv'

Python location: '/Library/Frameworks/Python.framework/Versions/3.7/bin/python3'

Python installations found:

3.7.3: /Library/Frameworks/Python.framework/Versions/3.7/bin/python3
3.7.3: /Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7m
3.7.3: /Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7
3.7.3: /usr/local/bin/python3
3.7.3: /usr/local/bin/python3.7m
3.7.3: /usr/local/bin/python3.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.3',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '19.6.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 19.6.0: Tue Jan 12 22:13:05 PST '
                     '2021; root:xnu-6153.141.16~1/RELEASE_X86_64',
 'python_full_version': '3.7.3',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

TERM_PROGRAM
TERM
SHELL
TESTRAIL_KEY
TMPDIR
TERM_PROGRAM_VERSION
OLDPWD
TERM_SESSION_ID
UID_GID
USER
TERM_SESSION_CLASS_ID
SSH_AUTH_SOCK
BASH_SILENCE_DEPRECATION_WARNING
MAVEN_OPTS
MAVEN_HOME
PATH
PWD
JAVA_HOME
LANG
ARTIFACTORY_PSW
TEST_PSW
PYTHONHOME
XPC_FLAGS
TESTRAIL_USER
XPC_SERVICE_NAME
ARTIFACTORY_USR
M2_HOME
HOME
SHLVL
TEST_USR
PYTHONPATH
LOGNAME
TESTRAIL_PSW
TESTRAIL_USR
_
__CF_USER_TEXT_ENCODING
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_SHIMS_BASE_MODULE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenvspecific environment variables:

Debugspecific environment variables:

PATH: /Library/Java/JavaVirtualMachines/jdk-1.8.0_232.jdk/Contents/Home/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Users/someuser/Library/Python/3.7/bin:/opt/maven/latest/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin
SHELL: /bin/zsh
LANG: en_US.UTF-8
PWD: /proj/survey-ws/source/somelocalappname

Contents of Pipfile (‘/System/Volumes/Data/proj/survey-ws/source/somelocalappname/Pipfile’):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
astropy = ">=4.0.1.post1"
beautifulsoup4 = "~=4.9"
dawgie = "~=1.2"
exotic = "~=0.41.0"
filelock = "~=3.0"
importlib-metadata = ">=3.6"
json2html = "~=1.3"
jdcal = "~=1.4.1"
jplephem = ">=2.14"
lmfit = "~=1.0.1"
matplotlib = ">=3.2.2"
paramiko = ">=2.7.2"
pymc3 = "~=3.9"
pytz = ">=2019.3"
pytesseract = "~=0.2.9"
python-dateutil = "~=2.8.1"
requests = "~=2.23"
scipy = "~=1.4"
skyfield = "~=1.18"
statsmodels = ">=0.12.0"
tenacity = "~=5.1"
LDTk = "~=1.4.1"
TwitterAPI = "~=2.5"

[dev-packages]

[requires]
python_version = "3.7"

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 24 (15 by maintainers)

Most upvoted comments

Agreed that the version quotes provoked an issue in pipenv but it didn’t result in errors with pip. Installation fails were only experienced in pipenv. The Versioning PEPs are a bit amorphous as some indicate quotes and others don’t. Also, the Setup Tools documentation lacks detail for python_requires arguments in setup.py. Anyway, as removing the quotes will only help, the PR will integrated.

Just to clear some misunderstanding, the error manifested with installing the requirements file by itself, and the requirements file doesn’t reference EXOTIC. It does however contain markers, similar to importlib-metadata>=3.6 ; python_version<='3.7'. Installing only the requirements file into pipenv failed with the LegacySpecifier error.

There may have been a component of system configuration cruft confounding this. Now I’ve completely reset my pip and pipenv installations, including removing caches and pulling entirely new distributions. Without updating EXOTIC with the PR, the error is no longer reproducible.

Here’s how I reset the environment:

rm -r dist *.egg* build Pipfile*
pip3 uninstall <app> 
cd <app_dir> && git reset --hard HEAD 
pipenv --clear 
pip uninstall pipenv 
pip cache purge 
rm -rf <path_to_virtual_envs>/<app>-<hash>
pip install pipenv (or pip install <path_to_branch_of_pipenv>)
<now proceed with reinstall>

I think we can call this closed. I’m quite certain a year ago this was a problem with the older release. If any similar errors arise again in the context of a requirements file, I will raise a new issue to cleanly indicate that. Thanks for working on Pipenv which is ultimately a great tool.

jpl-jengelke on Mar 13, 2022

@jpl-jengelke Yes that is the one, and fwiw I am trying it right now, just installed Cython first. Its still running … not sure how long it should take. I’ll edit this message hopefully soon. EDIT: Well I think I reproduced your bug, I was hopeful it would have been fixed as well.

(pipenv-4768) matteius@matteius-VirtualBox:~/shared-projects/pipenv-triage/pipenv-4681$ pipenv install exotic
Installing exotic...
Adding exotic to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock (7ed8a8) out of date, updating to (0df0f7)...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✘ Locking Failed! 

Traceback (most recent call last):
  File "/home/matteius/pipenv/pipenv/resolver.py", line 766, in <module>
    main()
  File "/home/matteius/pipenv/pipenv/resolver.py", line 760, in main
    _main(parsed.pre, parsed.clear, parsed.verbose, parsed.system, parsed.write,
  File "/home/matteius/pipenv/pipenv/resolver.py", line 743, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages, dev)
  File "/home/matteius/pipenv/pipenv/resolver.py", line 704, in resolve_packages
    results, resolver = resolve(
  File "/home/matteius/pipenv/pipenv/resolver.py", line 685, in resolve
    return resolve_deps(
  File "/home/matteius/pipenv/pipenv/utils.py", line 1376, in resolve_deps
    results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
  File "/home/matteius/pipenv/pipenv/utils.py", line 1107, in actually_resolve_deps
    resolver.resolve_constraints()
  File "/home/matteius/pipenv/pipenv/utils.py", line 900, in resolve_constraints
    marker = marker_from_specifier(requires_python)
  File "/home/matteius/pipenv/pipenv/vendor/requirementslib/models/markers.py", line 714, in marker_from_specifier
    for marker_segment in cleanup_pyspecs(spec):
  File "/home/matteius/pipenv/pipenv/vendor/requirementslib/models/markers.py", line 286, in cleanup_pyspecs
    for op_and_version_type, versions in _group_by_op(tuple(specs)):
  File "/home/matteius/pipenv/pipenv/vendor/requirementslib/models/markers.py", line 200, in _group_by_op
    specs = [_get_specs(x) for x in list(specs)]
  File "/home/matteius/pipenv/pipenv/vendor/requirementslib/models/markers.py", line 200, in <listcomp>
    specs = [_get_specs(x) for x in list(specs)]
  File "/home/matteius/pipenv/pipenv/vendor/requirementslib/models/markers.py", line 184, in _get_specs
    for spec in set(specset):
TypeError: 'LegacySpecifier' object is not iterable

matteius on Mar 12, 2022

Just to confirm, I retested just today with 2022.1.8. which is the log in my last post. I’ll try to scrape some time together to test with the new branch.

@matteius Do you mean the branch named vendor-pip-22.0.3-followup-changes?

jpl-jengelke on Mar 12, 2022