pip: Unable to install anything using pip after upgrade to 20.0

Environment

  • pip version: 20.0
  • Python version: 3.6.8
  • OS: Ubuntu 14.04.6 (Don’t judge)

Description

During our build process, we automatically upgrade to the newest version on pip. After upgrading to 20.0 (using pip install -U pip setuptools), we are unable to install any packages.

Output

Traceback (most recent call last):
  File "/home/svc-web/webshop/env/bin/pip", line 8, in <module>
    sys.exit(main())
  File "/home/svc-web/webshop/env/lib/python3.6/site-packages/pip/_internal/cli/main.py", line 73, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
  File "/home/svc-web/webshop/env/lib/python3.6/site-packages/pip/_internal/commands/__init__.py", line 96, in create_command
    module = importlib.import_module(module_path)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/svc-web/webshop/env/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 24, in <module>
    from pip._internal.cli.req_command import RequirementCommand
  File "/home/svc-web/webshop/env/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 20, in <module>
    from pip._internal.operations.prepare import RequirementPreparer
  File "/home/svc-web/webshop/env/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 16, in <module>
    from pip._internal.distributions import (
  File "/home/svc-web/webshop/env/lib/python3.6/site-packages/pip/_internal/distributions/__init__.py", line 1, in <module>
    from pip._internal.distributions.source import SourceDistribution
ImportError: cannot import name 'SourceDistribution'

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 196
  • Comments: 103 (20 by maintainers)

Commits related to this issue

Most upvoted comments

It’s worth mentioning that mistakes, problems and unforeseen mishaps happen to all projects, no matter how much you plan and try to mitigate such events.

Spare a thought and supportive comments for the volunteer Pythonistas looking into this and fixing it for all our benefit.

Do you not listen to your own CI tests?

Always remember that the maintainers are also humans, and programming mistakes are actually common among this species. We should be providing any support they or the community need than blaming people.

In the meanwhile this can be helpful if you use virtualenv:

virtualenv venv --no-pip
source venv/bin/activate
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py pip==19.3.1

A workaround to get pip working again is getting the get-pip.py script and running: python get-pip.py pip==19.3.1 This will revert pip to the previous version

For those people here with broken build processes because they do something similar to OP’s pip install -U pip setuptools - remember you can tweak this to ban or constrain any particular version.

Both of these seem to work OK for me locally:

$ pip install -U "pip<20" setuptools
$ pip install -U "pip!=20.0" setuptools

pip will warn you about not being on the latest version, but won’t install v20.0.

Alrighty, looks like this has been fixed. Closing.

Well, we figured out what caused this and are making the release process changes to fix that: https://github.com/pypa/pip/pull/7624

Out of sheer curiosity, why was this version released even though the last pipelines failed showing this error? https://travis-ci.org/pypa/pip/builds/639902383

Not a cutting remark at all, just interested in the release process and how can this kind of issues can be avoided in the future!

Many thanks for the ongoing work to fix it 👍

I see the same thing:

$ pip install requests
Traceback (most recent call last):
  File "/home/ntoll/.virtualenvs/sdclient/bin/pip", line 8, in <module>
    sys.exit(main())
  File "/home/ntoll/.virtualenvs/sdclient/lib/python3.7/site-packages/pip/_internal/cli/main.py", line 73, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
  File "/home/ntoll/.virtualenvs/sdclient/lib/python3.7/site-packages/pip/_internal/commands/__init__.py", line 96, in create_command
    module = importlib.import_module(module_path)
  File "/home/ntoll/.virtualenvs/sdclient/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/ntoll/.virtualenvs/sdclient/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 24, in <module>
    from pip._internal.cli.req_command import RequirementCommand
  File "/home/ntoll/.virtualenvs/sdclient/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 20, in <module>
    from pip._internal.operations.prepare import RequirementPreparer
  File "/home/ntoll/.virtualenvs/sdclient/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 16, in <module>
    from pip._internal.distributions import (
  File "/home/ntoll/.virtualenvs/sdclient/lib/python3.7/site-packages/pip/_internal/distributions/__init__.py", line 1, in <module>
    from pip._internal.distributions.source import SourceDistribution
ImportError: cannot import name 'SourceDistribution' from 'pip._internal.distributions.source' (/home/ntoll/.virtualenvs/sdclient/lib/python3.7/site-packages/pip/_internal/distributions/source/__init__.py)

Python 3.7.2 on Ubuntu 18.04

I can confirm 20.0.1 fixed the issue.

Btw, looks like it’s just an missing import. My fix was:

pip\_internal\distributions\source\__init__.py

from .legacy import SourceDistribution

Exactly the same with Python 3.7.5 on Fedora 29. The same happens both in virtualenvs and with --user. Also, doesn’t matter if I run pip3.7 or python3.7 -m pip.

And as @ntoll suggests below, kudos for everyone working to fix this unfortunate glitch! Nice to see only neutral and supportive comments here. Open Source community at its best.

No longer seeing issue with pip 20.0.1 release, thanks. Agree, it would be helpful to understand job failures above in build logs @lerela referenced.

Out of sheer curiosity, why was this version released even though the last pipelines failed showing this error? https://travis-ci.org/pypa/pip/builds/639902383

Not a cutting remark at all, just interested in the release process and how can this kind of issues can be avoided in the future!

Many thanks for the ongoing work to fix it 👍

@lerela @pabelanger that’s not what happened here: that build log is was run after the release was done (that commit was run during release by a script) as evidenced by those failed build logs (for example this one):

Using cached https://files.pythonhosted.org/packages/60/65/16487a7c4e0f95bb3fc89c2e377be331fd496b7a9b08fd3077de7f3ae2cf/pip-20.0-py2.py3-none-any.whl

That this would be the case makes sense: this change was merged some time ago and did not immediately cause failures because it is not a source code bug, an sich. For example, it does not occur when upgrading from an ‘old enough’ pip (9.0.1 is what I used). It is instead caused by artefacts being left behind when upgrading (the underlying bug of which is unclear at this time, at least to me).

This issue was not caught by the pipeline, and so further improvements to test upgrade paths in the pipeline there would be welcome @lerela @pabelanger and contributing to pip is easy!

Just wanted to chime in that our build system went down with this too. Luckily its not critical, and I agree with @ntoll that sometimes things goes south, and I’m sure you’re doing all you can to fix this.

Now that PyPI has support for yanked releases, I’ve gone ahead and marked pip 20.0 as “yanked”. pip > 19.2 would not install it by default and would print a warning, when installing it via pip install pip==20:

WARNING: The candidate selected for download or install is a yanked version: 'pip' candidate (version 20.0 at https://files.pythonhosted.org/packages/60/65/16487a7c4e0f95bb3fc89c2e377be331fd496b7a9b08fd3077de7f3ae2cf/pip-20.0-py2.py3-none-any.whl#sha256=eea07b449d969dbc8c062c157852cf8ed2ad1b8b5ac965a6b819e62929e41703 (from https://pypi.org/simple/pip/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*))

Probably related https://github.com/pypa/pipenv/issues/4128. Another fix I found working for me is

pip install -U pipenv virtualenv
virtualenv .venv --seeder pip
PIPENV_VENV_IN_PROJECT=1 pipenv install

@fideloper Thanks for asking! I answered this on a different issue just now: https://github.com/pypa/pip/issues/7217#issuecomment-576698030

@honnix’s suggestion also works, although I would recommend against using curl ... | python.

Thank you for fixing the issue so fast @pradyunsg .

A possible local workaround: https://github.com/pypa/pip/issues/7217#issuecomment-576653463

Removing folder ./site-packages/pip/_internal/distributions/source helps!

@james-gonzalez Not sure cutting off a new release is part of CI.

Reading https://pip.pypa.io/en/latest/development/release-process/#creating-a-new-release, maybe after checking out master a git clean should be applied.

A workaround to get pip working again is getting the get-pip.py script and running: python get-pip.py pip==19.3.1 This will revert pip to the previous version

Thanks bro… this workaround saves my archlinux distro =)

I’ve been getting the same issue for the last couple of weeks. I’m running MacOS 10.5.3 and I’ve installed Python 3.7.6 via Homebrew. I just upgraded Pip on one of my virtual environments today from version 19.x to version 20.0 and it has broken my virtual environment. I’m getting the following error:

Traceback (most recent call last): File "/Users/jeff/Envs/test/bin/pip3", line 5, in <module> from pip._internal.cli.main import main ModuleNotFoundError: No module named 'pip._internal.cli.main'

@Serialc0de See https://github.com/pypa/pip/issues/5599. Your /usr/bin/pip and your pip installation are out of sync. (And you shouldn’t be using sudo pip

@george542 you have a different pip installed normally while you also installed the newer pip in a broken way with setuptools, please clean out all pip installs and pip egg installs, then install pip using the get-pip script which deals a correct setup

I know that all the people working on this are volunteers and I am really glad for all the work they do!

But as @lerela mentioned we should investigate a bit how such thing could happen and how can we avoid such mistake.

Pip is one of the core and most used tools in the whole software industry. And it looks like that a new version was released even though it was not at least tested to run basic commands.

Do we check these builds before release? Is a release process somehow managed and reviewed or is it a one-man show?

I would understand this situation if such mistake happens in some urgent hurried hotfix release, but version 20 was released several months after 19.3.1.

I had recently upgraded to pip 20.0 using “python -m pip install --upgrade pip”. Following the upgrade, even “pip list” command broke. Now I see a new version 20.0.1 is released, but if I do “python -m pip install --upgrade pip” then I get the same error. How can I upgrade to latest pip version 20.0.1?

https://github.com/pypa/pip/issues/7217 Install an older version: curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py python get-pip.py pip==19.3.1

@xdralex Yes 👍 I was just typing something similar.

Plus, if you’ve got a usable pip in your global env, then virtualenv venv --no-download will use that one instead of downloading the broken version.

@AvAkanksh, using Google I found these probably much better places to look for hints to solve your specific issue:

If you find a solution, a most helpful thing to do would be to link to it in comments of all the above pages.

Good luck, hope you find the solution!

A workaround to get pip working again is getting the get-pip.py script and running: python get-pip.py pip==19.3.1 This will revert pip to the previous version

Thanks! this worked for me (python 3.7.4 running on cygwin64)

Hello to all, My operating system is Arch-Linux. I followed the documentation to install wfuzz, unfortunately I get this error again. How could I solve it? Thank you in advance.

sudo pip install wfuzz Traceback (most recent call last): File "/usr/bin/pip", line 11, in <module> load_entry_point('pip==19.3', 'console_scripts', 'pip')() File "/usr/lib/python3.8/site-packages/pip/_internal/main.py", line 45, in main command = create_command(cmd_name, isolated=("--isolated" in cmd_args)) File "/usr/lib/python3.8/site-packages/pip/_internal/commands/__init__.py", line 96, in create_command module = importlib.import_module(module_path) File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 783, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/usr/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 24, in <module> from pip._internal.cli.req_command import RequirementCommand File "/usr/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 17, in <module> from pip._internal.index import PackageFinder ImportError: cannot import name 'PackageFinder' from 'pip._internal.index' (/usr/lib/python3.8/site-packages/pip/_internal/index/__init__.py)

Hmm, it just updates pip at the start by default for me. But I think wiping the build environment could trigger it.

That worked! Thank you!

@pradyunsg @jamadden Working on putting together a minimal Dockerfile that reproduces the error; will file a new issue once I manage to. (We have several layers of internal base images I have to dig through to find something to put in FROM that’s publicly available…)

Comparable error for CentOS 7 with Python3.6:

$ cat /etc/redhat-release 
CentOS Linux release 7.7.1908 (Core)

$ mkvirtualenv --python=/usr/bin/python3.6 foo

(foo) $ which python
~/.virtualenvs/foo/bin/python


(foo) $ which pip
~/.virtualenvs/foo/bin/pip

(foo) $ pip --version
pip 20.0 from /home/foo/.virtualenvs/foo/lib/python3.6/site-packages/pip (python 3.6)


(foo) $ pip list
Traceback (most recent call last):
  File "/home/foo/.virtualenvs/foo/bin/pip", line 10, in <module>
    sys.exit(main())
  File "/home/foo/.virtualenvs/foo/lib/python3.6/site-packages/pip/_internal/cli/main.py", line 73, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
  File "/home/foo/.virtualenvs/foo/lib/python3.6/site-packages/pip/_internal/commands/__init__.py", line 96, in create_command
    module = importlib.import_module(module_path)
  File "/home/foo/.virtualenvs/foo/lib64/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/foo/.virtualenvs/foo/lib/python3.6/site-packages/pip/_internal/commands/list.py", line 13, in <module>
    from pip._internal.cli.req_command import IndexGroupCommand
  File "/home/foo/.virtualenvs/foo/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 20, in <module>
    from pip._internal.operations.prepare import RequirementPreparer
  File "/home/foo/.virtualenvs/foo/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 16, in <module>
    from pip._internal.distributions import (
  File "/home/foo/.virtualenvs/foo/lib/python3.6/site-packages/pip/_internal/distributions/__init__.py", line 1, in <module>
    from pip._internal.distributions.source import SourceDistribution
ImportError: cannot import name 'SourceDistribution'

@pradyunsg I confirm the issue. I suggest pulling it from PyPI.