pip: After pip 10 upgrade on pyenv "ImportError: cannot import name 'main'"

Fix for us was pinning to pip 9.03, so:

pip install --upgrade pip==9.0.3

instead of

pip install -U pip

obvious fix but in case it helps somebody else!

I upgraded with pyenv, both python2 and python3, now pip2 not work and pip3 works After comparing pip2 and pip3, the difference is the import line

pip2 from pip import main

pip3 from pip._internal import main

After substitute pip2 import line with pip3 version, it works

Simply restarting my terminal fixed it for me.

Watch out for bash caching the executable location:

$ which pip
/usr/bin/pip

$ pip install --user pip
Collecting pip
(...)
Successfully installed pip-10.0.0

$ pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

$ which pip
/usr/bin/pip

$ hash -d pip  # this clears the 'pip' entry from bash's executables locations hash table

$ which pip
/home/zwinny/.local/bin/pip

$ pip --version
pip 10.0.0 from /home/zwinny/.local/lib/python2.7/site-packages/pip (python 2.7)

While I appreciate that the underlying issue reported here and in https://github.com/pypa/pip/issues/5221 is the environment, the cause is primarily that the import from pip import main was broken as the package pip.main was moved to pip._internal.main. It would be trivial to add a link from pip.main to pip._internal.main to fix this (whereas fixing the environment is a lot of work across many locations for many people). Is there a good reason to not do this?

@davidjlloyd Because from pip import main was never supported, basically. And while it’s easy to say “yes, but it’s a simple API and it just worked”, it really didn’t - we’ve had multiple bug reports from people expecting certain behaviours from it, that we simply don’t cater for (running pip.main in multiple threads, expecting pip.main to not change the configuration of the logging system, …)

Rather than keep explaining to people that they shouldn’t do this, and continually dealing with the fact that people are assuming “if I can find a function to call, it’s supported” we moved everything to the _internal namespace to make it abundantly clear that you’re not supposed to call it.

The bulk of the complaints have come from people using pip.main - which is ironic, as it’s so easy to call pip in the supported command line way via subprocess. So of all the possible breakages, this is the easiest to fix - and yet people still haven’t fixed it, even after months of warnings. (Although to be fair to the Linux distributions, they don’t support people upgrading their system packages using pip, so reports like #5221 of cases where distro-supplied scripts break is fundamentally user error, not failure of the distros to address the pip 10 changes - something I’m sure they are handling perfectly well).

Temporary Fix upgrade pip using.

curl https://bootstrap.pypa.io/get-pip.py | python3

Instead of pip install -U pip

For pip2 pip2 install --upgrade pip

python -m pip install --upgrade pip 

Worked for me. I hope that helps someone.

Rather than keep explaining to people that they shouldn’t do this, and continually dealing with the fact that people are assuming “if I can find a function to call, it’s supported” we moved everything to the _internal namespace to make it abundantly clear that you’re not supposed to call it.

I’m not sure that aggressively breaking existing uses of an unsupported feature instead of deprecating the feature for a couple of major releases is the best approach. Our initial reaction was to pin pip back to 9.0.3, and more lazy developers would probably just call it a day at that point. This would leave a lot of users stubbornly clinging to old releases, which I doubt anyone wants. However your motivation makes sense, and the eventual result is the same.

For any users hitting this issue, I think the nicest solution for replacing system or pyenv installations was kindly provided by @standag here: https://github.com/pypa/pip/issues/5221#issuecomment-381568428

This solution should work for anyone building in Docker such as @peteflorence without pinning to stale releases or anything horrible like that.

We are encountering the same trouble.

// in host
$ docker pull ubuntu:xenial
$ docker run --name pip-test --rm -it ubuntu:xenial bash

// in container
# apt update
# apt install -y python-dev python-pip
# pip install --upgrade pip
Collecting pip
  Downloading pip-10.0.0-py2.py3-none-any.whl (1.3MB)
    100% |################################| 1.3MB 865kB/s 
Installing collected packages: pip
  Found existing installation: pip 8.1.1
    Not uninstalling pip at /usr/lib/python2.7/dist-packages, outside environment /usr
Successfully installed pip-10.0.0
# pip install requests
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

I just unrolled my pip installation. python -m pip install --user --upgrade pip==9.0.3

I’ve created a new Raspbian install on a Raspberry Pi 3B+. Nothing special - pretty vanilla configuration - and so far, everything has gone fine.

I’ve just reached the very last step in my install process:

pip install --upgrade pip

Collecting pip
  Downloading https://files.pythonhosted.org/packages/62/a1/0d452b6901b0157a0134fd27ba89bf95a857fbda64ba52e1ca2cf61d8412/pip-10.0.0-py2.py3-none-any.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 101kB/s 
Installing collected packages: pip
Successfully installed pip-10.0.0

…and now pip is completely borked:

$ pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

As per the comment above, I’ve confirmed that ~i/.local/bin/pip runs fine. The error occurs with /usr/bin/pip. Running hash -d pip did not update the cached location, though.

Force-downgrading to 9.0.1 (via options line -Iv and --force-reinstall) also did not resolve the issue. Apparently, the only solution, besides not upgrading pip, is to run pip from ~/.local/bin/pip.

@p00j4 Importing pip from within your program is not supported - see the docs

All those issues seem to stem from the same problem: upgrading pip, but keeping on using an old launcher that still use the old entrypoint. A good way to avoid this kind of issue is to use python -m pip.

Even after following everything I am not able to downgrade pip. Keep getting error. Here’s how i fixed this:

$ sudo apt-get remove python-pip $ pip -v bash: /usr/bin/pip: No such file or directory

but when i try this $ python -m pip -V pip 10.0.0 from /home/user/.local/lib/python2.7/site-packages/pip (python 2.7)

so i removed complete folder of pip $ sudo rm -r /home/user/.local/lib/python2.7/site-packages/pip*

then again i try this $ python -m pip -V /usr/bin/pip: No module named pip

then install pip again $ sudo apt install python-pip $ python -m pip -V pip 8.1.1 from usr/lib/python2.7/dist-packages (python 2.7)

Hope this solve’s the problem

I installed a fresh Ubuntu distro today and tried to get some basic Python packages up and running. I was prompted to upgrade pip, and so I ran the command it told me to. This upgraded pip to version 10, which is apparently broken with the same error at the top of this thread. I haven’t done anything other than what a normal user would do to upgrade pip and install their favourite packages. Not even using pyenv.

None of the solutions here fix my problem. If I run python -m pip install --upgrade pip I get “Requirement already up-to-date”. If I try to downgrade to version 9 I get the same initial error:

Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

After re-reading this thread, @sfsdfd’s solution to use ~/.local/bin/pip is what finally worked:

~/.local/bin/pip install my-favourite-package

Apparently this is working by (successfully) reverting back to my older version of pip. Adding this to ~/.bashrc is a nicer fix for me:

export PATH=$PATH:~/.local/bin

I’m not sure if I saw an answer / response to @davidjlloyd’s comment:

It’s been answered repeatedly, Maybe not on this specific issue, but a search of the issue tracker should find plenty of discussion on the matter.

Can I please ask why there was not a deprecation process in place for this?

Because it was never supported. Why would we warn that we’re desupporting something we never supported? People assumed it was OK to read pip’s source code and use functions they found in there in their own code. It never was. We said it could break, and in pip 10 it did.

It seems like it would have been quite easy, on the import of pip, check sys.argv[0]; if it’s not pip or pipX[.Y], spew some DeprecationWarnings

I’m not sure it’s as easy as you think. And I say that from the perspective of someone who has had to deal with issues where warnings added in pip 10 were being triggered when we hadn’t expected them to be…

And again, there’s no need to deprecate something that’s not supported in the first place.

Is there a process in place to try and ensure this kind of thing is hopefully avoided in the future?

What sort of thing? Breakage caused by us changing things that we don’t guarantee backward compatibility for? No. There’s no need for us to avoid that. Although in fact, we do try to manage the process, as a courtesy to our users (not an obligation!). In this case, we publicised the change 6 months in advance, offered suggestions for people who needed to change their code, and have been spending a lot of time since the release helping users who have had problems because software they rely on hasn’t heeded those warnings. That’s a lot of work that a very small volunteer group put into trying to mitigate a situation caused by people expecting support that was never offered or promised. You’re welcome.

We do have processes (deprecation warnings, etc) in place for changing things that we do guarantee compatibility for - but importing pip into your own program isn’t one of them.

The recommended approach is to remove the dependence of your library/application/tool from pip’s internal implementation details. easy_install is no longer being actively improved AFAIK and likely doesn’t support a lot of the things that pip does.

I’m guessing you didn’t read the rest of his post. He was referring to using easy_install to install pip, not instead of pip:

Switching to easy_install -U pip worked…

Also he is not relying on pip’s internal implementation details. He’s using a fresh spinup of AppVeyor and upgrading pip using the command he specified, and nothing more:

pip install --disable-pip-version-check --user --upgrade pip

Many other people in this thread are having the same problem. You can reproduce this with a basic installation of pip without doing anything special or dependent on pip’s internals. This has been documented by many people in the posts above.

I wanted to report that this issue also happens on Windows build when using AppVeyor for CI: The result is the following:

Running Install scripts
SET PATH=%PYTHON%;%PYTHON%\Scripts;%PATH%
pip install --disable-pip-version-check --user --upgrade pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl (1.3MB)
Installing collected packages: pip
Successfully installed pip-10.0.1
pip install -U setuptools
Traceback (most recent call last):
  File "c:\python27-x64\lib\runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "c:\python27-x64\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python27-x64\Scripts\pip.exe\__main__.py", line 5, in <module>
ImportError: cannot import name main
Command exited with code 1

I have: pip install --disable-pip-version-check --user --upgrade pip in several appveyor.yml scripts, based on the recommendation from this sample: https://github.com/ogrisel/python-appveyor-demo/blob/master/appveyor.yml#L111

This has worked well until pip 10.x, and I suspect if others have created appveyor.yml based on this will also run into this issue.

Switching to easy_install -U pip worked, but I do have several repos that worked before that now need to be updated to work with pip 10.x.

Seems like the recommended approach for CI setup should be sticking with 9.0.3 or using easy_install.

I’ve fixed the issue by update the code in /usr/bin/pip, change from pip import main to from pip._internal import main.

Here details:

$ dpkg -S /usr/bin/pip
python-pip: /usr/bin/pip
$ dpkg -S /usr/bin/pip2
python-pip: /usr/bin/pip2
$ apt-file search /usr/bin/pip
colorized-logs: /usr/bin/pipetty
pipebench: /usr/bin/pipebench
pipemeter: /usr/bin/pipemeter
pipexec: /usr/bin/pipexec
python-pip: /usr/bin/pip
python-pip: /usr/bin/pip2
python3-pip: /usr/bin/pip3
rt-tests: /usr/bin/pip_stress

After pip install --upgrade pip:

$ pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main
$ pip2
Traceback (most recent call last):
  File "/usr/bin/pip2", line 9, in <module>
    from pip import main
ImportError: cannot import name main

$ cat /usr/bin/pip
#!/usr/bin/python
# GENERATED BY DEBIAN

import sys

# Run the main entry point, similarly to how setuptools does it, but because
# we didn't install the actual entry point from setup.py, don't use the
# pkg_resources API.
from pip import main
if __name__ == '__main__':
    sys.exit(main())

After updated the /usr/bin/pip:

$ cat /usr/bin/pip
#!/usr/bin/python
# GENERATED BY DEBIAN

import sys

# Run the main entry point, similarly to how setuptools does it, but because
# we didn't install the actual entry point from setup.py, don't use the
# pkg_resources API.
# from pip import main
from pip._internal import main
if __name__ == '__main__':
    sys.exit(main())

$ pip --version
pip 10.0.0 from /home/devops/.local/lib/python2.7/site-packages/pip (python 2.7)

$ pip2
Traceback (most recent call last):
  File "/usr/bin/pip2", line 9, in <module>
    from pip import main
ImportError: cannot import name main

My system information:

$ uname -a
Linux devops-kubernetes-master 4.13.0-38-generic #43-Ubuntu SMP Wed Mar 14 15:20:44 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/*-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=17.10
DISTRIB_CODENAME=artful
DISTRIB_DESCRIPTION="Ubuntu 17.10"
NAME="Ubuntu"
VERSION="17.10 (Artful Aardvark)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 17.10"
VERSION_ID="17.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=artful
UBUNTU_CODENAME=artful

$ python --version
Python 2.7.14

$ apt list --installed | grep python-

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libpython-all-dev/artful,now 2.7.14-2ubuntu1 amd64 [installed]
libpython-dev/artful,now 2.7.14-2ubuntu1 amd64 [installed,automatic]
libpython-stdlib/artful,now 2.7.14-2ubuntu1 amd64 [installed,automatic]
python-all/artful,now 2.7.14-2ubuntu1 amd64 [installed,automatic]
python-all-dev/artful,now 2.7.14-2ubuntu1 amd64 [installed,automatic]
python-apt-common/artful,artful,now 1.4.0~beta3build2 all [installed]
python-asn1crypto/artful,artful,now 0.22.0-1 all [installed,automatic]
python-cffi-backend/artful,now 1.9.1-2build2 amd64 [installed,automatic]
python-crypto/artful-updates,artful-security,now 2.6.1-7ubuntu0.1 amd64 [installed,automatic]
python-cryptography/artful,now 1.9-1 amd64 [installed,automatic]
python-dbus/artful,now 1.2.4-1build3 amd64 [installed,automatic]
python-dev/artful,now 2.7.14-2ubuntu1 amd64 [installed,automatic]
python-enum34/artful,artful,now 1.1.6-1 all [installed,automatic]
python-gi/artful,now 3.24.1-2build1 amd64 [installed,automatic]
python-idna/artful,artful,now 2.5-1 all [installed,automatic]
python-ipaddress/artful,artful,now 1.0.17-1 all [installed,automatic]
python-keyring/artful,artful,now 10.4.0-1 all [installed,automatic]
python-keyrings.alt/artful,artful,now 2.2-2 all [installed,automatic]
python-minimal/artful,now 2.7.14-2ubuntu1 amd64 [installed,automatic]
python-pip/artful,artful,now 9.0.1-2 all [installed]
python-pip-whl/artful,artful,now 9.0.1-2 all [installed,automatic]
python-pkg-resources/artful,artful,now 36.2.7-2 all [installed,automatic]
python-secretstorage/artful,artful,now 2.3.1-2 all [installed,automatic]
python-setuptools/artful,artful,now 36.2.7-2 all [installed,automatic]
python-setuptools-doc/artful,artful,now 36.2.7-2 all [installed]
python-six/artful,artful,now 1.10.0-4 all [installed,automatic]
python-talloc/artful,now 2.1.9-2ubuntu1 amd64 [installed]
python-wheel/artful,artful,now 0.29.0-2 all [installed,automatic]
python-xdg/artful,artful,now 0.25-4 all [installed,automatic]

Just to mention that pip install --user --upgrade pip on Ubuntu 16.04 also breaks.

Same here … Exact same output as @HayaoSuzuki and we don’t use pyenv

All those issues seem to stem from the same problem: upgrading pip, but keeping on using an old launcher that still use the old entrypoint. A good way to avoid this kind of issue is to use python -m pip.

Yep, I think that seems to be the root of the issue, and happens across all different platforms mentioned in this thread, including Windows.

In case this helps anyone else, I can confirm that switching appveyor.yml pip upgrade from:

pip install --disable-pip-version-check --user --upgrade pip

to:

python -m pip install --upgrade pip

does fix the issue. Now to update several more repos!

@arvoelke:

He was referring to using easy_install to install pip, not instead of pip

Installing pip using easy_install could well have issues, and if it does, we won’t be able to support you, because (as @pradyunsg said) easy_install is old, not actively maintained, and missing recent features. But if it works for you, and you don’t need us to help, then fine - no-one is stopping you.

Many other people in this thread are having the same problem

Define “the same”. If you mean "seeing errors from code that imports pip.main then yes, but that’s not a problem with pip, it’s a deliberate, backward incompatible, change to the internal implementation of pip. If you want us to just say “tough, you shouldn’t be using the internal APIs of pip”, then fine - a single issue is enough, and we’ll just close it as “not a bug”. But if you want us to help you to work out what part of your environment has failed to follow the advice published 6 months ago, and find a workaround for you that lets you carry on using pip while your environment gets fixed by the vendor, then you’ll have to help us to help you. And saying “I have the same problem” on a report about a purely Unix utility, when you’re running on Windows, is definitely not helping us to help you…

AFTER WASTING 3 HOURS THIS IS WHAT I GOT

TO UNINSTALL pip 10.0.0: sudo apt-get remove python-pip

but then u wont be able to install the correct required version of pip by the direct methods.

TO DOWNGRADE TO A KNOWN WORKING VERSION AFTER INSTALLING when pip10 throws the error: ImportError: cannot import name main you will have to run: python -m pip install pip==KNOW_WORKING_VERSION>

TO INSTALL ANY PACKAGE USING PIP10: sudo python -m pip install PACKAGE_NAME this worked.

Cheers 😃

pip 10.0, I also face

AttributeError: 'module' object has no attribute 'main'

while trying to use pip.main(['install', '-r', 'requirements.txt'])

try to use pip2 install xx @HayaoSuzuki