pip: wonky pth file breaks upgrade

nstalling collected packages: pbr, pip
  Found existing installation: pbr 0.10.0
Cleaning up...
  Removing temporary dir /tmp/pip_build_root...
Cannot remove entries from nonexistent file /home/robertc/work/openstack/oslo.config/easy-install.pth
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/local/lib/python2.7/dist-packages/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/usr/local/lib/python2.7/dist-packages/pip/req.py", line 555, in uninstall
    paths_to_remove.add_pth(easy_install_pth, './' + easy_install_egg)
  File "/usr/local/lib/python2.7/dist-packages/pip/req.py", line 1785, in add_pth
    self.pth[pth_file] = UninstallPthEntries(pth_file)
  File "/usr/local/lib/python2.7/dist-packages/pip/req.py", line 1868, in __init__
    raise UninstallationError("Cannot remove entries from nonexistent file %s" % pth_file)
UninstallationError: Cannot remove entries from nonexistent file /home/robertc/work/openstack/oslo.config/easy-install.pth

About this issue

  • Original URL
  • State: closed
  • Created 9 years ago
  • Comments: 24 (7 by maintainers)

Commits related to this issue

Most upvoted comments

I tried filing this as an anaconda bug, but @ilanschnell closed it as a setuptools/pip issue. I’m including a summary of the discussion here:

Setup

D:\workspace>conda info
Current conda install:

             platform : win-32
        conda version : 3.18.5
  conda-build version : 1.14.1
       python version : 2.7.10.final.0
     requests version : 2.8.1
     root environment : D:\anaconda32  (writable)
  default environment : D:\anaconda32
     envs directories : D:\anaconda32\envs
        package cache : D:\anaconda32\pkgs
         channel URLs : https://repo.continuum.io/pkgs/free/win-32/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/win-32/
                        https://repo.continuum.io/pkgs/pro/noarch/
          config file : None
    is foreign system : False

D:\workspace>conda create --name tst python
Fetching package metadata: ....
Solving package specifications: ............
Package plan for installation in environment D:\anaconda32\envs\tst:

The following NEW packages will be INSTALLED:

    msvc_runtime: 1.0.1-vc9_0   [vc9]
    pip:          7.1.2-py27_0
    python:       2.7.10-4
    setuptools:   18.4-py27_0
    wheel:        0.26.0-py27_1

Proceed ([y]/n)? y

Linking packages ...
[      COMPLETE      ]|##################################################| 100%
#
# To activate this environment, use:
# > activate tst
#

D:\workspace>activate tst
Activating environment "D:\anaconda32\envs\tst"...

The bug

[tst] D:\workspace>pip install -U setuptools
Collecting setuptools
  Downloading setuptools-18.5-py2.py3-none-any.whl (462kB)
    100% |################################| 462kB 787kB/s
Installing collected packages: setuptools
  Found existing installation: setuptools 18.4
Cannot remove entries from nonexistent file d:\anaconda32\envs\tst\lib\site-pack
ages\easy-install.pth

Trying to install -U any package depending on setuptools also fails with the same error.

Known workarounds

Adding the --ignore-installed option will sucessfully upgrade setuptools and the problem will be solved for the lifetime of that particular conda environment. Downloading and running ez_setup.py also fixes the problem for that conda environment. The second workaround will create the missing pth file, while the first will not, but in both cases everything seems to work afterwards.

While using conda to upgrade setuptools is also a workaround, it is not a good solution, since a lot of packages on pypi (1822 in 2013) depend on setuptools and therefore try to upgrade setuptools too, when upgraded:

[tst] D:\workspace>pip install -U zc.buildout
Collecting zc.buildout
  Downloading http://python/root/pypi/+f/739/582d22e3ddd5e/zc.buildout-2.5.0-py2
.py3-none-any.whl (261kB)
    100% |################################| 262kB 6.8MB/s
Collecting setuptools>=8.0 (from zc.buildout)
  Downloading http://python/root/pypi/+f/d40/182384798e286/setuptools-18.8.1-py2
.py3-none-any.whl (463kB)
    100% |################################| 466kB 10.2MB/s
Installing collected packages: setuptools, zc.buildout
  Found existing installation: setuptools 18.5
Cannot remove entries from nonexistent file d:\anaconda32\envs\tst\lib\site-pack
ages\easy-install.pth

A lot of pypi packages are not installable using conda install, and so using conda to upgrade these is not an option.

None of these workarounds are good permanent solutions to this bug because:

  1. They will trip a lot of users who do not know that adding -I fixes the problem and will have to spend a considerable amount of time searching for some solution.
  2. They break automated testing (e.g. tox)
  3. They do not fix the bug permanently on that PC, but just work for the lifetime of the environment, which is no good if the environment is a throwaway environment.

Cause

The error message is correct, there is no easy-install.pth file in site-packages. The error occurs when pip tries to uninstall the old setuptools package, in pip/req/req_uninstall.py:160. The file pip is trying to remove (easy-install.pth) does not exist in a fresh Anaconda/Miniconda installation because no conda packages are able to include this file.

This is a periodically sleeping bug, since it rears its head whenever the conda setuptools version (currently 18.8.1) lags behind the pip setuptools version (currently 19.1.1).

The assumption of easy-install.pth file is guarded by a check for .egg:

        elif dist.location.endswith('.egg'):
            # package installed by easy_install
            # We cannot match on dist.egg_name because it can slightly vary
            # i.e. setuptools-0.6c11-py2.6.egg vs setuptools-0.6rc11-py2.6.egg
            paths_to_remove.add(dist.location)
            easy_install_egg = os.path.split(dist.location)[1]
            easy_install_pth = os.path.join(os.path.dirname(dist.location),
                                            'easy-install.pth')
            paths_to_remove.add_pth(easy_install_pth, './' + easy_install_egg)

So my guess is that your conda package looks like an easy_install egg: this may confuse other tools as well. Perhaps don’t do that?

I’m marking this as awaiting response. Please comment if the recently-released 20.1b1 still has this problem. Otherwise the bot will auto-close this after some time.

No, you’ve misunderstood. .egg-info is not the same as .egg. The test I pasted is for “SOMETHING.egg”, not “SOMETHING.egg-info”. What you are doing with .egg-info files is probably correct (though I haven’t read your code to authoritatively say so).

Correct, conda packages usually include .egg-info (as it is installed by setuptools during the build process). If we removed all .egg-info files from conda packages, pip wouldn’t be able to see what’s installed at all (and try to overwrite packages). Not including .egg-info files wouldn’t be a problem for conda itself. For conda, .egg-info files are just like any other files that are part of a conda package. Many conda packages (the ones which are not Python specific, such as libxml2, hdf5, r, etc.) have no .egg-info files either.