setuptools: race condition replacing a setuptools dependency IOError METADATA no such file
This is a strange one. I have narrowed it down to setuptools new version (patch) as the only thing that has changed recently that causes a flow to break. I have reduced the problem to the bare minimum, but the flow that is actually occurring is a bit more complicated, here are the minimum steps needed to reproduce the problem.
I have been installing junit-xml and zope.interface in a test environment using the following command:
python -m pip install --upgrade --force-reinstall junit-xml zope.interface
This is the output:
Collecting junit-xml
Using cached junit-xml-1.7.tar.gz
Collecting zope.interface
Using cached zope.interface-4.3.3-cp27-cp27m-win_amd64.whl
Collecting six (from junit-xml)
Using cached six-1.10.0-py2.py3-none-any.whl
Collecting setuptools (from zope.interface)
Using cached setuptools-34.1.0-py2.py3-none-any.whl
Collecting appdirs>=1.4.0 (from setuptools->zope.interface)
Using cached appdirs-1.4.0-py2.py3-none-any.whl
Collecting packaging>=16.8 (from setuptools->zope.interface)
Using cached packaging-16.8-py2.py3-none-any.whl
Collecting pyparsing (from packaging>=16.8->setuptools->zope.interface)
Using cached pyparsing-2.1.10-py2.py3-none-any.whl
Installing collected packages: six, junit-xml, appdirs, pyparsing, packaging, setuptools, zope.interface
Found existing installation: six 1.10.0
Uninstalling six-1.10.0:
Successfully uninstalled six-1.10.0
Rolling back uninstall of six
Exception:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\pip\basecommand.py", line 215, in main
status = self.run(options, args)
File "C:\Python27\lib\site-packages\pip\commands\install.py", line 342, in run
prefix=options.prefix_path,
File "C:\Python27\lib\site-packages\pip\req\req_set.py", line 784, in install
**kwargs
File "C:\Python27\lib\site-packages\pip\req\req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "C:\Python27\lib\site-packages\pip\req\req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "C:\Python27\lib\site-packages\pip\wheel.py", line 247, in move_wheel_files
prefix=prefix,
File "C:\Python27\lib\site-packages\pip\locations.py", line 140, in distutils_scheme
d = Distribution(dist_args)
File "C:\Python27\lib\site-packages\setuptools\dist.py", line 320, in __init__
_Distribution.__init__(self, attrs)
File "C:\Python27\lib\distutils\dist.py", line 287, in __init__
self.finalize_options()
File "C:\Python27\lib\site-packages\setuptools\dist.py", line 386, in finalize_options
ep.require(installer=self.fetch_build_egg)
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2318, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 862, in resolve
new_requirements = dist.requires(req.extras)[::-1]
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2562, in requires
dm = self._dep_map
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2809, in _dep_map
self.__dep_map = self._compute_dependencies()
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2818, in _compute_dependencies
for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2800, in _parsed_pkg_info
metadata = self.get_metadata(self.PKG_INFO)
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 1468, in get_metadata
value = self._get(self._fn(self.egg_info, name))
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 1577, in _get
with open(path, 'rb') as stream:
IOError: [Errno 2] No such file or directory: 'c:\\python27\\lib\\site-packages\\six-1.10.0.dist-info\\METADATA'
However, the file DOES exist. (the path displayed is not the correct case of the actual path though, which is C:\Python27\Lib\site-packages\six-1.10.0.dist-info\metadata
uninstalling everything and then trying the command again yields the same result. changing the command to remove the upgrade or the force reinstall does not have the same problem.
this is very weird, and the only thing that changed is the new version of setuptools
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 1
- Comments: 25 (12 by maintainers)
Commits related to this issue
- Disable import of setuptools for sdist builds. Fixes #4264 and pypa/setuptools#951. — committed to jaraco/pip by jaraco 7 years ago
- Temporarily fix pypa/setuptools#951 — committed to gridsingularity/gsy-e by ulope 7 years ago
- Use setuptools 33.1.1 As per [1] and [2] setuptools 34.0.0-34.2.0 does not handle re-installs or upgrades properly. This patch reduces the version we use to the last known good version. [1] https:/... — committed to openstack/openstack-ansible by deleted user 7 years ago
- Use setuptools 33.1.1 As per [1] and [2] setuptools 34.0.0-34.2.0 does not handle re-installs or upgrades properly. This patch reduces the version we use to the last known good version. [1] https:/... — committed to openstack/openstack-ansible by deleted user 7 years ago
- Updated openstack/openstack Project: openstack/requirements 705628e39674746e4bbeeefb7b5ca0dbe7571b3a global-requirements: Avoid setuptools 34.0.0-34.2.0 setuptools seems to have some race issues wh... — committed to openstack/openstack by deleted user 7 years ago
- global-requirements: Avoid setuptools 34.0.0-34.2.0 setuptools seems to have some race issues when forcing a package reinstallation with 'pip install --upgrade --force-reinstall' as described in http... — committed to openstack/requirements by deleted user 7 years ago
- Updated openstack/openstack Project: openstack/requirements 705628e39674746e4bbeeefb7b5ca0dbe7571b3a global-requirements: Avoid setuptools 34.0.0-34.2.0 setuptools seems to have some race issues wh... — committed to openstack/openstack by deleted user 7 years ago
- playbooks: roles: {ironic,keystone}: Drop --force-reinstall setuptools seems to have some race issues when forcing a package reinstallation with 'pip install --upgrade --force-reinstall' as described... — committed to openstack/bifrost by deleted user 7 years ago
- install setuptools 33.1.1 running into this bug with the latest version: https://github.com/pypa/setuptools/issues/951 — committed to cloudbase/cinder-ci by andreibacos 7 years ago
- Fix the vagrant setup The Vagrant bootstrapping was not working correctly. I have got it working by basing it on the Dockerfile setup that is working correctly. - Debian dependencies changed to matc... — committed to hmcts/courtfinder-search by robyoung 7 years ago
- playbooks: roles: {ironic,keystone}: Drop --force-reinstall setuptools seems to have some race issues when forcing a package reinstallation with 'pip install --upgrade --force-reinstall' as described... — committed to openstack/bifrost by deleted user 7 years ago
- Revert to setuptools 33.1.1 As per [1] and [2] setuptools 34.0.0-34.2.0 does not handle re-installs or upgrades properly. This patch reduces the version we use to the last known good version. [1] h... — committed to openstack/openstack-ansible by deleted user 7 years ago
- Revert to setuptools 33.1.1 As per [1] and [2] setuptools 34.0.0-34.2.0 does not handle re-installs or upgrades properly. This patch reduces the version we use to the last known good version. [1] h... — committed to openstack/openstack-ansible by deleted user 7 years ago
- Updated openstack/openstack Project: openstack-infra/project-config 3689aa75337d92c40e6f6673de15c7706d1ce213 Temporary make stable compat ocata jobs non-voting New version setuptools[1] breaks oslo... — committed to openstack/openstack by deleted user 7 years ago
- Temporary make stable compat ocata jobs non-voting New version setuptools[1] breaks oslo gate jobs: gate-tempest-dsvm-neutron-src-oslo.config-ubuntu-xenial-(newton|ocata) It's not easy to fix complet... — committed to openstack/project-config by glongwave 7 years ago
- For more info, see: https://github.com/pypa/setuptools/issues/951 — committed to cpoppema/docker-flexget by cpoppema 7 years ago
- #7: Pin setuptools to version 32 to avoid race condition when updating packages. For more info, see: https://github.com/pypa/setuptools/issues/951 — committed to cpoppema/docker-flexget by cpoppema 7 years ago
- Blacklist setuptools versions 34.0.0-34.3.2 Excluding setuptools versions 34.0.0-34.3.2 until this has been fixed: https://github.com/pypa/setuptools/issues/951 Change-Id: I256c771869b4399d9fb9d4ea... — committed to openstack-archive/tripleo-validations by flofuchs 7 years ago
- Blacklist setuptools versions 34.0.0-34.3.2 Excluding setuptools versions 34.0.0-34.3.2 until this has been fixed: https://github.com/pypa/setuptools/issues/951 (cherry picked from commit 1f862615b... — committed to openstack-archive/tripleo-validations by flofuchs 7 years ago
- Fix TravisCI Pin setuptools to version 32 to avoid race condition https://github.com/pypa/setuptools/issues/951#issuecomment-279150378 — committed to alexcoman/arestor by mateimicu 7 years ago
I’ve been hitting (what appears to be) the same bug but with
appdirs
instead ofsix
I also just encountered this as well. Downgrading from setuptools-35.0.2 to setuptools-33.1.1 worked…
This only started happening since the patch a day or two ago.
Probably from Jan 28.
I may be wrong and it’s unrelated, but this is where my sleuthing got me.
The problem is that I can’t install each dependency individually, because once I run the --force-reinstall, it also reinstall the dependencies.