pyproject-hooks: Version 1.2 breaks pypa/build on Python 3.8 and 3.9
1.2 has broken build’s test suite when using backend-path on Python 3.8 and 3.9 (and check-sdist’s setuptools test, though I just dropped the setuptools test there). See https://github.com/pypa/build/pull/820, https://github.com/python/importlib_metadata/issues/508, https://github.com/henryiii/check-sdist/pull/55. I’ve finally narrowed it down to this package’s 1.2 release.
Backend path is ".", and setuptools is making test_no_prepare.egg-info, which is then getting picked up as a nameless distribution, which causes a confusing error when trying to get the name and falling back on None, then not handling the None case.
_Originally posted by @henryiii in https://github.com/pypa/pyproject-hooks/issues/199#issuecomment-2395287934_
Traceback (most recent call last):
File "/build/.tox/py39/lib/python3.9/site-packages/importlib_metadata/compat/py39.py", line 19, in normalized_name
return dist._normalized_name
AttributeError: 'PathDistribution' object has no attribute '_normalized_name'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/build/.tox/py39/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
main()
File "/build/.tox/py39/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
File "/build/.tox/py39/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
return _build_backend().build_wheel(
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/build_meta.py", line 421, in build_wheel
return self._build_with_temp_dir(
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/build_meta.py", line 403, in _build_with_temp_dir
self.run_setup()
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/build_meta.py", line 318, in run_setup
exec(code, locals())
File "<string>", line 1, in <module>
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
return distutils.core.setup(**attrs)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 183, in setup
return run_commands(dist)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
dist.run_commands()
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
self.run_command(cmd)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/dist.py", line 950, in run_command
super().run_command(command)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
cmd_obj.run()
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/command/bdist_wheel.py", line 433, in run
self.run_command("install")
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
self.distribution.run_command(command)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/dist.py", line 950, in run_command
super().run_command(command)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
cmd_obj.run()
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/command/install.py", line 91, in run
return super().run()
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/command/install.py", line 706, in run
self.run_command(cmd_name)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
self.distribution.run_command(command)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/dist.py", line 950, in run_command
super().run_command(command)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
cmd_obj.run()
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/command/install_egg_info.py", line 32, in run
self.run_command('egg_info')
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
self.distribution.run_command(command)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/dist.py", line 950, in run_command
super().run_command(command)
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
cmd_obj.run()
File "/build/.tox/py39/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 302, in run
for ep in metadata.entry_points(group='egg_info.writers'):
File "/build/.tox/py39/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 1051, in entry_points
return EntryPoints(eps).select(**params)
File "/build/.tox/py39/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 1048, in <genexpr>
eps = itertools.chain.from_iterable(
File "/build/.tox/py39/lib/python3.9/site-packages/importlib_metadata/_itertools.py", line 17, in unique_everseen
k = key(element)
File "/build/.tox/py39/lib/python3.9/site-packages/importlib_metadata/compat/py39.py", line 23, in normalized_name
return Prepared.normalize(getattr(dist, "name", None) or dist.metadata['Name'])
File "/build/.tox/py39/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 889, in normalize
return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
File "/usr/local/lib/python3.9/re.py", line 210, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 1
- Comments: 20 (16 by maintainers)
In that case, maybe the issue could be addressed in Setuptools by keeping the egg-info directory off sys.path until it’s populated.
@jaraco, I suspect that this parish aspect is an issue in setuptools because it uses entry-points to generate the
.egg-infodirectory itself. So setuptools create an empty.egg-infofolder, and then it fails while populating it (race-condition-ish unravelled by the change).Oh, do we understand why this issue only affects Python 3.9 and earlier? If the underlying problem is a distribution with no
Name, I’d expect that also to cause problems. I guess it doesn’t becauseDistribution.name(or._normalized_name) happens to returnNonefor these invalid distributions and because theuniquecheck when scanning entry points happens not to care if it getsNonewhere it expects astr. That means that this issue will also affect Python 3.14, whereimportlib.metadata.Distribution.namewill now raise aKeyErrorwhen no name is present in the metadata.As a reminder,
importlib.metadataprior to Python 3.10 was considered “provisional”, so it’s probably preferable to useimportlib_metadataat least on those versions.importlib_metadatahas attempted to accommodate objects from those implementations, but it’s complicated due to the two API surfaces that library manages (providers and consumers). In python/importlib_metadata#486, we’re exploring ways to shore up some of those relationships, but that still won’t fix the fact thatimportlib.metadataon Python 3.9 and earlier will never honor the newer interfaces and guarantees of later releases. Later versions ofimportlib_metadatawill provide an error earlier when theNamekey is missing, but only forDistributionsconstructed fromimportlib_metadata.That said, an
egg-infodirectory containing no metadata is in fact invalid, right? Shouldn’t the solution be to address the root cause (prevent that invalid metadata from existing)? What is the origin of these empty metadata directories?This is only about Python 3.8 and 3.9, everything works fine after that.
It breaks both:
If you avoid the new pyproject-hooks, it works:
You have to remove the
*.dist-infodir after a successful build, however, otherwise it starts working.My inclination is to say that this is either a bug in setuptools (which should spot when it gets a stdlib object and deal with it) or in the
importlib_metadatabackport (which should offer some sort of compatibility).None of which helps that much, unless those projects can release fixes ASAP, but at the same time, I sympathise with @takluyver’s point that if we can’t rely on being able to safely pass distribution objects around, we’re going to be at risk of yet more areas of incompatibility popping up.