setuptools: [BUG] TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'
setuptools version
setuptools>=71.0.0
Python version
Python 3.12
OS
Ubuntu
Additional environment information
I found this bug when running the test for the project in GitHub actions
Description
When I install the requirements in my Python package project (pip-sync requirements-dev.txt) I get an error within setuptools which says:
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
...
TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'
This error was caused by using setuptools >= 71.0.0
Not to speculate but it looks like the issue was introduced by this commit in setuptools: https://github.com/pypa/setuptools/commit/00384a5f4fd22c653172b99feefe13b0009eb870
And this commit is including new changes from the packaging project and its this commit here where we can see the issue: https://github.com/pypa/packaging/commit/cc938f984bbbe43c5734b9656c9837ab3a28191f https://github.com/pypa/packaging/blame/4493dfcd95a893f676a7aa4bd17c547bea676371/src/packaging/utils.py#L58
Expected behavior
I would expect pip-sync requirements-dev.txt to run successfully
How to Reproduce
Clone https://github.com/tim-s-ccs/example-python-package.git
Make sure you are using Python 3.12 and you have pip-tools installed (pip install --upgrade pip wheel pip-tools)
Make sure you are using the correct version of setuptools pip install --upgrade setuptools==71.0.0
Run pip-compile requirements-dev.in
Output
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [42 lines of output]
running egg_info
creating /private/var/folders/kt/n16_msf56_z3kfyb65l37wxhb_m4s_/T/pip-pip-egg-info-3sca_19y/example_python_package.egg-info
writing /private/var/folders/kt/n16_msf56_z3kfyb65l37wxhb_m4s_/T/pip-pip-egg-info-3sca_19y/example_python_package.egg-info/PKG-INFO
writing dependency_links to /private/var/folders/kt/n16_msf56_z3kfyb65l37wxhb_m4s_/T/pip-pip-egg-info-3sca_19y/example_python_package.egg-info/dependency_links.txt
writing top-level names to /private/var/folders/kt/n16_msf56_z3kfyb65l37wxhb_m4s_/T/pip-pip-egg-info-3sca_19y/example_python_package.egg-info/top_level.txt
writing manifest file '/private/var/folders/kt/n16_msf56_z3kfyb65l37wxhb_m4s_/T/pip-pip-egg-info-3sca_19y/example_python_package.egg-info/SOURCES.txt'
reading manifest file '/private/var/folders/kt/n16_msf56_z3kfyb65l37wxhb_m4s_/T/pip-pip-egg-info-3sca_19y/example_python_package.egg-info/SOURCES.txt'
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/Users/my.name/Code/test/example-python-package/setup.py", line 16, in <module>
setup(
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/__init__.py", line 106, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
dist.run_commands()
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 970, in run_commands
self.run_command(cmd)
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/dist.py", line 974, in run_command
super().run_command(command)
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
cmd_obj.run()
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run
self.find_sources()
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources
mm.run()
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 555, in run
self.prune_file_list()
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 621, in prune_file_list
base_dir = self.distribution.get_fullname()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/_core_metadata.py", line 266, in get_fullname
return _distribution_fullname(self.get_name(), self.get_version())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/setuptools/_core_metadata.py", line 284, in _distribution_fullname
canonicalize_version(version, strip_trailing_zero=False),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
Traceback (most recent call last):
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 64, in generate_metadata
call_subprocess(
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/utils/subprocess.py", line 209, in call_subprocess
raise error
pip._internal.exceptions.InstallationSubprocessError: python setup.py egg_info exited with 1
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/my.name/.pyenv/versions/3.12.2/bin/pip-compile", line 8, in <module>
sys.exit(cli())
^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/piptools/scripts/compile.py", line 470, in cli
results = resolver.resolve(max_rounds=max_rounds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/piptools/resolver.py", line 604, in resolve
is_resolved = self._do_resolve(
^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/piptools/resolver.py", line 636, in _do_resolve
resolver.resolve(
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 76, in resolve
collected = self.factory.collect_root_requirements(root_reqs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 538, in collect_root_requirements
reqs = list(
^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 494, in _make_requirements_from_install_req
cand = self._make_base_candidate_from_link(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 210, in _make_base_candidate_from_link
self._editable_candidate_cache[link] = EditableCandidate(
^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 328, in __init__
super().__init__(
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 158, in __init__
self.dist = self._prepare()
^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 235, in _prepare
dist = self._prepare_distribution()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 338, in _prepare_distribution
return self._factory.preparer.prepare_editable_requirement(self._ireq)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 698, in prepare_editable_requirement
dist = _get_prepared_distribution(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 72, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 69, in prepare_distribution_metadata
self.req.prepare_metadata()
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/req/req_install.py", line 580, in prepare_metadata
self.metadata_directory = generate_metadata_legacy(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/my.name/.pyenv/versions/3.12.2/lib/python3.12/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 71, in generate_metadata
raise MetadataGenerationFailed(package_details=details) from error
pip._internal.exceptions.MetadataGenerationFailed: metadata generation failed
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 31
- Comments: 39 (9 by maintainers)
Links to this issue
Commits related to this issue
- Update changelog to reflect common experience seen in #4478 and #4483. — committed to pypa/setuptools by jaraco 2 years ago
- Use `setuptools [core]` @ collections_runtime_pythonpath This is now necessary since `setuptools >= 71` started preferring externally present stdlib deps over the vendored ones. Refs: * https://gith... — committed to webknjaz/ansible by webknjaz 2 years ago
- Add documentation clarifying how to reliably install setuptools with its dependencies. Ref #4483 — committed to pypa/setuptools by jaraco 2 years ago
- pin setuptools to 65 (#87) Required to avoid https://github.com/pypa/setuptools/issues/4483 — committed to nvidia-riva/python-clients by virajkarandikar 2 years ago
- Update packaging constraint to match 2023.2 A recent release of setuptools breaks when packaging is <22.0 [0]. Because we can't reliably pin setuptools in the real world (we expect people to simple c... — committed to openstack/requirements by cboylan 2 years ago
- [CHIP project] fix setuptools bug (#12262) - this fixes docker build failure, due to bug introduced by update of [setuptools](https://pypi.org/project/setuptools/) python package to version 71 -... — committed to google/oss-fuzz by Alami-Amine 2 years ago
- CI setuptools bugfix Latest version of setuptools requires to update packaging manually as it is not in the dependency section. More info: https://github.com/pypa/setuptools/issues/4483 It also rem... — committed to softwarefactory-project/DLRN by evallesp 2 years ago
- Pin setuptools In the containers where beam jobs are executed, it appears that the latest setuptools version is used. This seems to result in an error which blocks beam job runs on the server. See ht... — committed to kevintab95/oppia by deleted user 2 years ago
- Pin setuptools version This is needed until the issue in [1] is fixed, as it's preventing builds from working. [1] - https://github.com/pypa/setuptools/issues/4483/ — committed to simonsobs/ocs by BrianJKoopman 2 years ago
- Fix CI about packaging version, see desc The issue is described on GitHub https://github.com/pypa/setuptools/issues/4483 — committed to 3liz/py-qgis-server by Gustry 2 years ago
- Fix CI about packaging version again, see desc The issue is described on GitHub https://github.com/pypa/setuptools/issues/4483 — committed to 3liz/py-qgis-server by Gustry 2 years ago
- Update packaging constraint to match 2023.2 A recent release of setuptools breaks when packaging is <22.0 [0]. Because we can't reliably pin setuptools in the real world (we expect people to simple c... — committed to openstack/requirements by cboylan 2 years ago
- Update packaging version If we want to use setuptools with version > 71, we have to update the packaging version to 22 or higher. Otherwise the build process fails with TypeError: canonicalize_versi... — committed to mkosiarc/atomic-reactor by mkosiarc 2 years ago
- Update setuptools and packaging version The setuptools update addresses a vulnerability alert by dependabot: A vulnerability in the package_index module of pypa/setuptools versions up to 69.1.1 allow... — committed to mkosiarc/atomic-reactor by mkosiarc 2 years ago
- Update setuptools and packaging version The setuptools update addresses a vulnerability alert by dependabot: A vulnerability in the package_index module of pypa/setuptools versions up to 69.1.1 allow... — committed to containerbuildsystem/atomic-reactor by mkosiarc 2 years ago
- Bump packaging to avoid setuptools error To avoid this issue in setuptools>=71: https://github.com/pypa/setuptools/issues/4483 Also need to list pyparsing explicitly as requirement as it is no long... — committed to ckan/ckan by amercader 2 years ago
- Update Dockerfile solution: https://github.com/pypa/setuptools/issues/4483 — committed to data-for-change/anyway-etl by atalyaalon 2 years ago
- Update Dockerfile solution: https://github.com/pypa/setuptools/issues/4483#issuecomment-2236327987 — committed to data-for-change/anyway-etl by atalyaalon 2 years ago
- Upgrade packaging. Background: pypa/setuptools#4483 — committed to termim/pymssql by termim 2 years ago
- fix: set packaging minimum version Set packaging minimum version to 22.0 to avoid the following error with setuptools 71.x: TypeError: canonicalize_version() got an unexpected keyword argument 'stri... — committed to ffontaine/cve-bin-tool by ffontaine 2 years ago
Also it can be fixed by update packaging to 22.0 or higher
This behavior is by design. Similar to the issue reported in #4478, the new Setuptools 71 will prefer installed dependencies over the vendored ones. You’ll want to either uninstall the older (incompatible) dependencies from the environment or install these dependencies.
You may be tempted to installsetuptools[core], but please avoid doing that ascoreis an internal implementation detail.Could reproduce the same issue. Fixed by downgrading to 70.x
You save me!!! all you need to do is upgrade to setuptools==70.0.0. Don’t try > 74 or less than 70. None of them can work!!! Packaging I use is 24.1.
For me:
pip3.10 install --upgrade setuptools==70.0.0You can use:
pip install --upgrade setuptools==70.0.0Downgrading setuptools or adding packaging with a specific version as a dependency are both weird. The docs just say that people need to add setuptools as a build dependency, nothing about packaging. If setuptools depends on packaging, and it is incompatible with older versions of packaging, then
"packaging>=MIN_VERSION"should be a required dependency of setuptools, not in acoreoptional dependency recommended to not be used.@jaraco how is this behavior “by design”? What I see in the docs does not help.
Just to confirm, we’re seeing this happen in this environment:
Is that correct? It sounds like you’re saying the issue is old dependencies existing in the environment, but we should have the most current packages here.
Issue is visible when you use packaging<22.0
This actually works. Thanks!
Solution: https://github.com/pypa/setuptools/issues/4483#issuecomment-2322520170
Thank you really much, this solved the issue. We were running setuptools before 70.x.x. In our setup we have Databricks which installs a whl to execute our libraries. Here the config for others:
Yes, probably.
Agreed, it is a bit of a sharp corner, and we made the major release as a signal of this shift in expectations. The issue is that while you’re holding it correctly and
pip install -U setuptoolsshould theoretically upgrade all the required dependencies, it cannot currently due to the bootstrapping problem. I’m actively working on solving that issue, but in the meantime, one needs topip install -U setuptools[core]to prepare an environment safely.I think I may have been too cautious about this advice. We specifically want to avoid packages themselves making this declaration in
build-requires. I believe it’s safe to use in systems integration tooling.Thanks @jaraco this was helpful. In my project where I was having the issue because on older version of
packagingwas a dependancy (not compatible with setuptools v71.0.0 or maybe earlier). Updating that dependancy did indeed fix the issue.It’s a bit annoying that the behaviour of setuptools changed which messed up my tests (which gets the latest version of setuptools when setting up) but that is very much my fault and not your teams.
Again, thanks for your comment as it helped my to understand the issue better and what the cause was.
For anyone who has a situation like me where my tests just started failing I’d check to make sure your dependancies and the dependancies of setuptools are compatible.
Why do I need to downgrade to 70.0.0? This is nuts…
Thanks it worked
Per the guidance, please make sure that the
setuptools[core]dependencies are installed. In particular,packaging>=24.installing thedependencies in the link works ! please make new build requierment file inseatd of setup tools py and include all dependencies along with setup tools like below: packaging>=24 ordered-set>=3.1.1 more_itertools>=8.8 jaraco.text>=3.7 importlib_resources>=5.10.2 importlib_metadata>=6 tomli>=2.0.1 wheel>=0.43.0
pkg_resources
platformdirs >= 2.6.2 setuptools~=70.0 running this file with apt update and pip install makes the packages run without error
Oof. This brought all of our CI to a halt, which is the last thing I needed to deal with during a massive migration from gitlab to github.
canonicalize_versionis in thepackagingmodule.(aside) Now that I write (4), I realize that
packaginguses flit-core, so maybe it’s viable to move that dependency (and others that aren’t affected by the bootstrapping problem) to required dependencies and not optional.When I say it’s by design, I mean I’m unaware of any other design that meets Setuptools stated goal to be able to follow standard dependency practices and declare dependencies on any packages it requires. This goal is something I’ve been working on for years.
I recognize and sympathize with the downstream effects of this transition, but I have to say, I’m really appreciating the progress this project has made toward having a simpler and more manageable vendoring strategy and possibly eventually removing the need for vendoring altogether.
I’m open to suggestions, but “just bundle everything all the time” is a regression in the design and a non-starter. What is your suggestion?
I think here is probably the answer.
I added the following to setup.py:
And that gave me:
I think that’s all the info I need to remedy this: ~/.local/lib/… comes before .conda/envs/my_project/lib/… I also tried explicitly adding packaging>=24.0 to the environment.yml, but no change. Ah, the intricacies of Python sourcing.
Also, to note, the build script I’m using to run this definitely specifies the pip installed in the environment. And, with the environment activated,
which pipreferences the environment’s pip. So, I don’t think it’s because pip is getting called from elsewhere.One way or another, I think I have all the info now to figure out some way to properly import the correct version of packaging.
Thank you very much for looking that over, and providing some feedback, Jason!
Thank you @doctornkz-intelas @jaraco for your inputs and guidance, We are planning to upgrade MWAA to 2.8.1 and I have tested the virtual environment in airflow 2.8.1 the issue is gone as the packaging package came with upgraded version in constraints file.
Hey @jaraco I am using pipenv and running with these versions bellow
But still getting the same error
TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'Can confirm installing 6.1.14 is a temporary solution.
python -m pip install --force-reinstall -v "platformio==6.1.14"That’s a really good point. There’s a large scope of users affected by this issue but only a small number of system integrators affected by the bootstrapping problem. Perhaps we shouldn’t be imposing this extra burden on the larger group of users and should instead push it to integrators (require them, for example, to patch out the declared dependencies if there’s no solution to the bootstrap problem).
I agree - updating the installation guide to capture the current state makes sense.
Thanks for all the information. This change has broken our CI this morning too. Much as I appreciate the workarounds given here, I would have preferred this behaviour to not have gone mainstream until https://github.com/pypa/packaging-problems/issues/342 is resolved, so I don’t have to go back and undo these workarounds in the future.
It may be worth referencing this issue in the quickstart documentation since following the documentation may now not work in many existing environments? https://setuptools.pypa.io/en/latest/userguide/quickstart.html#installation
Thanks for posting your solution. I’m trying to reason through how we’re running into the same issue. We’re not specifying
packaginganywhere, so I would assume we would get the most current version. And when I install locally, I get the same version ofpackagingyou specify in your PR. Perhaps the CI image I’m using already haspackaginginstalled, but with an older version? I’ll investigate and post back if I find something. But it would definitely be a sharp corner ifpip install --upgrade setuptoolsno longer works onubuntu-latest.