setuptools: [BUG] Version 78.0.1 breaks install of ansible-vault package
setuptools version
78.0.1
Python version
3.11
OS
Ubuntu
Additional environment information
No response
Description
When installing the “ansible-vault” package, setuptools raises this exception:
$ pip install ansible-vault --upgrade
Collecting ansible-vault
Using cached ansible-vault-2.1.0.tar.gz (3.5 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [34 lines of output]
Traceback (most recent call last):
File "/Users/maiera/virtualenvs/vpcauto311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
main()
File "/Users/maiera/virtualenvs/vpcauto311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/maiera/virtualenvs/vpcauto311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
self.run_setup()
File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 522, in run_setup
super().run_setup(setup_script=setup_script)
File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 320, in run_setup
exec(code, locals())
File "<string>", line 11, in <module>
File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 116, in setup
_install_setup_requires(attrs)
File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 87, in _install_setup_requires
dist.parse_config_files(ignore_option_errors=True)
File "/Users/maiera/virtualenvs/vpcauto311/lib/python3.11/site-packages/_virtualenv.py", line 22, in parse_config_files
result = old_parse_config_files(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 730, in parse_config_files
self._parse_config_files(filenames=inifiles)
File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 599, in _parse_config_files
opt = self._enforce_underscore(opt, section)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 629, in _enforce_underscore
raise InvalidConfigError(
setuptools.errors.InvalidConfigError: Invalid dash-separated key 'description-file' in 'metadata' (setup.cfg), please use the underscore name 'description_file' instead.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Expected behavior
ansible-vault has not changed since 2021 and should install as before.
How to Reproduce
- Create and activate a fresh Pyhton virtual env, using Python 3.11:
mkvirtualenv test -p python3.11
- Install version 78.0.1 of setuptools
pip install setuptools==78.0.1
- Install ansible-vault
pip install ansible-vault
Output
See above
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 221
- Comments: 120 (14 by maintainers)
Links to this issue
Commits related to this issue
- Pin setuptools It looks like there are issue with the new release: https://github.com/pypa/setuptools/issues/4910 — committed to apache/iceberg-python by Fokko 2 years ago
- fix(SI-313): underscore key See https://github.com/pypa/setuptools/issues/4910 for more details — committed to prolific-oss/prolific-django-rest-framework-mongoengine by benjefferies 2 years ago
- Fix setup.cfg dashes https://github.com/pypa/setuptools/issues/4910 — committed to Avatar-Cognition/assertpy by deleted user 2 years ago
- fix setup.cfg dash in description-file https://github.com/pypa/setuptools/issues/4910 — committed to dpinol/colourmap by deleted user 2 years ago
- Use the underscore name `description_file` in `setup.cfg` instead of deprecated `description-file` - https://github.com/pypa/setuptools/pull/4870 - https://github.com/pypa/setuptools/issues/4910 — committed to MeltanoLabs/singer-io--singer-python by edgarrmondragon 2 years ago
- Update setup.cfg https://github.com/pypa/setuptools/issues/4910 — committed to scschneider44/admet_ai by scschneider44 2 years ago
- Workaround setuptools 78.0.1 mayhem The setuptools 78.0.1 started to fail old packages that had dash metadata keys and had only .sdist installation. This caused failures of three of our dependencies:... — committed to potiuk/airflow by potiuk 2 years ago
- Workaround setuptools 78.0.1 mayhem The setuptools 78.0.1 started to fail old packages that had dash metadata keys and had only .sdist installation. This caused failures of three of our dependencies:... — committed to potiuk/airflow by potiuk 2 years ago
- chore: Modernize packaging (#110) - https://github.com/pypa/setuptools/pull/4870 - https://github.com/pypa/setuptools/issues/4910 - https://github.com/astral-sh/uv/issues/12440 - https://github.com/a... — committed to MeltanoLabs/tap-gitlab by edgarrmondragon 2 years ago
- Workaround setuptools 78.0.1 mayhem The setuptools 78.0.1 started to fail old packages that had dash metadata keys and had only .sdist installation. This caused failures of three of our dependencies:... — committed to potiuk/airflow by potiuk 2 years ago
- Workaround setuptools 78.0.1 mayhem The setuptools 78.0.1 started to fail old packages that had dash metadata keys and had only .sdist installation. This caused failures of three of our dependencies:... — committed to potiuk/airflow by potiuk 2 years ago
- solarwinds/orionsdk-python fix - updating setup.cfg to use underscore in description_file per pypa/setuptools#4910 — committed to FliesLikeABrick/orionsdk-python by FliesLikeABrick 2 years ago
- Pin `setuptools<78` cf. https://github.com/pypa/setuptools/issues/4910 — committed to single-cell-data/TileDB-SOMA by ryan-williams 2 years ago
- Workaround setuptools 78.0.1 mayhem The setuptools 78.0.1 started to fail old packages that had dash metadata keys and had only .sdist installation. This caused failures of three of our dependencies:... — committed to potiuk/airflow by potiuk 2 years ago
- [#739 #741] Recent changes to setuptools (see https://github.com/pypa/setuptools/issues/4910) mean that pylint-flask does not install correctly, and it is no longer maintained or necessary (see https:... — committed to prospector-dev/prospector by carlio 2 years ago
- build: pin setuptools to <78 We currently have a dependency (macaroons) that cannot build on >=78 due to a removal: https://github.com/pypa/setuptools/issues/4910 — committed to canonical/rockcraft by tigarmo 2 years ago
- build: pin setuptools to <78 We currently have a dependency (macaroons) that cannot build on >=78 due to a removal: https://github.com/pypa/setuptools/issues/4910 — committed to canonical/rockcraft by tigarmo 2 years ago
- Use underscore instead of dash in setup.cfg https://github.com/pypa/setuptools/issues/4910 — committed to singer-io/singer-python by bryantgray 2 years ago
- Use underscore instead of dash in setup.cfg (#171) * Use underscore instead of dash in setup.cfg https://github.com/pypa/setuptools/issues/4910 * disable some pylint warnings — committed to singer-io/singer-python by bryantgray 2 years ago
- Remove Murano This removes all of the Murano code that is deprecated in the 3.0.0 release. When rally pulls in Murano it also pulls python-muranoclient that depends on murano-pkg-check that is old c... — committed to openstack/rally-openstack by tobias-urdin 2 years ago
I wonder how much money and wasted CPU this is costing the world today on broken builds. I really hope this gets rolled back 🙏
This is pretty bad and hitting a large amount of packages, please consider yanking this as soon as possible.
Hi @andy-maier, please contact the package developers and inform about the problem highlighted in the error message:
This has been deprecated in 2021.
Seeing this for other packages as well.
The only explanation I can understand here is that installing pip itself drags in the unpinned latest version of setuptools, and this is hard or impossible to override. The only good solution here is to yank 78.0.1 from Pypi.
This seems to have caused an unhelpful level of disruption for the sake of a hyphen. As a core build tool, it is to be expected that the newest version of the build tool would be used to install very old versions of packages, so expecting it to only interact with modern package versions seems like an error.
The change that introduced this needs to be rolled back - way too many package builds and installs are failing all over the place!
78.0.2 is out: https://pypi.org/project/setuptools/78.0.2/
Rather than generating a ton of work for everyone and trying to get devs to fix this in >12,000 places on GitHub alone (many of which will be in unmaintained projects), I would humbly suggest that one accepts this is part of the API and required for backwards compatibility, and simply allow it / silently rewrite it. I cannot see the benefit in forcing a tonne of busy-work on other projects given both the lack of benefit and the inability to force updates to unmaintained projects.
Amazed at the level of breakage that happened here. 😂
This is history 👏🏽 😂
While its great that they’re going to fix this, how has this not been yanked yet?
@abravalheri Would you be able to provide clarity to the community here if you expect to roll this change back or not? We need to know if we should invest time in implementing a workaround.
Hi, I would strongly encourage the team to roll this back. This is breaking many packages. It also breaks Neptune through bravado-core, which is a dinosaur package that will probably never switch to an underscore. Left-pad moment 😃
This is 1000% what should have been queried for before making this change
This change is breaking a small, but significant portion of the Python library ecosystem, some of which are no longer maintained. Those libraries still run without issue, but now cannot be installed because of this change.
We have seen breakages in
fpdfandpytest-watch, in addition to those listed above.In testing, there was no visible
DeprecationWarningwhen installing frompip, meaning this change was invisible until it started breaking builds. A build system needs to be extremely careful with breaking changes like these. This needs to be reverted and reimplemented in a way that does not break old packages.Roughly 12 thousand repositories are affected just for
description-field: https://github.com/search?q=path%3A**%2Fsetup.cfg+"description-file"&type=code&ref=advsearchIt might be appropriate to yank the new version from PyPi in the interim.
You are absolutely correct, there was. We bumped the major version of setuptools from 77.0.3 to 78.0.0 to indicate an intentional breaking change, namely the removal for the deprecated handling of
--separated options insetup.cfg. They are described in this section: https://setuptools.pypa.io/en/latest/history.html#deprecations-and-removals.is this the python
left-padmoment?@abravalheri I can do that, but I do not understand why this issue comes up now, just 30min after 78.0.1 was released, and the same packages worked before. There must have been some change that “activated” that check only now.
Seems their plan all along will be to roll this back after alerting everyone of the problem https://github.com/pypa/setuptools/issues/4864#issuecomment-2709033980
Hello, having the same issue with https://pypi.org/project/opentracing/ where the project is not maintained anymore, we’ll try to get rid of it, but would be great to keep it backward compatible, if possible 😃
Thank you!
This is what helped me using
poetryin a docker setup:Update: Downgrading setuptools to below 78.0.1 does not make the error go away. That is consistent with @abravalheri stating that the check was introduced in 2021. So it must be something else. Any idea where to look?
I understand how you have had this deprecation warning since 2021, but I’m not sure the rest of the community has caught on to this change. Others have mentioned this happens in a package that is no longer maintained but we are experiencing this in a repo that is still active https://github.com/olirice/alembic_utils/issues/153
Is it reasonable to force everyone to make this change by breaking their application builds?
Is your recommendation that maintainers of all affected libraries release new patch versions for every minor version they’ve released in recent years to remove a dash? New versions of libraries should use current best practices, but a core build tool needs to be able to interact with old versions as well. The setuptools team do great work for the community, but they should be encouraged to avoid predictable (and predicted) large-scale disruption.
This is getting reverted. See https://github.com/pypa/setuptools/pull/4911
On “Removing 4 year long deprecated functionality during a major release should be a no brainer.”:
While setuptools appears to use semantic versioning because its versions have three parts, its first version part cannot be considered “major” in the way semantic versioning describes it, i.e. there is an incompatibility for its users. Or does the setuptools team mean to have 10000’s of package owners check for incompatibilities on average 10 times each year?
Has anyone using
uv syncfound a workaround? Withconstraint-dependenciesandbuild-constraint-dependenciesboth specifyingsetuptools==77.0.3, I’m still seeing the build using 78.0.1:(Warnings for very old skipped files elided.)
This is in a workspace.
uv syncdoesn’t accept the--build-constraintCLI arg.This is obvious. But the job of us - users of the open-source software done by volunteers is to help them succeed. If every single person who commented “you should not do it” would go to their dependencies - like I did - and tell them “improve” - and spend their time and energy in leading that, this is the only way how to drive down those 12.000 packages down to maybe a handful in a year or two.
And we can help. Those people in the packaging team working on it do all the contributions here in their free time - away of their families and day jobs. They work nights and weekends and sometimes, it happens that they make mistakes.
Are we all responsible enough, and can raise up to the task of helping those people to succeed in the future and thank them for the hard work they do?
Or all that we are capable of is to tell what THEY should do and how much they screwed up? Unlike rolling your own sleeves up and spending your own time on making things better, that is not helpful nor constructive.
I would really love if that experience turns into all those 100s of people who looked and watched that issue into agents of change who will help the whole ecosystem to modernize their build systems.
Are you up to the challenge? I am and i am going to do so. I hope many of other people here will do it as well.
Please prioritize backwards compatibility, we have a pretty complex build environment and this has similarly killed our morning as we try to figure out how to pin the old version in part of our requirements generation system.
I don’t argue that preventing predictable disruption is a good idea. I 100% agree with you. But that does not preclude spending effort by people like you and others to:
help fixing the latest released versions (in our case kylinpy latest 2.8.4 version had this issue)
you - yourself and your project - spending time and effort to make sure to use (and likely lower-bind the dependencies and work with others to do the same). If we all work in that direction and spend our engineering time on it, and help others, there is a chance that in a year or two we will be able to get rid of the deprecations.
What I am asking for is that it’s quite a good idea by everyone affected here not to “negatively” complain on things but “positively” improve the world around them. Clean your own beach.
This is a theme that I am actively working with Alpha-Omega team on “Airflow Beach Cleaning” project - you can watch the Airflow Summit Keynotte from October 2024 where we explain it with Michael Winser what we are trying to do and encourage others: https://www.youtube.com/watch?v=f6gfoVJXWEE
Yes, This is a long and multi-year effort, but if we all, as community, get used to helping others who we depend on and start doing it regularly and do it at scale, problems like that will be far less often and we will be able to leave the world quite a bit better than when we started.
BTW. It’s not always easy to predict such things - in case of
kylinpyI found out that the fix has already been merged and 2.8.5 release was planned some time ago with the fix but it has never been released. And now, when I reached out, it turns out that there are people interested in doing so. Let’s see what happens with it.But if you look at my talk - just reaching out is already a good time spent by you and others because you might find out that when you have such problematic dependency, you can do one of three F’s
kylinprovider and the whole dependencyThis is what I propose here:
Anyone from any project can do it. And it’s pretty much our moral obligation towards all our dependencies that we build our value on, to make sure that we also care about them (because your security and availabilty depends on your engagement with your dependencies). And yes. You can change the world. We already did in quite a number of cases in Airflow Beach Cleaning - I am going to share more and more findings and results of it as it progresses.
This highlights how as a community we need to be pinning our dependencies and have more isolated builds in the future. Each package should be able to pin their compatible version of build dependencies independent of those building them. This way “dinosaur” packages would have been pinned to an older version.
For what it’s worth, whether or not the package that causes the failure is printed largely depends on the build environment. When building with uv for example, this is the output:
Which is clear enough. In our case, we could just pin
setuptools<78using the build constraints mechanisms thatuvprovides. That resolved the issue easily enough. I appreciate that that is rather build tool specific.@abravalheri, I sympathise strongly with you here. Removing 4 year long deprecated functionality during a major release should be a no brainer. It’s been on our todo list to pin our build environment for almost a year, this was the push we needed to finally implement that.
While library maintainers should aim to do things properly for newer versions, part of the job of a build tool is to be able to work with many package versions, old and new.
For pip in Docker, here’s the workaround that worked for us
INSTEAD of
use this
Same issue with Airflow CI: https://github.com/apache/airflow/pull/48210
Failed to build. We got rid of json-merge-path and had to pin setuptools and disable builds isolation for pyspark and pykylin. I am going to open issues against those repos (in the unlikely event someone has not done it already of course) but yeah - it seems it’s quite a big of mayhem with this release of
setuptools- unless that was deliberate test of course - so I guess it should be reverted.Happy to help in bringing awareness to the dependencies affected, however.
And I stronly suggest anyone who is complaining what SHOULD be done, to do the same - this is the only way we can help packaging team to get rid of some old luggage they have to carry.
Please - anyone affected here - make the effort, and open issues to all the affected repos you found so that they fix it. Let’s crowsdource better future for Python.
We’re seeing this issue pop in in packages which aren’t using
setup.cfg, i.e. https://github.com/olirice/alembic_utilsSo, this isn’t as simple as “don’t use the deprecated way” since they aren’t at all…
EDIT: I think the exception message needs to state what package is causing the issue at a minimum
@raylu agreed. Not sure if here is the right place to discuss this, but I think this is exposing a key vulnerability in python build systems.
When building an app, a requirements.txt or simliar can easily pin exact versions of libraries in order to guarantee a level of stability. But for python libraries that serve as build backends (like setuptools, flit or hatch) - those aren’t subjected to any such pins. Ideally, build backends would specify a range like “build_tool>3.4,<=4.2”, but all it takes is for a single library maintainer to fail to do this, and an applications build process is theoretically unstable.
This comment thread is nearing 5000 comments, so clearly affected a whole bunch of people. I don’t think we’d see anywhere near this affect if setuptools was a standard library (rather than a build backend) - it’d just be a case of “fix the deprecation or pin”.
It’s easy to get frustrated at maintainers of tools like setuptools when we percieve that as being the “thing that’s broken my build process” - but all things considered, they did a pretty good job communicating on a deprecation (several years), and then pragmatically rolling back when things didn’t work.
The honus is maybe on tools like pip/uv, and us as a community, to work towards something better than just requiring tools like setuptools have 100% backwards compatibility. Especially if we want maintainers of things like this to be able to continually improve their libraries without burning out.
This issue would have been a lot easier to resolve if the error message on install-time just pointed out the name of the package that causes an error. Right now it seems like you can only use guessing to determine this?
@abravalheri No, specifying a constraints file for pip does not work, neither with
-cnor withPIP_CONSTRAINTenv var. That constraints file is what pip uses for its install, but not in its isolated build env.It seems that “uv” allows specifying constraints for its isolated build env. I have not yet found such an option for pip.
Update: I stand corrected: Using
-cwith pip does not work, but using thePIP_CONSTRAINTenv var does work:With
pip-build-constraints.txtcontaining:The issue pops up for any package that still uses the deprecated
description-fileand doesn’t have a version pin on setuptools as a build dependency.Caching may render the issue hard to reproduce: if you or your CI agent locally has access to a pre-built version of the affected package, the issue won’t pop-up.
There is only one option right now for fresh installs: install an earlier version of setuptools and disable build isolation.
This affects very mainstream packages, such as
dash(dependency onstringcase).This does not apply 100% here. We have been fighting an issue similar to this all day from
distutilswhen this one happened.If you are using Poetry, you cannot pin
setuptools, as far as we can tell poetry will download and use it’s own version of setuptools to install the packages, regardless of what version of setuptools you pin.Therefore if you use poetry, you are using the latest setuptools when installing your packages, E.G you cant get away from this bug.
We have tried pinning setuptools as a package, and installing setuptools as a specific version on the host, poetry will use it’s own and therefore will use this latest version
78.0.1Also breaking fpdf. Agree the team should roll this back.
Our system broke here as well. Downgrading to 77.0.3 did not fix it for us.
the same as stringcase
We’re seeing this issue as well, and we use
poetryas our package manager, which doesn’t allow configuring or restraining the version of setuptools to use for builds, likepipdoes with--no-build-isolation.requests is also affected:
https://github.com/psf/requests/blob/v2.32.3/setup.cfg
UPDATE: The problem with requests was detected by setuptools’ CI, #4909 .
We had the same issue with pysdf which is also no longer maintained, but used by us. Downgrading to
setuptools==77.0.3seems to fix the issue for usHad to manually run
pip install setuptools==77.0.3beforepip install -r requirements.txtto make it work thoughIs there any recommendation from anyone for library maintainers with dependencies on broken packages? Spotted this issue on an attempted install of a package that isn’t using setup-tools, but is installing via subdependencies a package that is using setuptools.
Anyone installing that package would get an error, but it seems a little out of scope to start including requirements “setuptools<76.1.0” solely to affect the builds of subdependencies.
breaking alembic-utils
Breaking pyspark also (all of them)
Adding this to pyproject.toml works
uv pip installprovides a--build-constraintthat lets you provide a constraints file withsetuptools<78. This allows non-conforming packages to build correctly even with the isolated build backend. I don’t know how to constrain the build backend forpip install+setuptools, does that respect thePIP_CONSTRAINTenv var?And … just to let everyone know: this works - and I am going to create a PR fixing
kylinpyand thekylinpymaintainers are very responsive on that one so far - https://github.com/Kyligence/kylinpy/issues/57 also I am going to follow up with all the other (we also havepyjenkinsandmysqland help them to fix their projects.And yes anyone else can also change and improve the world and go to their dependencies they saw failing and help to fix them. Maybe we can all show the power of Python community and - for a change - help to fix things rather than just complain?
We updated our poetry build command to
poetry run pip install setuptools==75.7.0 && poetry run pip install --no-build-isolation alembic_utils==0.8.3 && poetry install --no-interaction --no-rootfor the time being and it’s passing.alembic_utilsis the one that breaks for us. you can replace it by the packages that break for you. YMMVcc @lukehsiao
@acompa @luisiacc Did you just do this before running any
poetrycommands? Or before even installingpoetry. Tried this and no luck here yet.It looks like some pip builds can fail even with a downgraded setuptools if you don’t specify
--no-build-isolation. With that flag I was able to work around the issue and install packages.To figure out which package with
pip, you can run with-vvvand look for the package name in the name of the temporary directory it prints:Breaking docx2txt
Middleware is also affected. I thought my buggy code broke the issue, been debugging for an hours. @jayantbh found this thread and we moved to 77.0.3, still breaking.
any solution for a library that does not have a setup.cfg but utilizes the setup.py approach such as alembic-utils?
**Workaround for Poetry: **
poetry exporttorequirements.txt, thenuv pip installwithUV_BUILD_CONSTRAINT=build-requirements.txtset (can also use CLI variable--build-constraint), contents of that file beingsetuptools~=77.0.0Yes, not really a clean workaround, but it’s enough to at least be decent. Poetry does not seem to provide a mechanism for build-time dependency pinning, so a shim to another format is all we’ve got.
I now seem to have found a reliable way to reproduce the error and to circumvent it. It was the pip cache that needed to be cleared, because a prior install left the generated wheel file there, and that is used by pip with priority, if present.
So the error should always show up for packages that do not have a wheel file for the target environment (and use dashed properties in setup.cfg). For example, “ansible-vault” has only a source tarball and no wheel archive at all.
Commands to reproduce the error:
Commands to circumvent the error:
Does anyone know how to influence the setuptools version that is used by pip in its build-isolated environment?
Both we and sentry i see (https://github.com/getsentry/sentry-python/pull/4192/files) are trying all the suggestions presented here and none of them work.
As long as none of us know why our setups are failing it’s hard to prescribe the correct solution. Both pinning with a constraints file and other versions of pinning setuptools has no effect.
@potiuk
adamcallison is not talking about the latest versions. they are talking about the ability to install older versions. requoting with emphasis:
the setuptools breakage in this issue affects more than the latest version
@webknjaz
none of the advice in that post is relevant to this issue. see https://github.com/pypa/setuptools/issues/4910#issuecomment-2749037152 to understand why
Adding this to pyproject.toml for poetry builds fixed for me:
Yeah, I don’t think there’s actually an end-user mitigation here as
pip-compilew/ build isolation will execute hooks in a custom venv built around that package’s requirements that you don’t have control over AFAICT. I can’t seem to get around that with a constraints file or anything.Same issue is faced when installing pyspark or any other package for that matter. Please revert the changes. Thanks
How can I find out which package has the broken field? The error message mentions the field and the section but not the package.
I’ve had success with
uvand updating pyproject.toml toFurther to @maxfenv 's comment - for anybody using
uv, this can be avoided by specifying a build constraint as below:Or set a
UV_BUILD_CONSTRAINTenvironment variable - for example, to set this in a Dockerfile, you could do:Where constraints.txt looks like:
Hashes are not required, it should still work with a more minimal configuration like this:
We have the same issue with multiple third-party packages. Downgrading to
77.0.3produces the same error.We are circumventing this by excluding setuptools 78.0.1.
Update: That was a thought, but it does not work reliably. See the discussion below.
For people using uv. You can set a build constraint this way:
More details and alternatives are available here for you to read: https://github.com/astral-sh/uv/issues/12440
We don’t install things really with
poetry, we do:And then we install
requirements.txtwith pip normally.@MerlinR We use
poetryand, for what it’s worth, a version of this suggestion has also unblocked us.That said I agree that this isn’t as simple as a dependency pin. It’s a dependency pin and poetry fixing this unaddressed issue!
Let the team breathe I guess they’re already reverting the changes! ☮ Thanks
This also breaks swifter (https://github.com/jmcarpenter2/swifter)
When using
poetry, there’s no good way to specify whichsetuptoolsversion used. See more details here: https://github.com/python-poetry/poetry/issues/4511The only workaround that worked for me was running this before
poetry install(took from the above issue)I don’t know why that works at all, but it does.
Given there’s no consistent way to override this behavior and the amount of different package management solutions available for python at this point, this seems like an unnecessary nuisance rather than an useful update for users.
also breaking drain3, see https://github.com/logpai/Drain3/issues/106
@andy-maier does PIP_CONSTAINT work for you? (or the mechanism pointed out by https://github.com/pypa/setuptools/issues/4910#issuecomment-2748686922 for uv).
@raylu that’s not true. pip-tools can be used to generate a constraint file for the (transitive) build deps specifically. I personally merged a pull request implementing it quite a while ago: https://pip-tools.rtfd.io/en/stable/#maximizing-reproducibility.
Using that is a bit unobvious, but setting the
PIP_CONSTRAINTenvironment variable applies to said ephemeral build environments that are populated by apip installin a subprocess that understands that env var just fine (it’s passing it on CLI via--constraintthat is problematic).There are too many comments here but in case people didn’t mention this earlier, make sure to read this blog post on handling dependency upgrades reliably: https://hynek.me/articles/semver-will-not-save-you/#taking-responsibility.
did you mean
setuptools<78?Haven’t found working solution for poetry, breaking packages:
pyspark==3.5.1.Thanks, we have briefly tried this with no luck, but going to test it further as breaks about 15 of our apps.
@adiberk, I’m not certain myself, but I think the things you specified work for building the project itself, but for dependent packages being built from source, uv will install that package’s build-dependencies in a isolated venv, so your constraints will be ignored
Details here: https://docs.astral.sh/uv/concepts/projects/config/#build-isolation
I think
no-build-isolation = truemight be the fix you want, or maybeno-build-isolation-package = [...]to just disable the packages failing to buildHas anyone figured out a workaround when using uv sync (I believe there is a workaround for uv pip). build-constraints don’t seem to work in pyproject for me I tried both of these
A note: I am baffled that this hasn’t been rolled back already.
Breaking pytest-watch too
Same with ecs-deploy
Haven’t found working solution for poetry, breaking packages:
django-fast-update,django-computedfields,django-extensionsJust for posterity, this behavior change was intentionally introduced by https://github.com/pypa/setuptools/pull/4870
The ‘build constraints mechanism’ to which I refer above is documented here, for what it’s worth: https://docs.astral.sh/uv/pip/compile/#adding-build-constraints