setuptools: [BUG] Version 78.0.1 breaks install of ansible-vault package

setuptools version

78.0.1

Python version

3.11

OS

Ubuntu

Additional environment information

No response

Description

When installing the “ansible-vault” package, setuptools raises this exception:


$ pip install ansible-vault --upgrade
Collecting ansible-vault
  Using cached ansible-vault-2.1.0.tar.gz (3.5 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [34 lines of output]
      Traceback (most recent call last):
        File "/Users/maiera/virtualenvs/vpcauto311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
          main()
        File "/Users/maiera/virtualenvs/vpcauto311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
          json_out["return_val"] = hook(**hook_input["kwargs"])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/maiera/virtualenvs/vpcauto311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 522, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
        File "<string>", line 11, in <module>
        File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 116, in setup
          _install_setup_requires(attrs)
        File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 87, in _install_setup_requires
          dist.parse_config_files(ignore_option_errors=True)
        File "/Users/maiera/virtualenvs/vpcauto311/lib/python3.11/site-packages/_virtualenv.py", line 22, in parse_config_files
          result = old_parse_config_files(self, *args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 730, in parse_config_files
          self._parse_config_files(filenames=inifiles)
        File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 599, in _parse_config_files
          opt = self._enforce_underscore(opt, section)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/tg/fy7jbcg96lv1_hvlmcv263kr0000gn/T/pip-build-env-1hnvit19/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 629, in _enforce_underscore
          raise InvalidConfigError(
      setuptools.errors.InvalidConfigError: Invalid dash-separated key 'description-file' in 'metadata' (setup.cfg), please use the underscore name 'description_file' instead.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Expected behavior

ansible-vault has not changed since 2021 and should install as before.

How to Reproduce

  1. Create and activate a fresh Pyhton virtual env, using Python 3.11:
mkvirtualenv test -p python3.11
  1. Install version 78.0.1 of setuptools
pip install setuptools==78.0.1
  1. Install ansible-vault
pip install ansible-vault

Output

See above

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 221
  • Comments: 120 (14 by maintainers)

Commits related to this issue

Most upvoted comments

I wonder how much money and wasted CPU this is costing the world today on broken builds. I really hope this gets rolled back 🙏

This is pretty bad and hitting a large amount of packages, please consider yanking this as soon as possible.

Hi @andy-maier, please contact the package developers and inform about the problem highlighted in the error message:

  setuptools.errors.InvalidConfigError: Invalid dash-separated key 'description-file' in 'metadata' (setup.cfg), please use the underscore name 'description_file' instead.

This has been deprecated in 2021.

Seeing this for other packages as well.

The only explanation I can understand here is that installing pip itself drags in the unpinned latest version of setuptools, and this is hard or impossible to override. The only good solution here is to yank 78.0.1 from Pypi.

This seems to have caused an unhelpful level of disruption for the sake of a hyphen. As a core build tool, it is to be expected that the newest version of the build tool would be used to install very old versions of packages, so expecting it to only interact with modern package versions seems like an error.

The change that introduced this needs to be rolled back - way too many package builds and installs are failing all over the place!

open issues to all the affected repos you found so that they fix it

Rather than generating a ton of work for everyone and trying to get devs to fix this in >12,000 places on GitHub alone (many of which will be in unmaintained projects), I would humbly suggest that one accepts this is part of the API and required for backwards compatibility, and simply allow it / silently rewrite it. I cannot see the benefit in forcing a tonne of busy-work on other projects given both the lack of benefit and the inability to force updates to unmaintained projects.

Amazed at the level of breakage that happened here. 😂

This is history 👏🏽 😂

While its great that they’re going to fix this, how has this not been yanked yet?

@abravalheri Would you be able to provide clarity to the community here if you expect to roll this change back or not? We need to know if we should invest time in implementing a workaround.

Hi, I would strongly encourage the team to roll this back. This is breaking many packages. It also breaks Neptune through bravado-core, which is a dinosaur package that will probably never switch to an underscore. Left-pad moment 😃

Roughly 12 thousand repositories are affected just for description-field: https://github.com/search?q=path%3A**%2Fsetup.cfg+"description-file"&type=code&ref=advsearch

This is 1000% what should have been queried for before making this change

Hi @andy-maier, please contact the package developers and inform about the problem highlighted in the error message:

  setuptools.errors.InvalidConfigError: Invalid dash-separated key 'description-file' in 'metadata' (setup.cfg), please use the underscore name 'description_file' instead.

This has been deprecated in 2021.

This change is breaking a small, but significant portion of the Python library ecosystem, some of which are no longer maintained. Those libraries still run without issue, but now cannot be installed because of this change.

We have seen breakages in fpdf and pytest-watch, in addition to those listed above.

In testing, there was no visible DeprecationWarning when installing from pip, meaning this change was invisible until it started breaking builds. A build system needs to be extremely careful with breaking changes like these. This needs to be reverted and reimplemented in a way that does not break old packages.

Roughly 12 thousand repositories are affected just for description-field: https://github.com/search?q=path%3A**%2Fsetup.cfg+"description-file"&type=code&ref=advsearch

It might be appropriate to yank the new version from PyPi in the interim.

There must have been some change that “activated” that check only now.

You are absolutely correct, there was. We bumped the major version of setuptools from 77.0.3 to 78.0.0 to indicate an intentional breaking change, namely the removal for the deprecated handling of --separated options in setup.cfg. They are described in this section: https://setuptools.pypa.io/en/latest/history.html#deprecations-and-removals.

is this the python left-pad moment?

@abravalheri I can do that, but I do not understand why this issue comes up now, just 30min after 78.0.1 was released, and the same packages worked before. There must have been some change that “activated” that check only now.

Seems their plan all along will be to roll this back after alerting everyone of the problem https://github.com/pypa/setuptools/issues/4864#issuecomment-2709033980

That said, I’m leaning toward option 1, get some signal out there to non-compliant projects, and then if the disruption is large, rollback and fallback to option 2 (now with the affected projects having visibility to the issue).

Hello, having the same issue with https://pypi.org/project/opentracing/ where the project is not maintained anymore, we’ll try to get rid of it, but would be great to keep it backward compatible, if possible 😃

Thank you!

This is what helped me using poetry in a docker setup:

# Create a constraints file to limit setuptools version
RUN echo "setuptools<78" > /constraints.txt

# Set the PIP_CONSTRAINT environment variable to the constraints file
ENV PIP_CONSTRAINT=/constraints.txt

Update: Downgrading setuptools to below 78.0.1 does not make the error go away. That is consistent with @abravalheri stating that the check was introduced in 2021. So it must be something else. Any idea where to look?

I understand how you have had this deprecation warning since 2021, but I’m not sure the rest of the community has caught on to this change. Others have mentioned this happens in a package that is no longer maintained but we are experiencing this in a repo that is still active https://github.com/olirice/alembic_utils/issues/153

Is it reasonable to force everyone to make this change by breaking their application builds?

Are you up to the challenge? I am and i am going to do so. I hope many of other people here will do it as well.

And … just to let everyone know: this works - and I am going to create a PR fixing kylinpy and the kylinpy maintainers are very responsive on that one so far - Kyligence/kylinpy#57 also I am going to follow up with all the other (we also have pyjenkins and mysql and help them to fix their projects.

And yes anyone else can also change and improve the world and go to their dependencies they saw failing and help to fix them. Maybe we can all show the power of Python community and - for a change - help to fix things rather than just complain?

Is your recommendation that maintainers of all affected libraries release new patch versions for every minor version they’ve released in recent years to remove a dash? New versions of libraries should use current best practices, but a core build tool needs to be able to interact with old versions as well. The setuptools team do great work for the community, but they should be encouraged to avoid predictable (and predicted) large-scale disruption.

On “Removing 4 year long deprecated functionality during a major release should be a no brainer.”:

While setuptools appears to use semantic versioning because its versions have three parts, its first version part cannot be considered “major” in the way semantic versioning describes it, i.e. there is an incompatibility for its users. Or does the setuptools team mean to have 10000’s of package owners check for incompatibilities on average 10 times each year?

Has anyone using uv sync found a workaround? With constraint-dependencies and build-constraint-dependencies both specifying setuptools==77.0.3, I’m still seeing the build using 78.0.1:

   Building pyspark==3.5.3
DEBUG Building: pyspark==3.5.3
DEBUG Assessing Python executable as base candidate: /home/alex/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/bin/python3.12
DEBUG Using base executable for virtual environment: /home/alex/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/bin/python3.12
DEBUG Ignoring empty directory
DEBUG Resolving build requirements
DEBUG Solving with installed Python version: 3.12.8
DEBUG Solving with target Python version: >=3.12.8
DEBUG Adding direct dependency: setuptools>=40.8.0
...
DEBUG Searching for a compatible version of setuptools (>=40.8.0)
DEBUG Selecting: setuptools==78.0.1 [compatible] (setuptools-78.0.1-py3-none-any.whl)

(Warnings for very old skipped files elided.)

This is in a workspace. uv sync doesn’t accept the --build-constraint CLI arg.

While library maintainers should aim to do things properly for newer versions, part of the job of a build tool is to be able to work with many package versions, old and new.

This is obvious. But the job of us - users of the open-source software done by volunteers is to help them succeed. If every single person who commented “you should not do it” would go to their dependencies - like I did - and tell them “improve” - and spend their time and energy in leading that, this is the only way how to drive down those 12.000 packages down to maybe a handful in a year or two.

And we can help. Those people in the packaging team working on it do all the contributions here in their free time - away of their families and day jobs. They work nights and weekends and sometimes, it happens that they make mistakes.

Are we all responsible enough, and can raise up to the task of helping those people to succeed in the future and thank them for the hard work they do?

Or all that we are capable of is to tell what THEY should do and how much they screwed up? Unlike rolling your own sleeves up and spending your own time on making things better, that is not helpful nor constructive.

I would really love if that experience turns into all those 100s of people who looked and watched that issue into agents of change who will help the whole ecosystem to modernize their build systems.

Are you up to the challenge? I am and i am going to do so. I hope many of other people here will do it as well.

Please prioritize backwards compatibility, we have a pretty complex build environment and this has similarly killed our morning as we try to figure out how to pin the old version in part of our requirements generation system.

Is your recommendation that maintainers of all affected libraries release new patch versions for every minor version they’ve released in recent years to remove a dash? New versions of libraries should use current best practices, but a core build tool needs to be able to interact with old versions as well. The setuptools team do great work for the community, but they should be encouraged to avoid predictable (and predicted) large-scale disruption.

I don’t argue that preventing predictable disruption is a good idea. I 100% agree with you. But that does not preclude spending effort by people like you and others to:

  1. help fixing the latest released versions (in our case kylinpy latest 2.8.4 version had this issue)

  2. you - yourself and your project - spending time and effort to make sure to use (and likely lower-bind the dependencies and work with others to do the same). If we all work in that direction and spend our engineering time on it, and help others, there is a chance that in a year or two we will be able to get rid of the deprecations.

What I am asking for is that it’s quite a good idea by everyone affected here not to “negatively” complain on things but “positively” improve the world around them. Clean your own beach.

This is a theme that I am actively working with Alpha-Omega team on “Airflow Beach Cleaning” project - you can watch the Airflow Summit Keynotte from October 2024 where we explain it with Michael Winser what we are trying to do and encourage others: https://www.youtube.com/watch?v=f6gfoVJXWEE

Yes, This is a long and multi-year effort, but if we all, as community, get used to helping others who we depend on and start doing it regularly and do it at scale, problems like that will be far less often and we will be able to leave the world quite a bit better than when we started.

BTW. It’s not always easy to predict such things - in case of kylinpy I found out that the fix has already been merged and 2.8.5 release was planned some time ago with the fix but it has never been released. And now, when I reached out, it turns out that there are people interested in doing so. Let’s see what happens with it.

But if you look at my talk - just reaching out is already a good time spent by you and others because you might find out that when you have such problematic dependency, you can do one of three F’s

  1. Help to (F)ix it - this is what I attempt to do with kylinpy
  2. (F) fork it - this is essentially what I’ve done by vendoring in 70 lines of code we needed from json-merge-patch - which was last released in 2017 https://github.com/apache/airflow/pull/48210 - this way the problem is gone for us and we have one-less-dependency that can break our user’s installation in the future
  3. (F)orego it - if we find that kylin community is not responsive/ not really interested, we will simply remove kylin provider and the whole dependency

This is what I propose here:

  • reach out and make sure you use the latest deps
  • help to fix problems
  • decide if you want to continue using the dependency if your reach out brought silence

Anyone from any project can do it. And it’s pretty much our moral obligation towards all our dependencies that we build our value on, to make sure that we also care about them (because your security and availabilty depends on your engagement with your dependencies). And yes. You can change the world. We already did in quite a number of cases in Airflow Beach Cleaning - I am going to share more and more findings and results of it as it progresses.

This highlights how as a community we need to be pinning our dependencies and have more isolated builds in the future. Each package should be able to pin their compatible version of build dependencies independent of those building them. This way “dinosaur” packages would have been pinned to an older version.

For what it’s worth, whether or not the package that causes the failure is printed largely depends on the build environment. When building with uv for example, this is the output:

Creating virtual environment at: /venv
11.97 Using Python 3.12.3 environment at: /venv
18.66   × Failed to build `fortranformat==2.0.0`
18.66   ├─▶ The build backend returned an error
18.66   ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit
18.66       status: 1)
18.66
18.66       [stderr]
18.66       Traceback (most recent call last):
18.66         File "<string>", line 14, in <module>
18.66         File
18.66       "/root/.cache/uv/builds-v0/.tmpjkIDMC/lib/python3.12/site-packages/setuptools/build_meta.py",
18.66       line 334, in get_requires_for_build_wheel
18.66           return self._get_build_requires(config_settings, requirements=[])
18.66                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
18.66         File
18.66       "/root/.cache/uv/builds-v0/.tmpjkIDMC/lib/python3.12/site-packages/setuptools/build_meta.py",
18.66       line 304, in _get_build_requires
18.66           self.run_setup()
18.66         File
18.66       "/root/.cache/uv/builds-v0/.tmpjkIDMC/lib/python3.12/site-packages/setuptools/build_meta.py",
18.66       line 522, in run_setup
18.66           super().run_setup(setup_script=setup_script)
18.66         File
18.66       "/root/.cache/uv/builds-v0/.tmpjkIDMC/lib/python3.12/site-packages/setuptools/build_meta.py",
18.66       line 320, in run_setup
18.66           exec(code, locals())
18.66         File "<string>", line 9, in <module>
18.66         File
18.66       "/root/.cache/uv/builds-v0/.tmpjkIDMC/lib/python3.12/site-packages/setuptools/__init__.py",
18.66       line 116, in setup
18.66           _install_setup_requires(attrs)
18.66         File
18.66       "/root/.cache/uv/builds-v0/.tmpjkIDMC/lib/python3.12/site-packages/setuptools/__init__.py",
18.66       line 87, in _install_setup_requires
18.66           dist.parse_config_files(ignore_option_errors=True)
18.66         File
18.66       "/root/.cache/uv/builds-v0/.tmpjkIDMC/lib/python3.12/site-packages/_virtualenv.py",
18.66       line 20, in parse_config_files
18.66           result = old_parse_config_files(self, *args, **kwargs)
18.66                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
18.66         File
18.66       "/root/.cache/uv/builds-v0/.tmpjkIDMC/lib/python3.12/site-packages/setuptools/dist.py",
18.66       line 730, in parse_config_files
18.66           self._parse_config_files(filenames=inifiles)
18.66         File
18.66       "/root/.cache/uv/builds-v0/.tmpjkIDMC/lib/python3.12/site-packages/setuptools/dist.py",
18.66       line 599, in _parse_config_files
18.66           opt = self._enforce_underscore(opt, section)
18.66                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
18.66         File
18.66       "/root/.cache/uv/builds-v0/.tmpjkIDMC/lib/python3.12/site-packages/setuptools/dist.py",
18.66       line 629, in _enforce_underscore
18.66           raise InvalidConfigError(
18.66       setuptools.errors.InvalidConfigError: Invalid dash-separated key
18.66       'decsription-file' in 'metadata' (setup.cfg), please use the underscore
18.66       name 'decsription_file' instead.
18.66
18.66       hint: This usually indicates a problem with the package or the build
18.66       environment.

Which is clear enough. In our case, we could just pin setuptools<78 using the build constraints mechanisms that uv provides. That resolved the issue easily enough. I appreciate that that is rather build tool specific.

@abravalheri, I sympathise strongly with you here. Removing 4 year long deprecated functionality during a major release should be a no brainer. It’s been on our todo list to pin our build environment for almost a year, this was the push we needed to finally implement that.

Same issue with Airflow CI: apache/airflow#48210

  • json-merge-patch
  • kylinpy
  • pyspark

Failed to build. We got rid of json-merge-path and had to pin setuptools and disable builds isolation for pyspark and pykylin. I am going to open issues against those repos (in the unlikely event someone has not done it already of course) but yeah - it seems it’s quite a big of mayhem with this release of setuptools - unless that was deliberate test of course - so I guess it should be reverted.

Happy to help in bringing awareness to the dependencies affected, however.

And I stronly suggest anyone who is complaining what SHOULD be done, to do the same - this is the only way we can help packaging team to get rid of some old luggage they have to carry.

Please - anyone affected here - make the effort, and open issues to all the affected repos you found so that they fix it. Let’s crowsdource better future for Python.

While library maintainers should aim to do things properly for newer versions, part of the job of a build tool is to be able to work with many package versions, old and new.

For pip in Docker, here’s the workaround that worked for us

INSTEAD of

RUN pip3 install --upgrade pip setuptool
RUN pip3 install -r requirements.txt --target ./

use this

RUN echo "setuptools<78.0.0" > /tmp/constraints.txt
RUN pip3 install --upgrade pip
ENV PIP_CONSTRAINT=/tmp/constraints.txt
RUN pip3 install -r requirements.txt --target ./

Same issue with Airflow CI: https://github.com/apache/airflow/pull/48210

  • json-merge-patch
  • kylinpy
  • pyspark

Failed to build. We got rid of json-merge-path and had to pin setuptools and disable builds isolation for pyspark and pykylin. I am going to open issues against those repos (in the unlikely event someone has not done it already of course) but yeah - it seems it’s quite a big of mayhem with this release of setuptools - unless that was deliberate test of course - so I guess it should be reverted.

Happy to help in bringing awareness to the dependencies affected, however.

And I stronly suggest anyone who is complaining what SHOULD be done, to do the same - this is the only way we can help packaging team to get rid of some old luggage they have to carry.

Please - anyone affected here - make the effort, and open issues to all the affected repos you found so that they fix it. Let’s crowsdource better future for Python.

We’re seeing this issue pop in in packages which aren’t using setup.cfg, i.e. https://github.com/olirice/alembic_utils

So, this isn’t as simple as “don’t use the deprecated way” since they aren’t at all…

EDIT: I think the exception message needs to state what package is causing the issue at a minimum

@raylu agreed. Not sure if here is the right place to discuss this, but I think this is exposing a key vulnerability in python build systems.

When building an app, a requirements.txt or simliar can easily pin exact versions of libraries in order to guarantee a level of stability. But for python libraries that serve as build backends (like setuptools, flit or hatch) - those aren’t subjected to any such pins. Ideally, build backends would specify a range like “build_tool>3.4,<=4.2”, but all it takes is for a single library maintainer to fail to do this, and an applications build process is theoretically unstable.

This comment thread is nearing 5000 comments, so clearly affected a whole bunch of people. I don’t think we’d see anywhere near this affect if setuptools was a standard library (rather than a build backend) - it’d just be a case of “fix the deprecation or pin”.

It’s easy to get frustrated at maintainers of tools like setuptools when we percieve that as being the “thing that’s broken my build process” - but all things considered, they did a pretty good job communicating on a deprecation (several years), and then pragmatically rolling back when things didn’t work.

The honus is maybe on tools like pip/uv, and us as a community, to work towards something better than just requiring tools like setuptools have 100% backwards compatibility. Especially if we want maintainers of things like this to be able to continually improve their libraries without burning out.

This issue would have been a lot easier to resolve if the error message on install-time just pointed out the name of the package that causes an error. Right now it seems like you can only use guessing to determine this?

@abravalheri No, specifying a constraints file for pip does not work, neither with -c nor with PIP_CONSTRAINT env var. That constraints file is what pip uses for its install, but not in its isolated build env.

It seems that “uv” allows specifying constraints for its isolated build env. I have not yet found such an option for pip.

Update: I stand corrected: Using -c with pip does not work, but using the PIP_CONSTRAINT env var does work:

With pip-build-constraints.txt containing:

setuptools<78.0.0
export PIP_CONSTRAINT=pip-build-constraints.txt 
mkvirtualenv test -p python3.11
pip install pip --upgrade
pip install setuptools --upgrade     # installs 77.0.3 because of the pip constraint
rm -rf ~/Library/Caches/pip   # this is macOS specific
pip uninstall ansible-vault -y; pip install ansible-vault     # no error

The issue pops up for any package that still uses the deprecated description-file and doesn’t have a version pin on setuptools as a build dependency.

Caching may render the issue hard to reproduce: if you or your CI agent locally has access to a pre-built version of the affected package, the issue won’t pop-up.

There is only one option right now for fresh installs: install an earlier version of setuptools and disable build isolation.

This affects very mainstream packages, such as dash (dependency on stringcase).

This highlights how as a community we need to be pinning our dependencies and have more isolated builds in the future. Each package should be able to pin their compatible version of build dependencies independent of those building them. This way “dinosaur” packages would have been pinned to an older version.

This does not apply 100% here. We have been fighting an issue similar to this all day from distutils when this one happened.

If you are using Poetry, you cannot pin setuptools, as far as we can tell poetry will download and use it’s own version of setuptools to install the packages, regardless of what version of setuptools you pin.

Therefore if you use poetry, you are using the latest setuptools when installing your packages, E.G you cant get away from this bug.

We have tried pinning setuptools as a package, and installing setuptools as a specific version on the host, poetry will use it’s own and therefore will use this latest version 78.0.1

Also breaking fpdf. Agree the team should roll this back.

Our system broke here as well. Downgrading to 77.0.3 did not fix it for us.

the same as stringcase

We’re seeing this issue as well, and we use poetry as our package manager, which doesn’t allow configuring or restraining the version of setuptools to use for builds, like pip does with --no-build-isolation.

requests is also affected:

setuptools.errors.InvalidConfigError: Invalid dash-separated key 'provides-extra' in 'metadata' (setup.cfg), please use the underscore name 'provides_extra' instead.

https://github.com/psf/requests/blob/v2.32.3/setup.cfg

UPDATE: The problem with requests was detected by setuptools’ CI, #4909 .

We had the same issue with pysdf which is also no longer maintained, but used by us. Downgrading to setuptools==77.0.3 seems to fix the issue for us

Had to manually run pip install setuptools==77.0.3 before pip install -r requirements.txt to make it work though

Is there any recommendation from anyone for library maintainers with dependencies on broken packages? Spotted this issue on an attempted install of a package that isn’t using setup-tools, but is installing via subdependencies a package that is using setuptools.

Anyone installing that package would get an error, but it seems a little out of scope to start including requirements “setuptools<76.1.0” solely to affect the builds of subdependencies.

breaking alembic-utils

Breaking pyspark also (all of them)

Adding this to pyproject.toml works

[tool.uv]
build-constraint-dependencies = ["setuptools==v77.0.3"]

uv pip install provides a --build-constraint that lets you provide a constraints file with setuptools<78. This allows non-conforming packages to build correctly even with the isolated build backend. I don’t know how to constrain the build backend for pip install + setuptools, does that respect the PIP_CONSTRAINT env var?

Are you up to the challenge? I am and i am going to do so. I hope many of other people here will do it as well.

And … just to let everyone know: this works - and I am going to create a PR fixing kylinpy and the kylinpy maintainers are very responsive on that one so far - https://github.com/Kyligence/kylinpy/issues/57 also I am going to follow up with all the other (we also have pyjenkins and mysql and help them to fix their projects.

And yes anyone else can also change and improve the world and go to their dependencies they saw failing and help to fix them. Maybe we can all show the power of Python community and - for a change - help to fix things rather than just complain?

We updated our poetry build command to poetry run pip install setuptools==75.7.0 && poetry run pip install --no-build-isolation alembic_utils==0.8.3 && poetry install --no-interaction --no-root for the time being and it’s passing.

alembic_utils is the one that breaks for us. you can replace it by the packages that break for you. YMMV

cc @lukehsiao

This is what helped me using poetry in a docker setup:

# Create a constraints file to limit setuptools version
RUN echo "setuptools<78" > /constraints.txt

# Set the PIP_CONSTRAINT environment variable to the constraints file
ENV PIP_CONSTRAINT=/constraints.txt

This worked for me

@acompa @luisiacc Did you just do this before running any poetry commands? Or before even installing poetry. Tried this and no luck here yet.

It looks like some pip builds can fail even with a downgraded setuptools if you don’t specify --no-build-isolation. With that flag I was able to work around the issue and install packages.

How can I find out which package has the broken field? The error message mentions the field and the section but not the package.

To figure out which package with pip, you can run with -vvv and look for the package name in the name of the temporary directory it prints:

Image

Middleware is also affected. I thought my buggy code broke the issue, been debugging for an hours. @jayantbh found this thread and we moved to 77.0.3, still breaking.

any solution for a library that does not have a setup.cfg but utilizes the setup.py approach such as alembic-utils?

import os
import re

from setuptools import find_packages, setup


def get_version(package):
    """
    Return package version as listed in `__version__` in `__init__.py`.
    """
    with open(os.path.join("src", package, "__init__.py")) as f:
        return re.search("__version__ = ['\"]([^'\"]+)['\"]", f.read()).group(1)


DEV_REQUIRES = [
    "black",
    "pylint",
    "pre-commit",
    "mypy",
    "psycopg2-binary",
    "pytest",
    "pytest-cov",
    "mkdocs",
]

setup(
    name="alembic_utils",
    version=get_version("alembic_utils"),
    author="Oliver Rice",
    author_email="oliver@oliverrice.com",
    license="MIT",
    description="A sqlalchemy/alembic extension for migrating procedures and views ",
    python_requires=">=3.7",
    packages=find_packages("src"),
    package_dir={"": "src"},
    install_requires=[
        "alembic>=1.9",
        "flupy",
        "parse>=1.8.4",
        "sqlalchemy>=1.4",
        "typing_extensions",
    ],
    extras_require={
        "dev": DEV_REQUIRES,
        "nvim": ["neovim", "python-language-server"],
        "docs": ["mkdocs", "pygments", "pymdown-extensions", "mkautodoc"],
    },
    package_data={"": ["py.typed"]},
    include_package_data=True,

)

**Workaround for Poetry: ** poetry export to requirements.txt, then uv pip install with UV_BUILD_CONSTRAINT=build-requirements.txt set (can also use CLI variable --build-constraint), contents of that file being setuptools~=77.0.0

Yes, not really a clean workaround, but it’s enough to at least be decent. Poetry does not seem to provide a mechanism for build-time dependency pinning, so a shim to another format is all we’ve got.

I now seem to have found a reliable way to reproduce the error and to circumvent it. It was the pip cache that needed to be cleared, because a prior install left the generated wheel file there, and that is used by pip with priority, if present.

So the error should always show up for packages that do not have a wheel file for the target environment (and use dashed properties in setup.cfg). For example, “ansible-vault” has only a source tarball and no wheel archive at all.

Commands to reproduce the error:

mkvirtualenv test -p python3.11
pip install pip --upgrade
pip install setuptools==78.0.1
rm -rf ~/Library/Caches/pip   # this is macOS specific
pip uninstall ansible-vault -y; pip install ansible-vault     # error shows up
pip uninstall ansible-vault -y; pip install --no-build-isolation ansible-vault     # error shows up

Commands to circumvent the error:

mkvirtualenv test -p python3.11
pip install pip --upgrade
pip install setuptools==77.0.3
rm -rf ~/Library/Caches/pip   # this is macOS specific
pip uninstall ansible-vault -y; pip install ansible-vault     # error shows up
pip uninstall ansible-vault -y; pip install --no-build-isolation ansible-vault      # no error

Does anyone know how to influence the setuptools version that is used by pip in its build-isolated environment?

Both we and sentry i see (https://github.com/getsentry/sentry-python/pull/4192/files) are trying all the suggestions presented here and none of them work.

As long as none of us know why our setups are failing it’s hard to prescribe the correct solution. Both pinning with a constraints file and other versions of pinning setuptools has no effect.

@potiuk

help fixing the latest released versions

adamcallison is not talking about the latest versions. they are talking about the ability to install older versions. requoting with emphasis:

Is your recommendation that maintainers of all affected libraries release new patch versions for every minor version they’ve released in recent years to remove a dash?

the setuptools breakage in this issue affects more than the latest version


@webknjaz

There are too many comments here but in case people didn’t mention this earlier, make sure to read this blog post

none of the advice in that post is relevant to this issue. see https://github.com/pypa/setuptools/issues/4910#issuecomment-2749037152 to understand why

Adding this to pyproject.toml for poetry builds fixed for me:

[build-system] requires = [ <YOUR_OTHER_REQUIRES>, “setuptools<78”]

Yeah, I don’t think there’s actually an end-user mitigation here as pip-compile w/ build isolation will execute hooks in a custom venv built around that package’s requirements that you don’t have control over AFAICT. I can’t seem to get around that with a constraints file or anything.

Same issue is faced when installing pyspark or any other package for that matter. Please revert the changes. Thanks

How can I find out which package has the broken field? The error message mentions the field and the section but not the package.

It also breaks https://github.com/neptune-ai/neptune-client/issues/1907 through bravado-core

I’ve had success with uv and updating pyproject.toml to

[dependency-groups]
dev = [
    ...
    "setuptools<76.1.0",
    "bravado-core<6.0.0",
]

Further to @maxfenv 's comment - for anybody using uv, this can be avoided by specifying a build constraint as below:

uv pip install --build-constraint /requirements/constraints.txt -r /requirements/base.txt 

Or set a UV_BUILD_CONSTRAINT environment variable - for example, to set this in a Dockerfile, you could do:

ENV UV_BUILD_CONSTRAINT="/requirements/constraints.txt"

Where constraints.txt looks like:

setuptools==77.0.3 \
    --hash=sha256:583b361c8da8de57403743e756609670de6fb2345920e36dc5c2d914c319c945 \
    --hash=sha256:67122e78221da5cf550ddd04cf8742c8fe12094483749a792d56cd669d6cf58c
    # via -r requirements/constraints.in

Hashes are not required, it should still work with a more minimal configuration like this:

setuptools<78

We have the same issue with multiple third-party packages. Downgrading to 77.0.3 produces the same error.

We are circumventing this by excluding setuptools 78.0.1.

Update: That was a thought, but it does not work reliably. See the discussion below.

For people using uv. You can set a build constraint this way:

echo "setuptools<78" | uv pip install -b - -r requirements.txt

More details and alternatives are available here for you to read: https://github.com/astral-sh/uv/issues/12440

This is what helped me using poetry in a docker setup:

# Create a constraints file to limit setuptools version
RUN echo "setuptools<78" > /constraints.txt

# Set the PIP_CONSTRAINT environment variable to the constraints file
ENV PIP_CONSTRAINT=/constraints.txt

This worked for me

@acompa @luisiacc Did you just do this before running any poetry commands? Or before even installing poetry. Tried this and no luck here yet.

We don’t install things really with poetry, we do:

poetry export --without-hashes --extras deploy --format requirements.txt > requirements.txt

And then we install requirements.txt with pip normally.

Therefore if you use poetry, you are using the latest setuptools when installing your packages, E.G you cant get away from this bug.

@MerlinR We use poetry and, for what it’s worth, a version of this suggestion has also unblocked us.

That said I agree that this isn’t as simple as a dependency pin. It’s a dependency pin and poetry fixing this unaddressed issue!

Let the team breathe I guess they’re already reverting the changes! ☮ Thanks

This also breaks swifter (https://github.com/jmcarpenter2/swifter)

When using poetry, there’s no good way to specify which setuptools version used. See more details here: https://github.com/python-poetry/poetry/issues/4511

The only workaround that worked for me was running this before poetry install (took from the above issue)

RUN poetry run pip install swifter==1.4.0

I don’t know why that works at all, but it does.

Given there’s no consistent way to override this behavior and the amount of different package management solutions available for python at this point, this seems like an unnecessary nuisance rather than an useful update for users.

There are too many comments here but in case people didn’t mention this earlier, make sure to read this blog post

none of the advice in that post is relevant to this issue. see #4910 (comment) to understand why

@raylu that’s not true. pip-tools can be used to generate a constraint file for the (transitive) build deps specifically. I personally merged a pull request implementing it quite a while ago: https://pip-tools.rtfd.io/en/stable/#maximizing-reproducibility.

Using that is a bit unobvious, but setting the PIP_CONSTRAINT environment variable applies to said ephemeral build environments that are populated by a pip install in a subprocess that understands that env var just fine (it’s passing it on CLI via --constraint that is problematic).

There are too many comments here but in case people didn’t mention this earlier, make sure to read this blog post on handling dependency upgrades reliably: https://hynek.me/articles/semver-will-not-save-you/#taking-responsibility.

0.78

did you mean setuptools<78?

Haven’t found working solution for poetry, breaking packages: pyspark==3.5.1.

Therefore if you use poetry, you are using the latest setuptools when installing your packages, E.G you cant get away from this bug.

@MerlinR We use poetry and, for what it’s worth, a version of this suggestion has also unblocked us.

That said I agree that this isn’t as simple as a dependency pin. It’s a dependency pin and poetry fixing this unaddressed issue!

Thanks, we have briefly tried this with no luck, but going to test it further as breaks about 15 of our apps.

A note: I am baffled that this hasn’t been rolled back already.

@adiberk, I’m not certain myself, but I think the things you specified work for building the project itself, but for dependent packages being built from source, uv will install that package’s build-dependencies in a isolated venv, so your constraints will be ignored

Details here: https://docs.astral.sh/uv/concepts/projects/config/#build-isolation

I think no-build-isolation = true might be the fix you want, or maybe no-build-isolation-package = [...] to just disable the packages failing to build

Has anyone figured out a workaround when using uv sync (I believe there is a workaround for uv pip). build-constraints don’t seem to work in pyproject for me I tried both of these

[tool.uv]
constraint-dependencies = ["setuptools<=77.0.3"]
build-constraint-dependencies = ["setuptools<=77.0.3"]

A note: I am baffled that this hasn’t been rolled back already.

Breaking pytest-watch too

Haven’t found working solution for poetry, breaking packages: django-fast-update, django-computedfields, django-extensions

Just for posterity, this behavior change was intentionally introduced by https://github.com/pypa/setuptools/pull/4870

The ‘build constraints mechanism’ to which I refer above is documented here, for what it’s worth: https://docs.astral.sh/uv/pip/compile/#adding-build-constraints