setuptools: [BUG] Many packages are no longer installable after test command is removed

For those landing on this issue, please see: (thank you @delfick for summarizing this)

  • This functionality has been deprecated for 5 years, there is a separate issue for discussing if there would have been a better way of removing the functionality [FR] More gradual breakage of setuptools.command.test #4520
  • For people using pip (or tools using pip, like poetry), PIP_CONSTRAINT set to a file with setuptools<72.0 should work
  • For people using uv, there seems to be a bug where UV_CONSTRAINT doesn’t affect the version of setuptools used in build isolation (see ModuleNotFoundError: No module named ‘setuptools.command.test’ astral-sh/uv#5551) and in that case the solution is either --no-build-isolation (which should be considered a temporary solution) or getting packages to not use setuptools.command.test or using forks of those packages that comment out the use of setuptools.command.test (until setuptools releases a fix/revert)
  • To press subscribe to the ticket and/or thumbs up the top post instead of adding “same” comments

Quoted from https://github.com/pypa/setuptools/issues/4519#issuecomment-2255389347


setuptools version

setuptools==72.0.0

Python version

Python 3.9

OS

Linux

Additional environment information

No response

Description

The breakage change released on 72.0 breaks the default build isolation build of many packages since many of these packages do not pin on a particular setuptools version. There is also no way to pin to an older setuptools version as pip doesn’t offer a way to do this. Installing setuptools==71 first, then installing the package doesn’t work as the default build isolation resolves the dependencies without considering the lock file nor currently installed packages.

Based on these packages need to be patched with setup_requires before install, which is nearly impossible since we’d need to patch all packages.

Expected behavior

Install is successful.

How to Reproduce

  1. Create a virtual environment
  2. Upgrade to latest pip 24.2: python -m pip install --upgrade pip
  3. pip3.9 install doubles==1.4.0

Output

me@my ~/repro-pip-fail
 % pip3.9 install doubles==1.4.0                    
Looking in indexes: http://…
Collecting doubles==1.4.0
  Using cached http://…/doubles-1.4.0.tar.gz (16 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 497, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 2, in <module>
      ModuleNotFoundError: No module named 'setuptools.command.test'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 387
  • Comments: 127 (10 by maintainers)

Commits related to this issue

Most upvoted comments

Thanks for the report. I’ve yanked the 72.0.0 release while I investigate and determine if there’s a less invasive way to remove the command.

Given the lack of noise or reports since the release of 72.1.0, I believe the fix addressed the vast majority of use-cases affected by 72.0.0. My hope is that any remaining issues are isolated and readily addressable, but feel free to follow up here or in a new issue.

I want to thank everybody who was impacted by this for remaining civil, for your efforts in finding and documenting workarounds, and for providing constructive feedback on a change that was unnecessarily hostile to the ecosystem.

Yank, Yank, Yank!!!

dependency_xkcd_setuptools

Here’s how to work around this breakage:

Option 1: Create a file called constraints.txt

setuptools<72

Then set PIP_CONSTRAINT=/path/to/constraints.txt

Option 2: Use --no-build-isolation (and make sure you have an old setuptools installed)

Option 3: Ask the maintainers of each package that is broken like this to adapt to new setuptools and wait for them to make a release

In general, I find it unfortunate that build isolation intends to increase reproducibility, but the tools given to users make this really tricky in practice. I think ^ is pretty hard for most Python users I know to come up with (and lacks e.g. the ability to hash pin). Probably wrong place to rant though, setuptools maintainers probably feel the pain of this more than anyone else! 😃

Edit: if setuptools does decide to revert something here, I have a suggestion in https://github.com/pypa/setuptools/issues/4520 for how the underlying change could be rolled out more gradually

Next, let me acknowledge - I did expect this change to have some disruption, but not nearly as much as it had. Sorry for the disruption.

I was anticipating that impacted users would have had visibility to the deprecation warnings and most packages would long ago have migrated and that the handful that remained could be addressed promptly. It seems instead the deprecation warning wasn’t placed with enough visibility for those installing affected packages, so the message only served to reduce reliance on the command for running tests, but not necessarily removing the command entirely. This statement is not an excuse, just context.

Maybe unpopular opinion - don’t yank it! The test command has been deprecated for 5 years, which is more than enough time for people to update their packages. Clearly something drastic like this was necessary to force stragglers to make the change (else it wouldn’t be an issue).

If the packages you are using don’t resolve this in the near future, maybe consider not using unmaintained packages ¯_(ツ)_/¯

Yank v72.0.0 ASAP!

Guys, advising “downgrade setuptools, downgrade pip” and putting minuses under revert proposals, it’s easy to do when you develop single project on your own pc in your own venv. When you have tens or hundreds of projects, that use dependencies of dependencies, the game becomes much more exciting. rebuild base images, rebuild depending images, rebuild libraries… Do you really understand the scope of work?

p.s. don’t yank) let’s finally update things

holding prod environments hostage is not the best way to go about making package maintainers do their job is my opinion on the matter

epic morning

after an hour of googling, looks like I found the right bug 😃 this is an issue when installing dbt-core==1.8.4, which lists Logbook==1.5.3 as a requirement.

Latest Logbook (1.7.0) seems to have fixed this but dbt-core still points to the older broken module.

Would it be possible to mark this version as yanked in pypi and follow a more gradual approach with deprecation warnings on install as mentioned in https://github.com/pypa/setuptools/issues/4520 ?

Especially installs with poetry seem to have issues and currently I couldn’t find a solution to overcome it. Did anybody find a solution for following or similar combination?

  • docker
  • poetry==1.7.1
  • python:3.10
  • some package not supporting PEP 517 (in our case sqlalchemy==1.4.49 )

EDIT: Found a solution thanks to help of @ilpianista and @Hannes0730 , see my post below: https://github.com/pypa/setuptools/issues/4519#issuecomment-2255390400

(Of course we want to upgrade, but we’re still using sqlalchemy==1.4.49 which requires a major upgrade to 2.0 to support PEP 517).

seems that this one is going out of the control, based on quick Github search. it would be really nice if somebody from the dev team would make a definite statement whether we should do “dirty patches” in our deployment scripts with some random workarounds OR we should wait for the resolution in the setuptools itself. in our case, we have more than a dozen of projects with custom deployment scripts and complex inter-dependencies.

currently closely watching for this issue/thread to decide on future steps

Maybe unpopular opinion - don’t yank it! The test command has been deprecated for 5 years, which is more than enough time for people to update their packages. Clearly something drastic like this was necessary to force stragglers to make the change (else it wouldn’t be an issue).

If the packages you are using don’t resolve this in the near future, maybe consider not using unmaintained packages ¯_(ツ)_/¯

While I agree with this in generally, this issue explains the problem with this “deprecation” quite well? As in - it should’ve been at least “screaming” in the console each time the deprecated piece of code was utilised. That way consumers of packages with this deprecation could’ve reached out to maintainers, asking for upgrade or smth similar

This is a tricky one in that regard, however. Main problem is that a maintaner might’ve been using this command in a way that consumer wouldn’t know about - and might’ve never encountered - and this deprecation notice, when the command runs, wasn’t actually shown, because it’s never called by consumers. image

please yank this, none of these workarounds should be necessary and some of them don’t even work - disabling build isolation results in build errors for me.

i understand the desire to remove something that’s obsolete but clearly this wasn’t the right moment or way to do it.

I’m releasing the hot fix, which restores a small amount of the previous code to limit the breakage to “running setup.py test” but doesn’t break building a package that is importing the command, adding a deprecation warning for that case. It’s releasing as v72.1.0. I’ll be keeping a close eye on this issue and the issue tracker in case that change doesn’t adequately limit the disruption.

Not to be rude, but every comment in this thread (including this one, regrettably) leads to a large number of notifications and a large amount of noise that drowns out anything useful.

At this point, we should probably wait for a maintainer to provide an update.

this is ridiculous, this package is one of the most core packages and it has over 12 years of runway. Im very surprised as this bug is obvious the moment a realisitic test build is perform.

Please yank this version.

Please YANK!!! We can implement five different workarounds for various use cases, but the solution shouldn´t be this one. YANK!!

This functionality has been deprecated for 5 years

image

Pls YANK it!! All my CI dead bcs of it

Crowdstrike round 2?

I got a similar error today.

tbh… yes the workarounds work, but defining packages (versioned) outside of your normal tools just to get them installed is just not a workable solution for any kind of production app.

we are also getting similar error.

I just wanted to deploy at Monday instead of Friday and thats what happened

Could it 72.0.0 marked as yanked before other essential tools will be working with this change? 🙏

Can we revert this PR https://github.com/pypa/setuptools/pull/4458 ? I tried everything and was still not able to get it to work. We are working with poetry.

i got a similiar error too

This function can never be removed because removing it renders old package versions non-installable. It’s already too late to warn past developers about deprecation.

Python packaging has a huge problem with semantic versioning. The official documentation, to this day), recommends to always depend on the latest version of setuptools regardless of major version (and setuptools is not the only build backend doing this). This means that changes to setuptools can unexpectedly break nearly any sdist package, in this case rendering them non-installable. The change to setuptools version 72.0.0 indicates that it is not compatible with old packages, but the old packages accept any version and don’t even specify a single version that is known to be compatible. The builder has no better option than to install 72.0.0 and hope.

The simple way to fix this would be to create a setuptools2 and keep setuptools as a compatible version for packages that use it. However, even if the documentation says to specify a compatible major version, developers may continue not doing it because it’s convenient to them at the time, leading to this problem again years later. Maybe setuptools2 can error out if it detects that it is being required without a major version specified to avoid having to create a setuptools3.

To keep the name setuptools, maybe setuptools could detect that it is being required without a major version and replace itself with an older version or compatibility wrapper. It’s probably good enough to require the version be set in pyproject.toml and assume anything without pyproject.toml build-backend should use the oldest compatibility mode.

It’d also be possible for the package builder to do this kind of thing if it detects an unconstrained build backend version, but every package builder would need to know about the compatible fallback versions of every package build backend and that sounds like a good way to create inconsistent behavior across builders.

Oh this sucks so much!

@chrisirhc we may wanna add a couple more dotpoints to the description of this issue

  • the problem is with the package manager building wheels in an isolated environment and installing it’s own version of setuptools for this purpose (which is why the version specified in your environment doesn’t make a difference). The package manager ends up using the latest version of setuptools which doesn’t have a feature used by some packages at build time. The PIP_CONSTRAINT environment variable mentioned below constrains what versions the package manager uses for this purpose
  • Yanking this release is a popular request, the devs have yet to respond to this issue with what will happen moving forward. The solutions in this thread are only meant as temporary solutions until a real fix happens in setuptools.

edit: the version has been yanked https://github.com/pypa/setuptools/issues/4519#issuecomment-2255940870

#(https://github.com/pypa/setuptools/issues/4519#issuecomment-2255390400)

Thank you, this solved our issue (using Poetry 1.8.3, Python 3.11.4).

RUN echo "setuptools<72" > constraints.txt

ENV PIP_CONSTRAINT=constraints.txt

RUN poetry config virtualenvs.create false && \
    poetry run pip install sqlalchemy==1.3.24 && \
    poetry install --no-cache && \

I could not understand why when I degraded the setuptools to version below 72.0.0 it kept cruising with the error of ModuleNotFoundError: No module named 'setuptools.command.test'.

Next, let me acknowledge - I did expect this change to have some disruption, but not nearly as much as it had. Sorry for the disruption.

I was anticipating that impacted users would have had visibility to the deprecation warnings and most packages would long ago have migrated and that the handful that remained could be addressed promptly. It seems instead the deprecation warning wasn’t placed with enough visibility for those installing affected packages, so the message only served to reduce reliance on the command for running tests, but not necessarily removing the command entirely. This statement is not an excuse, just context.

Maybe having a daily pipeline that tries to install the top N packages (say top 250 or other reasonable value) in pypi using the develop branch would help catching widespread regressions like these?

Revert plz, this breaks lots of projects

@chrisirhc are you able to please add to the description of the ticket that

  • This functionality has been deprecated for 5 years, there is a separate issue for discussing if there would have been a better way of removing the functionality https://github.com/pypa/setuptools/issues/4520
  • For people using pip (or tools using pip, like poetry), PIP_CONSTRAINT set to a file with setuptools<72.0 should work
  • For people using uv, there seems to be a bug where UV_CONSTRAINT doesn’t affect the version of setuptools used in build isolation (see https://github.com/astral-sh/uv/issues/5551) and in that case the solution is either --no-build-isolation (which should be considered a temporary solution) or getting packages to not use setuptools.command.test or using forks of those packages that comment out the use of setuptools.command.test (until setuptools releases a fix/revert)
  • To press subscribe to the ticket and/or thumbs up the top post instead of adding “same” comments

😃

edit: I’ve since suggested a couple more dotpoints https://github.com/pypa/setuptools/issues/4519#issuecomment-2255690881

edit2: the version has been yanked https://github.com/pypa/setuptools/issues/4519#issuecomment-2255940870

The official version of setuptools is now 71.1.0 again. https://pypi.org/project/setuptools/#history 😅

Facing same error here.

09:04:19    ERROR: Command errored out with exit status 1:
09:04:19     command: /var/lib/****/.pex/venvs/s/301ee4ad/venv/bin/python3.9 /var/lib/****/.pex/venvs/s/301ee4ad/venv/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpato8w6eb
09:04:19         cwd: /tmp/pip-download-xmdfaim1/sqlalchemy
09:04:19    Complete output (19 lines):
09:04:19    /tmp/pip-build-env-kra8zma_/overlay/lib/python3.9/site-packages/_distutils_hack/__init__.py:55: UserWarning: Reliance on distutils from stdlib is deprecated. Users must rely on setuptools to provide the distutils module. Avoid importing distutils or import setuptools first, and avoid setting SETUPTOOLS_USE_DISTUTILS=stdlib. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
09:04:19      warnings.warn(
09:04:19    Traceback (most recent call last):
09:04:19      File "/var/lib/****/.pex/venvs/s/301ee4ad/venv/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
09:04:19        main()
09:04:19      File "/var/lib/****/.pex/venvs/s/301ee4ad/venv/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
09:04:19        json_out['return_val'] = hook(**hook_input['kwargs'])
09:04:19      File "/var/lib/****/.pex/venvs/s/301ee4ad/venv/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 114, in get_requires_for_build_wheel
09:04:19        return hook(config_settings)
09:04:19      File "/tmp/pip-build-env-kra8zma_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
09:04:19        return self._get_build_requires(config_settings, requirements=[])
09:04:19      File "/tmp/pip-build-env-kra8zma_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
09:04:19        self.run_setup()
09:04:19      File "/tmp/pip-build-env-kra8zma_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 497, in run_setup
09:04:19        super().run_setup(setup_script=setup_script)
09:04:19      File "/tmp/pip-build-env-kra8zma_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
09:04:19        exec(code, locals())
09:04:19      File "<string>", line 14, in <module>
09:04:19    ModuleNotFoundError: No module named 'setuptools.command.test'

@ziga7631 The problem release (72.0.0) was yanked yesterday. So in theory things should be ok now regardless. Just make sure your system isn’t using 72.0.0 specifically.

There’s also a newer 72.1.0 release with a fix in it as well.

@ilpianista @Hannes0730 Thanks for your help, following worked for me too in poetry:

RUN echo "setuptools<72" > /code/constraints.txt
ENV PIP_CONSTRAINT=/code/constraints.txt
RUN poetry run pip install sqlalchemy==1.4.49
RUN poetry install

(sqlalchemy==1.4.49 is the problematic package here not supporting PEP 517) The current workaround for poetry seems to be… not using poetry to install 👌 (Somehow RUN pip install sqlalchemy==1.4.49 wasn’t required but that might be due to still using python 3.10 🤔 )

Any workarounds for poetry ?

setuptools = "<72.0"

doesn’t seem to work for some reason.

PS: YANK it!

Coming here from https://github.com/AUTOMATIC1111/stable-diffusion-webui

Putting 👇 in requirements.txt fixed it for me.

setuptools==69.5.1

Next, let me acknowledge - I did expect this change to have some disruption, but not nearly as much as it had. Sorry for the disruption.

I was anticipating that impacted users would have had visibility to the deprecation warnings and most packages would long ago have migrated and that the handful that remained could be addressed promptly. It seems instead the deprecation warning wasn’t placed with enough visibility for those installing affected packages, so the message only served to reduce reliance on the command for running tests, but not necessarily removing the command entirely. This statement is not an excuse, just context.

The good news is I think this has exposed what is maybe not a bug, but an oversight at least on my part - that despite having versions pinned throughout our code base and pipelines, isolated build of packages were not subject to those constraints.

Here’s a repo demonstrating how you can work around this in your Github Actions.

https://github.com/simonwhitaker/setuptools-demo

Here’s how to workaround this breakage:

Option 1: Create a file called constraints.txt

setuptools<72

Then set PIP_CONSTRAINT=/path/to/constraints.txt (or UV_CONSTRAINT if using uv)

Option 2: Use --no-build-isolation (and make sure you have an old setuptools installed)

Option 3: Ask the maintainers of each package that is broken like this to adapt to new setuptools and wait for them to make a release

In general, I find it unfortunate that build isolation intends to increase reproducibility, but in practice the tools given to users make this really tricky in practice. I think ^ is pretty hard for most Python users I know to come up with. Probably wrong place to rant though, setuptools maintainers probably feel the pain of this more than anyone else! 😃

Hi, is option 1 compatible with poetry add <package-name>? or is that pip dependent? I tried option 2 but I think --no-build-isolation is not supported in poetry (1.8.2).

RUN poetry install --no-interaction --no-build-isolation --without dev --no-ansi -vvv

I am currently running this on Dockerfile

This breaks ~10 packages in my dependency chain which just specify:

[build-system]
requires = ["setuptools"]

so they don’t pin their setuptools build dependency, which makes them pull the latest version on every build. And that build happens in the isolated build environments for each package, so I don’t even really directly affect that somehow.

Not a good start into the week.

Makes me wonder if you should generally pin your build environment, there shouldn’t be any harm since it does not conflict with anything else? Assuming that you use build isolation though.

Also unable to use poetry at the moment…

For poetry users, exporting requirements.txt and pip install with the constraint worked for me.

export PIP_CONSTRAINT=constraints.txt
echo "setuptools<72" > constraints.txt
poetry export -o requirements.txt
pip install -r requirements.txt

I was also faced this issue with cssbeautifier library, and solved by installing with --no-build-isolation option in my case.

❯ docker run --rm -it 'python:3.9' /bin/bash

root@81f338564edc:/# python -m pip install --upgrade pip
Requirement already satisfied: pip in /usr/local/lib/python3.9/site-packages (23.0.1)
Collecting pip
  Downloading pip-24.2-py3-none-any.whl (1.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 5.0 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.0.1
    Uninstalling pip-23.0.1:
      Successfully uninstalled pip-23.0.1
Successfully installed pip-24.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

root@81f338564edc:/# pip3.9 install cssbeautifier
Collecting cssbeautifier
  Downloading cssbeautifier-1.15.1.tar.gz (25 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-sd2vprw8/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/tmp/pip-build-env-sd2vprw8/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-sd2vprw8/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 497, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-sd2vprw8/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 9, in <module>
      ModuleNotFoundError: No module named 'setuptools.command.test'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

root@81f338564edc:/# pip3.9 install --no-build-isolation cssbeautifier
Collecting cssbeautifier
  Using cached cssbeautifier-1.15.1.tar.gz (25 kB)
  Preparing metadata (pyproject.toml) ... done
Collecting jsbeautifier (from cssbeautifier)
  Downloading jsbeautifier-1.15.1.tar.gz (75 kB)
  Preparing metadata (pyproject.toml) ... done
Collecting six>=1.13.0 (from cssbeautifier)
  Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting editorconfig>=0.12.2 (from cssbeautifier)
  Downloading EditorConfig-0.12.4.tar.gz (13 kB)
  Preparing metadata (pyproject.toml) ... done
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: cssbeautifier, editorconfig, jsbeautifier
  Building wheel for cssbeautifier (pyproject.toml) ... done
  Created wheel for cssbeautifier: filename=cssbeautifier-1.15.1-py3-none-any.whl size=30536 sha256=fe16be1043cad7c149be1d60d9320b61f2c64ba672681fab279b2a88a71d9ff4
  Stored in directory: /root/.cache/pip/wheels/7f/2d/19/5d9d82d3c7280e2c68a07b906c458f0a5506cb2b812d330383
  Building wheel for editorconfig (pyproject.toml) ... done
  Created wheel for editorconfig: filename=EditorConfig-0.12.4-py3-none-any.whl size=16377 sha256=9acd52f9cb94f6fabd33cc641d4acb07a4714ada4678388b08e45722eabd5607
  Stored in directory: /root/.cache/pip/wheels/0b/7b/d7/1979a68ec6ce660c89044dc57d175e32f1943bcb8937e43f6e
  Building wheel for jsbeautifier (pyproject.toml) ... done
  Created wheel for jsbeautifier: filename=jsbeautifier-1.15.1-py3-none-any.whl size=94721 sha256=c9fefe5c1d947c492e3b8ea0173783df9821f8ca8a46637a8d4a645b9070ae18
  Stored in directory: /root/.cache/pip/wheels/1f/82/d9/63c09b95f4576d2822b61d073de8f4e3b6fd386112dcb9835d
Successfully built cssbeautifier editorconfig jsbeautifier
Installing collected packages: editorconfig, six, jsbeautifier, cssbeautifier
Successfully installed cssbeautifier-1.15.1 editorconfig-0.12.4 jsbeautifier-1.15.1 six-1.16.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.

For me the fix for the issue seems to be to run pip install --upgrade pip-tools pip wheel before running the command that runs into the ModuleNotFoundError: No module named 'setuptools.command.test' error message

Same issue here +1

same here +1

This functionality has been deprecated for 5 years

image image

Unfortunately this is what leaving something in a long-term state of deprecation leads to. It becomes de-prioritized.

If I have code that’s been deprecated for an extended period, I prefer to change the deprecation notice to a stronger one including a date when we’re ready to remove it, which prompts people to re-prioritize.

No need to be laying blame anywhere here though, rather look to recommendations to make the process smoother in future.

In terms of workarounds, PIP_CONSTRAINTS alone didn’t work for us with poetry, We had to do the following in our pipelines:

    - echo "Init Poetry and create requirements"
    - poetry config virtualenvs.in-project true
    - poetry config <snip>
    - poetry export --with dev -o requirements.txt --without-hashes
    - poetry export --with dev -f constraints.txt -o constraints.txt --without-hashes
    - PIP_CONSTRAINT="$(pwd)/constraints.txt" poetry run pip install -r requirements.txt

not sure how to write this down without offending anyone. pretty much similar behavior as with python core devs - “[4 letter word] you, you’ve been warned in time, not our fault”. some people just want to see the world burn

p.s. at the end we decided to use the dirty hack with PIP_CONSTRAINT in our production environment until the dust settles

I am having this issue with the autobahn package while installing with pip (https://github.com/crossbario/autobahn-python/issues/1644).

Changing my local projects requirements was not enough - the new version of setuptools was still being used despite pinning setuptools<72 for the reasons mentioned here https://github.com/pypa/setuptools/issues/4519#issuecomment-2255690881

Since I’m using poetry to build pip requirements.txt I was able to successfully generate and use constraints.txt as mentioned above (https://github.com/pypa/setuptools/issues/4519#issuecomment-2255652133).

Thanks for the help everyone! 😅

It’d be great to know when this is solved so folks can upgrade again. setuptools wasn’t even an explicit dependency and now it is an explicit devdependency.

Same issue here +1

Same here - please revert 😦

This function can never be removed because removing it renders old package versions non-installable.

That is not correct, workarounds existed before setuptools 72.0.0 got yanked, see #4519 (comment)

I disagree Juanlu (hola!) , many build systems are centralised , meaning customers (teams) can provide a pyproject , requirements.txt or similar, and the system takes care of the rest. This means individual python teams do not have the option to specify commands, setup environment variables or other custom changes. For example, in my current company, adding this hotfix would have involved adding quite some hacky code into a non python codebase, just to support a temporary fix.

I also had problems with this. In my case I need to install the packages in a github action and in docker with poetry. It’s a DBT project, so as mentioned in https://github.com/pypa/setuptools/issues/4519#issuecomment-2254942140 it’s because of logbook. So here is how I fixed it

1. Constraints

Create the constraints.txt with just setuptools<72

2. Docker

COPY constraints.txt .
ENV PIP_CONSTRAINT=constraints.txt

RUN poetry run pip install logbook==1.5.3 && \
    poetry install --no-cache

Replace logbook==1.5.3 with whatever package is giving you problems

3. Github action

    - name: Handle nasty bug # https://github.com/pypa/setuptools/issues/4519#issuecomment-2255390400
      run: |
        echo "PIP_CONSTRAINT=constraints.txt" >> $GITHUB_ENV
        poetry run pip install logbook==1.5.3

    - name: Install dependencies
      run: poetry install --no-cache

    # I had some problems with `pre-commit`, so I had to remove the env var afterwards:
    - name: Remove PIP_CONSTRAINT
      run: echo "PIP_CONSTRAINT=" >> $GITHUB_ENV

Replace logbook==1.5.3 with whatever package is giving you problems

Hope it helps!

pipenv

Pipfile

[packages]
setuptools = "==71.1.0"  # sse docker/Dockerfile:54

pipenv lock

Dockerfile

# https://github.com/pypa/setuptools/issues/4519#issuecomment-2255390400
# try to remove these lines after the issue is resolved
# remove setuptools from Pipfile
# also perhaps need wait for when other packages will be updated with compatible setuptools
RUN echo "setuptools<72" > /app/constraints.txt
ENV PIP_CONSTRAINT=/app/constraints.txt
RUN pip install --no-build-isolation --no-cache-dir "setuptools<72"

Same issue.

Maybe having a daily pipeline that tries to install the top N packages (say top 250 or other reasonable value) in pypi using the develop branch would help catching widespread regressions like these?

See https://github.com/pypa/setuptools/issues/4520#issuecomment-2256130396. There are some tests, but they’re not as deep as you might hope. Probably that test could be expanded to the top 100 or top 250.

after an hour of googling, looks like I found the right bug 😃 this is an issue when installing dbt-core==1.8.4, which lists Logbook==1.5.3 as a requirement.

Latest Logbook (1.7.0) seems to have fixed this but dbt-core still points to the older broken module.

Yeah. Exact same issue. dbt-core==1.8.3 also pointed to Logbook==1.5.3. I tried to install Logbook==1.7.0 seperately but had conflict issue. It is failing only my C1/CD and I believed dbt needed to fix it asap.

@jaraco can you please roll back the latest version? As it impacting many packages. After that, we can do the deprecate solution to ensure old packages still supported.

@jamesbuddrige Try the stuff in #4519 (comment)

We’re working across 50+ repo’s so this isn’t an ideal solution, but will give it a go. @justinclift

same here +1, with Nylas sdk

Same issue with me.

With this work for me with Docker, python 3.12

RUN pip3 install -I --force-reinstall setuptools==71.0.0
RUN pip3 install --no-build-isolation --no-cache-dir -r requirements.txt

Issue for us as well. Persists even though setuptools are pinned to 71.1.0.

@Artui have a look to #4519 (comment)

requirements approach works, but we run poetry everywhere, would be nice to have a workaround for that too.

Was that command deprecated first at some point? Or was it removed right away? Feels like the release should mark it as deprecated first instead of ripping it out…

pip install -I --force-reinstall setuptools==71.0.0

works for me

Does that mean that building old projects relying on this will break again? If that were the case, we’d end up with a broken CI pipeline once more. Since we rely on uv we have no means of controlling which version of setuptools will be pulled in when building projects.

In our case, one of the breaking packages was ffmpy: https://github.com/Ch00k/ffmpy/blob/master/setup.py#L2

Please hold off with a new release at least until uv provides a workaround.

No, IIRC only explicitly running python setup.py test would give an error, but merely having a setup.py referencing the test command would not.

For me the fix for the issue seems to be to run pip install --upgrade pip-tools pip wheel before running the command that runs into the ModuleNotFoundError: No module named 'setuptools.command.test' error message

Thanks @Jorden28, this worked for me.

my poetry generates bad links for some reason, so fast and ugly workaround for today is

ENV PIP_CONSTRAINT=constraints.txt
RUN echo "setuptools<72" > constraints.txt && \
    poetry export --without-hashes -o requirements.txt && \
    sed -i 's/git+git+ssh/git+ssh/g' requirements.txt
RUN --mount=type=ssh pip install -r requirements.txt

seems that i should have also used --no-build-isolation but everything works well without it

p.s. don’t yank) let’s finally update things

For Poetry/Docker users, I’ve managed to contain the fix to these three steps, though I did have to iterate to find all the dependencies that were broken

RUN mkdir -p /opt/pip && echo "setuptools<72" >/opt/pip/constraints.txt
ENV PIP_CONSTRAINT=/opt/pip/constraints.txt
RUN \
  mkdir /build/ \
  && poetry export --without-hashes \
    | grep \
      -e intervaltree \
      -e vcrpy \
      # -e any_other_package
    >/build/requirements.setuptools-bugged.txt \
  && poetry run pip install -r /build/requirements.setuptools-bugged.txt

You can run that before your usual RUN poetry install steps, which won’t re-install those packages

For those using docker and poetry,

This is how my dockerfile looks now, and it works for me

# ---- Base Python ----
FROM python:3.11.4-slim AS base

# Create app directory
WORKDIR /app

# ---- Dependencies ----
FROM base AS dependencies

# Install pip and Poetry
RUN pip install --upgrade pip && \
    pip install poetry

# Copy poetry.lock* in case it doesn't exist in the repo
COPY ./poetry.lock ./pyproject.toml /app/

# Install project dependencies.
RUN export PIP_CONSTRAINT=constraints.txt
RUN echo "setuptools<72" > constraints.txt
RUN poetry export -o requirements.txt
RUN pip install setuptools==71.1.0
RUN pip install --no-build-isolation -r requirements.txt

# ---- Copy Files/Build ----
FROM dependencies AS build

WORKDIR /app

COPY . /app

# --- Release ----
FROM base AS release

COPY --from=dependencies /usr/local/bin /usr/local/bin
COPY --from=dependencies /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY --from=build /app /app

EXPOSE 8000

# Setup entrypoint
COPY ./docker/entrypoint.sh /app

ENTRYPOINT ["sh", "/app/entrypoint.sh"]
CMD ["gunicorn", "config.wsgi:application", "--bind", "0.0.0.0:8000"]

I was facing this problem when I wanted to install Django Channels with Daphne. I found this solution and used the following command: pip3 install --no-build-isolation --no-cache-dir "setuptools<72"

For uv users, this worked for me:

uv venv
source .venv/bin/activate
uv pip install setuptools==71.1.0 poetry==1.8.3
uv pip install --no-build-isolation -r requirements.txt

This worked for us using poetry and docker. In the dockerfile:

WORKDIR /code
# these two new lines are the fix
RUN echo "setuptools<72" > "constraints.txt"
ENV PIP_CONSTRAINT=/code/constraints.txt

RUN poetry install --no-interaction --no-ansi

Same here - please revert 😦

+1

Quote reply

FOR POETRY I think this worked for me with PIP_CONSTRAINT combined.

FROM python:3.12.2-slim as python
ENV PYTHONUNBUFFERED=true
WORKDIR /app

FROM python as poetry
ENV POETRY_HOME=/opt/poetry
ENV POETRY_VIRTUALENVS_IN_PROJECT=true
ENV PATH="$POETRY_HOME/bin:$PATH"
RUN python -c 'from urllib.request import urlopen; print(urlopen("https://install.python-poetry.org").read().decode())' | python -
COPY . ./

# Install litestar because of broken setuptools unable to install jsbeautifier
ENV PIP_CONSTRAINT=/app/constraints.txt
RUN pip install litestar["standard"]
RUN poetry run pip install litestar["standard"]


RUN poetry install --no-interaction --without dev --no-ansi -vvv

In my case , I deupgrade my setuptools to version 71.1 solved the problem as well

This function can never be removed because removing it renders old package versions non-installable.

That is not correct, workarounds existed before setuptools 72.0.0 got yanked, see https://github.com/pypa/setuptools/issues/4519#issuecomment-2254983472

We are also using poetry, but did anyone solved it without creating a constraints.txt file?

I suppose you could do poetry run pip install "setuptools<72" then poetry run pip install --no-build-isolation <your package> - the constraints.txt file is only a static way of doing it

We are also using poetry, but did anyone solved it without creating a constraints.txt file?

for those using poetry, this helped solve the issue (thanks to many comments above)

echo "setuptools<72" > constraints.txt
export PIP_CONSTRAINT=constraints.txt
poetry run pip install ffmpy==0.3.2 #  package with the error
poetry install

but still dunno how to fix this in a cleaner way 🤨

Did someone figure out how to fix this on Pants package builds?

after an hour of googling, looks like I found the right bug 😃 this is an issue when installing dbt-core==1.8.4, which lists Logbook==1.5.3 as a requirement. Latest Logbook (1.7.0) seems to have fixed this but dbt-core still points to the older broken module.

For some reason, this seems to work fine now with us:

python -m pip install --force-reinstall setuptools==71.1.0 && python -m pip install --no-cache-dir --no-build-isolation dbt-snowflake==1.7.2

I don’t know man

Works for us as well to fix the dbt package issue 👍

after an hour of googling, looks like I found the right bug 😃 this is an issue when installing dbt-core==1.8.4, which lists Logbook==1.5.3 as a requirement.

Latest Logbook (1.7.0) seems to have fixed this but dbt-core still points to the older broken module.

For some reason, this seems to work fine now with us:

python -m pip install --force-reinstall setuptools==71.1.0 && python -m pip install --no-cache-dir --no-build-isolation dbt-snowflake==1.7.2

I don’t know man

thanks @omerbe-nym worked for us

@ilpianista @Hannes0730 Thanks for your help, following worked for me too in poetry:

RUN echo "setuptools<72" > /code/constraints.txt
ENV PIP_CONSTRAINT=/code/constraints.txt
RUN poetry run pip install sqlalchemy==1.4.49
RUN poetry install

The current workaround for potery seems to be… not using poetry to install 👌 (Somehow RUN pip install sqlalchemy==1.4.49 wasn’t required but that might be due to still using python 3.10 🤔 )

This workaround seems to be working, install the problematic package via pip(with constraints) and the rest via poetry

FYI for people using pip-tools that wants a workaround, make sure you both install setuptools version 71.1.0 and use pip-sync and pip-compile with the --pip-args --no-build-isolation flag

pip install --upgrade pip-tools pip wheel;
pip install setuptools==71.1.0;
pip-sync requirements.txt --pip-args --no-build-isolation

Issue for us as well. Persists even though setuptools are pinned to 71.1.0.

@Artui have a look to https://github.com/pypa/setuptools/issues/4519#issuecomment-2255263220

Issue for us as well. Persists even though setuptools are pinned to 71.1.0. image

Good morning, everyone! Yup, ran into a similar problem when our GH workflows started crashing after setuptools dropped this major update. For me, it’s breaking down while installing SQLAlchemy (any solution apart from restricting setuptools to bump up to 72? Why they removed test without any sort of indication):

Collecting SQLAlchemy==1.3.24 (from -r requirements.txt (line 22))
  Downloading SQLAlchemy-1.3.24.tar.gz (6.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.4/6.4 MB 119.3 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-uiic32ek/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-uiic32ek/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-uiic32ek/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 497, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-uiic32ek/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 14, in <module>
      ModuleNotFoundError: No module named 'setuptools.command.test'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

Especially installs with poetry seem to have issues and currently I couldn’t find a solution to overcome it. Did anybody find a solution for following or similar combination?

@Luke31 after several attempts, it looks like this one worked:

$ pip install <problematic dependecy>
$ poetry run pip install <problematic dependecy>
$ poetry install