setuptools: [BUG] latest setuptools release (60.3.0) broke pip?

Yanked 60.3.0.

Setuptools 60.3.1 is going out now.

I chose not to release it last night because I wanted to be alert and around when it’s released, just in case there are other high-impact issues associated.

This caused some massive disruption (yikes!). I’d put too much faith in the test suite, which missed a crucial code path. The fix includes a regression test to ensure this code path is exercised and this issue doesn’t happen again.

Workaround is to add requires = ["setuptools==60.2.0", "wheel"]

Also, please use requires = ["setuptools!=60.3.0", "wheel"] if you have to avoid a bug like this in the future - that will not block future fixes, etc, so can be left in, while the pin has to be removed/updated. Don’t worry, bugs will be fixed. 😃

Hi @lhupfeldt thank you very much for the feedback, setuptools is working in a new version and it should be available soon.

Meanwhile, please consider the workaround previously discussed (i.e. installing the latest version of virtualenv, e.g. via pip/PyPI).

Please notice that using an outdated version of pip and virtualenv that are not able to handle the “yanked” metadata provided by the index server might interfere (and cause breakages) with other packages (not only setuptools).

It seems the offending commit is likely 41fa663c9da93ca1af98ce2ae397c02e4b3062e8.

$ python
Python 3.8.6 (default, Oct 28 2020, 19:59:21) 
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import __main__
>>> __main__.__file__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module '__main__' has no attribute '__file__'

Edit: cross-ref discussion at #2993 as well.

Something worth noting perhaps is that what is broken is explicitly python -m pip. Example:

You can break python -m pip with setuptools v60.3.0, but pip by itself will keep working fine, which is not great (though it does mean that your virtual environment isn’t destroyed forever! 👍).

$ docker run --rm -ti python:3.9 /bin/bash
root@767f3d487c7b:/# python -m venv venv && . venv/bin/activate && python -m pip --quiet install --upgrade setuptools && python -m pip list
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/venv/bin/python -m pip install --upgrade pip' command.
/venv/bin/python: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
(venv) root@767f3d487c7b:/# pip list
Package    Version
---------- -------
pip        21.2.4
setuptools 60.3.0
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/venv/bin/python -m pip install --upgrade pip' command.
(venv) root@767f3d487c7b:/# pip install --upgrade pip
Requirement already satisfied: pip in /venv/lib/python3.9/site-packages (21.2.4)
Collecting pip
  Downloading pip-21.3.1-py3-none-any.whl (1.7 MB)
     |████████████████████████████████| 1.7 MB 3.1 MB/s 
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.2.4
    Uninstalling pip-21.2.4:
      Successfully uninstalled pip-21.2.4
Successfully installed pip-21.3.1
(venv) root@767f3d487c7b:/# pip list
Package    Version
---------- -------
pip        21.3.1
setuptools 60.3.0
(venv) root@767f3d487c7b:/# python -m pip list
/venv/bin/python: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')

This change also breaks python -m build as well

Hi @delicb, would you have a way of reproducing this behaviour? When I tried to install build in a brand new virtualenv here and run it against one of my packages, everything seems to be OK.

It might be the case the index server/cache-proxy does not handle “yanked” versions very well… If that is true, it would be good to report these issues to them so they can also investigate (just try first updating virtualenv/pip and see if the problem persists…)

I would kindly ask you to bear with us just for a bit, while the new version is not out. setuptools has followed the correct protocol in the Python ecosystem, by yanking the problematic version. It is a pity not all tools are ready to interoperate with that.

Removing all caches and venvs that I know about did not solve my issue, 60.3 keeps comimng back 😦 A quick re-branding of 60.2 as 60.3.1 would probably solve the problem and help a lot of people.

@abravalheri Thanks for the response. I understand this is 99% due to AWS Code Artifact.

I can not share the project (closed source), but it is a very simple library with the following section in pyproject.toml and standard nothing-fancy setup.py:

[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

With those lines, python -m build fails with ERROR Command '['/private/var/folders/xh/ltk6fjb55zv9ykn3k767lhvh0000gn/T/build-env-tzlc3gel/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r', '/var/folders/xh/ltk6fjb55zv9ykn3k767lhvh0000gn/T/build-reqs-xrd9lz9x.txt']' returned non-zero exit status 1..

After setting setuptools==60.2.0, command passes.

As additional test, I have removed index URL from pip config (thus falling back to default PyPI, instead of AWS Code Artifact) and build is successful even when version is not explicitly set. So, it does look like it is AWS Code Artifact Problem (btw, I have tried deleting the 60.3.0 from codeartifact, but it just pops up again).

You are right, I will try to report this problem to AWS. I can check if there is a way to “forbid” some package versions or something similar, but workaround with fixing a version to 60.2.0 works so I might just wait for new version to be rleased.

A newer version should probably be released with a fix. Applying workarounds like --upgrade setuptools==60.2.0 is not feasible in automated builds. E.g. we have an RTD installation in docker, to apply that workaround I would have to hack the docker build. It can also be very difficult to remve all cached versions. I have to cleanup all pip caches on our RTD server plus caching in our pypi proxy.

Experiencing this as well. Pinning to the previous setuptools gets around this issue until this is fixed:

python3 -m pip install --upgrade setuptools==60.2.0 

@jaraco if you happen to see this before you go offline till tomorrow, would it be worth yanking 60.3.0 off PyPI?

Seen by @matthewfeickert and I as well. https://twitter.com/HEPfeickert/status/1478923370083545089