black: setuptools-scm is causing issues (especially since 20.8b1 doesn't have wheels)

Hi everyone,

We are aware of the recent installation issues especically with 20.8b1. This is due to a problem with the recently published versions of setuptools-scm, one of black’s build time dependencies, https://github.com/pypa/setuptools_scm/issues/608. The good news is that all of the offending releases have been yanked on PyPI so this issue should be now gone.

If you’re wondering why 20.8b1 is failing while the others are fine, it’s because 20.8b1’s wheel was deleted due to a mistake done during the release artifact building process (the build environment wasn’t cleaned up). This is important because building only needs to happen with sdists … and it’s the wheel building process that was borked by the setuptools-scm fires. We’ve moved our artifact building to GHA so these sort of release mistakes should be less common – hopefully. Sorry for the disruption!

~ @ichard26 (August 27th, 12:55 PM EST)


Currently

conda create --name test python==3.7.10 (just to have fresh environment of python 3.7 for showcase (not a conda issue)
conda activate test
pip install black==20.8b1

fails with

Collecting black==20.8b1
Collecting black==20.8b1
  Downloading black-20.8b1.tar.gz (1.1 MB)
     |████████████████████████████████| 1.1 MB 4.3 MB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/bin/python /opt/conda/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp6m1djknn
         cwd: /tmp/pip-install-y3c4u6lp/black_d9e87aa87d32486194850d6807c77340
    Complete output (30 lines):
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 349, in <module>
        main()
      File "/opt/conda/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 331, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/opt/conda/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 151, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-build-env-5vjv51tn/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 166, in prepare_metadata_for_build_wheel
        self.run_setup()
      File "/tmp/pip-build-env-5vjv51tn/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 150, in run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 104, in <module>
        "black-primer=black_primer.cli:main",
      File "/tmp/pip-build-env-5vjv51tn/overlay/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/opt/conda/lib/python3.7/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/tmp/pip-build-env-5vjv51tn/overlay/lib/python3.7/site-packages/setuptools/dist.py", line 456, in __init__
        k: v for k, v in attrs.items()
      File "/opt/conda/lib/python3.7/distutils/dist.py", line 292, in __init__
        self.finalize_options()
      File "/tmp/pip-build-env-5vjv51tn/overlay/lib/python3.7/site-packages/setuptools/dist.py", line 807, in finalize_options
        ep(self)
      File "/tmp/pip-build-env-5vjv51tn/overlay/lib/python3.7/site-packages/setuptools_scm/integration.py", line 52, in infer_version
        config = Configuration.from_file(dist_name=dist_name)
      File "/tmp/pip-build-env-5vjv51tn/overlay/lib/python3.7/site-packages/setuptools_scm/config.py", line 187, in from_file
        defn = _load_toml(data)
      File "/tmp/pip-build-env-5vjv51tn/overlay/lib/python3.7/site-packages/setuptools_scm/config.py", line 59, in _lazy_tomli_load
        from tomli import loads
    ModuleNotFoundError: No module named 'tomli'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/dc/7b/5a6bbe89de849f28d7c109f5ea87b65afa5124ad615f3419e71beb29dc96/black-20.8b1.tar.gz#sha256=1c02557aa099101b9d21496f8a914e9ed2222ef70336404eeeac8edba836fbea (from https://pypi.org/simple/black/) (requires-python:>=3.6). Command errored out with exit status 1: /opt/conda/bin/python /opt/conda/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp6m1djknn Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement black==20.8b1 (from versions: 18.3a0, 18.3a1, 18.3a2, 18.3a3, 18.3a4, 18.4a0, 18.4a1, 18.4a2, 18.4a3, 18.4a4, 18.5b0, 18.5b1, 18.6b0, 18.6b1, 18.6b2, 18.6b3, 18.6b4, 18.9b0, 19.3b0, 19.10b0, 20.8b0, 20.8b1, 21.4b0, 21.4b1, 21.4b2, 21.5b0, 21.5b1, 21.5b2, 21.6b0, 21.7b0)
ERROR: No matching distribution found for black==20.8b1

on another machine that it happened to work before fine it still works due to cached wheels

pip install black==20.8b1                                                                                                           (test) 
Collecting black==20.8b1
  Using cached black-20.8b1-py3-none-any.whl
Collecting appdirs
  Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting toml>=0.10.1
  Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting regex>=2020.1.8
  Using cached regex-2021.8.27-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (746 kB)
Collecting mypy-extensions>=0.4.3
  Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting typing-extensions>=3.7.4
  Using cached typing_extensions-3.10.0.0-py3-none-any.whl (26 kB)
Collecting typed-ast>=1.4.0
  Using cached typed_ast-1.4.3-cp37-cp37m-manylinux1_x86_64.whl (743 kB)
Collecting click>=7.1.2
  Using cached click-8.0.1-py3-none-any.whl (97 kB)
Collecting pathspec<1,>=0.6
  Using cached pathspec-0.9.0-py2.py3-none-any.whl (31 kB)
Collecting importlib-metadata
  Using cached importlib_metadata-4.7.1-py3-none-any.whl (17 kB)
Collecting zipp>=0.5
  Using cached zipp-3.5.0-py3-none-any.whl (5.7 kB)
Installing collected packages: zipp, typing-extensions, importlib-metadata, typed-ast, toml, regex, pathspec, mypy-extensions, click, appdirs, black
Successfully installed appdirs-1.4.4 black-20.8b1 click-8.0.1 importlib-metadata-4.7.1 mypy-extensions-0.4.3 pathspec-0.9.0 regex-2021.8.27 toml-0.10.2 typed-ast-1.4.3 typing-extensions-3.10.0.0 zipp-3.5.0

and

locate black-20.8b1-py3-none-any.whl                                                                                                (test) 
/home/rskolasinski/.cache/pip/wheels/18/f0/0f/0ea9a19fc7c9677435c6da5f765bf1a03b8e647f8b33ffec43/black-20.8b1-py3-none-any.whl
/home/rskolasinski/.cache/pip/wheels/22/58/5f/570e0ab54a223a614dc0c1d2c5f7a2095a1cef91eacf702625/black-20.8b1-py3-none-any.whl
/home/rskolasinski/.cache/pip/wheels/c5/85/79/f3af8daaf8037c0bf14beb3b7a1511a39b6e6902ca2aaf494e/black-20.8b1-py3-none-any.whl

so, the question, did you remove the wheels from PyPa for this release? https://pypi.org/project/black/20.8b1/#files Was it by accident or a conscious decision?

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 10
  • Comments: 20 (1 by maintainers)

Commits related to this issue

Most upvoted comments

This impacts everyone using black through pre-commit (was looking for the issue)

This error is not specific to version 20.8b1, but happens in 21.7b0 or 21.7b0 (and presumably others) as well. Timing-wise, it seems like this could be related to the release of setuptools-scm 6.1.1.

Looks like this is caused by pypa/setuptools_scm#608

Pinning the setuptools-scm version in setup.cfg might be a good way to avoid these issues in the future.

OK, for the record, the wheels for 20.8b1 were deleted because of a mistake done (unclean build environment) while building the release artifacts. Since we’ve now moved the building to GHA, these sort of issues shouldn’t happen again. Sorry for the trouble!

@piotrgredowski If I check out the tag, a pip install . (as executed by pre-commit) fails.

editorial note: I made some tweaks to both the issue title and the original post so anyone curious to why their CI systems suddenly failed will have an easier time finding this issue and understanding the problem ^^

Thanks for this report!

Am I correct in thinking that old versions will remain uninstallable, even if we pin setuptools-scm in the future?