black: Black install via pip fails on embedded Python on Windows using latest pip version (20.3)

Describe the bug A new version of pip has recently been released, which is a lot stricter in many ways. When installing the latest version of Black on a Windows machine using the embedded Python distribution, I get an error:

ERROR: Requested black from https://files.pythonhosted.org/packages/dc/7b/5a6bbe89de849f28d7c109f5ea87b65afa5124ad615f3419e71beb29dc96/black-20.8b1.tar.gz#sha256=1c02557aa099101b9d21496f8a914e9ed2222ef70336404eeeac8edba836fbea has different version in metadata: '0.0.0'

The package it is trying to download is the latest source release (20.8b1), and from what I can gather, pip finds that the version in the filename (20.8b1) conflicts with the version in the metadata inside the package - which it says is 0.0.0.

To Reproduce

Install the embedded Python interpreter on Windows (https://docs.python.org/3/using/windows.html#the-embeddable-package)
Install pip in the embedded Python interpreter (download https://bootstrap.pypa.io/get-pip.py and run it using the embedded Python interpreter’s python.exe)
Run python.exe -m pip install black --no-cache-dir
You will see the following output:

c:\pepys-deploy\pepys-import>python -m pip install black --no-cache-dir
Collecting black
  Downloading black-20.8b1.tar.gz (1.1 MB)
     |████████████████████████████████| 1.1 MB 3.3 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
ERROR: Requested black from https://files.pythonhosted.org/packages/dc/7b/5a6bbe89de849f28d7c109f5ea87b65afa5124ad615f3419e71beb29dc96/black-20.8b1.tar.gz#sha256=1c02557aa099101b9d21496f8a914e9ed2222ef70336404eeeac8edba836fbea has different version in metadata: '0.0.0'

If the latest version which has a wheel release published to PyPI is specified instead, then it works fine:

c:\pepys-deploy\pepys-import>python -m pip install black==19.10b0 --no-cache-dir
Collecting black==19.10b0
  Downloading black-19.10b0-py36-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 1.1 MB/s
Requirement already satisfied: typed-ast>=1.4.0 in .\python\lib\site-packages (from black==19.10b0) (1.4.1)
Requirement already satisfied: toml>=0.9.4 in .\python\lib\site-packages (from black==19.10b0) (0.10.2)
Requirement already satisfied: regex in .\python\lib\site-packages (from black==19.10b0) (2020.11.13)
Requirement already satisfied: attrs>=18.1.0 in .\python\lib\site-packages (from black==19.10b0) (20.3.0)
Requirement already satisfied: appdirs in .\python\lib\site-packages (from black==19.10b0) (1.4.4)
Requirement already satisfied: pathspec<1,>=0.6 in .\python\lib\site-packages (from black==19.10b0) (0.8.1)
Requirement already satisfied: click>=6.5 in .\python\lib\site-packages (from black==19.10b0) (7.1.2)
Installing collected packages: black
  WARNING: The scripts black.exe and blackd.exe are installed in 'c:\pepys-deploy\pepys-import\python\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed black-19.10b0

So, it seems to be something to do with the metadata checking before building the wheel which is going wrong.

The latest version of Black worked fine with the previous version of pip which didn’t use the new strict metadata checker.

I can’t seem to reproduce this in non-embedded Python, but it may be an issue there too. I’m aware that pip isn’t officially supported for embedded Python on Windows, but it is widely used and all other packages seem to install fine:

Expected behavior Expected to install successfully.

Environment (please complete the following information):

Version: 20.8b1
OS and Python version: Windows embedded Python 3.7

Does this bug also happen on master? N/A as related to PyPI releases

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 6
Comments: 23 (3 by maintainers)

Links to this issue

We're the core team behind the popular Python autoformatter: Black. AMA!

Commits related to this issue

downgrade pip for the moment to deal with https://github.com/psf/black/issues/1847 — committed to fastavro/fastavro by scottbelden 3 years ago
downgrade pip for the moment to deal with https://github.com/psf/black/issues/1847 — committed to fastavro/fastavro by scottbelden 3 years ago
downgrade pip for the moment to deal with https://github.com/psf/black/issues/1847 — committed to fastavro/fastavro by scottbelden 3 years ago
downgrade pip for the moment to deal with https://github.com/psf/black/issues/1847 — committed to fastavro/fastavro by scottbelden 3 years ago
downgrade pip for the moment to deal with https://github.com/psf/black/issues/1847 — committed to fastavro/fastavro by scottbelden 3 years ago

Most upvoted comments

Just as an additional piece of information: the install works when when you tell pip to use it’s legacy resolver which isn’t as strict about metadata (see below). However, you can see that it is definitely confused about versions - it states Successfully installed black-0.0.0 at the end.

c:\pepys-deploy\pepys-import>python -m pip install black --no-cache-dir --use-deprecated=legacy-resolver
Collecting black
  Downloading black-20.8b1.tar.gz (1.1 MB)
     |████████████████████████████████| 1.1 MB 3.3 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: appdirs in .\python\lib\site-packages (from black) (1.4.4)
Requirement already satisfied: toml>=0.10.1 in .\python\lib\site-packages (from black) (0.10.2)
Requirement already satisfied: pathspec<1,>=0.6 in .\python\lib\site-packages (from black) (0.8.1)
Requirement already satisfied: typed-ast>=1.4.0 in .\python\lib\site-packages (from black) (1.4.1)
Requirement already satisfied: regex>=2020.1.8 in .\python\lib\site-packages (from black) (2020.11.13)
Requirement already satisfied: mypy-extensions>=0.4.3 in .\python\lib\site-packages (from black) (0.4.3)
Requirement already satisfied: click>=7.1.2 in .\python\lib\site-packages (from black) (7.1.2)
Requirement already satisfied: typing-extensions>=3.7.4 in .\python\lib\site-packages (from black) (3.7.4.3)
Building wheels for collected packages: black
  Building wheel for black (PEP 517) ... done
  Created wheel for black: filename=black-0.0.0-py3-none-any.whl size=124180 sha256=9d02c9cda7598e7caf1f73046db0bb8db2a1baf63bb54ac8c606c7ea8b143f82
  Stored in directory: C:\Users\ROBINW~1\AppData\Local\Temp\pip-ephem-wheel-cache-h9rra3jz\wheels\c5\85\79\f3af8daaf8037c0bf14beb3b7a1511a39b6e6902ca2aaf494e
Successfully built black
Installing collected packages: black
  WARNING: The scripts black-primer.exe, black.exe and blackd.exe are installed in 'c:\pepys-deploy\pepys-import\python\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed black-0.0.0

Note: the legacy resolver will be going away in the next pip release (Jan 2021 apparently), so it isn’t a long-term solution.

robintw on Nov 30, 2020

Agree. The pip king has spoken.

cooperlees on Apr 25, 2021

FWIW, I just saw that this involves using an embedded interpreter. Using pip with an embedded Python interpreter is explicitly not supported, by Python or pip.

From the link in OP:

Using pip to manage dependencies as for a regular Python installation is not supported with this distribution, though with some care it may be possible to include and use pip for automatic updates. In general, third-party packages should be treated as part of the application (“vendoring”) so that the developer can ensure compatibility with newer versions before providing updates to users.

I’d suggest closing this issue, since the specific usecase being asked for is not supported by the tooling being used.

pradyunsg on Apr 25, 2021

I’d love to. I just don’t have access. I’ve been personally trying to nudge @ambv …

Gonna be a rocky release tho. We have almost ~8 months of merges so people will need to be ready for that.

cooperlees on Mar 10, 2021

For folks looking to workaround this issue, downgrading has fixed the install issues in CI for our company:

$ pip list | grep black
black                    19.10b0

lucinvitae on Feb 26, 2021

Since @cooperlees asked nicely…

You seem to be hitting https://github.com/pypa/setuptools_scm/issues/386. The main solution is listed in https://github.com/pypa/setuptools_scm/issues/386#issuecomment-585075432. I’m not a 100% sure how you’re generating release artifacts, (I don’t see your release process documented – it’s usually a good idea to do so, to avoid mistakes!) but it does seem that the sdist doesn’t have the metadata that setuptools_scm needs to properly version it, likely due to setuptools_scm not being installed/used in the build environment.

Also, even though this is a pure-Python project, I strongly recommend uploading wheels to PyPI. Most users who use pip won’t really download/uses the sdist if there’s a wheel, so you’d also buy some time to investigate the underlying issue here by uploading a well-formed wheel. I’ll leave it up to the maintainers to decide whether they want the 20.8 beta wheel built (checkout the 20.8 tag and build the wheel with pip wheel .?) or if they wanna put out a completely new release. 😃

pradyunsg on Nov 30, 2020