build: Isolated build fails with generic `StopIteration` when pip is not provisioned in venv

I just hit this issue as well. I’m a developer for the Spack package manager. Spack installs each package into a separate installation prefix, so we rely on things like PYTHONPATH for packages to find their dependencies. Even if I install build[virtualenv], that doesn’t seem to solve the problem.

P.S. If pip is a run-time dependency, shouldn’t it be listed in setup.cfg under install_requires?

P.P.S. This wouldn’t be an issue if build vendored its dependencies like pip/setuptools/flit/poetry/etc.

Thanks for the detailed diagnosis @layday. I think you’re right, this is not an issue of how Python was built, but whether or not pip is present in PYTHONPATH. I tried building 3 combinations of --with-ensurepip={no,install,update} and all 3 caused the above issue when pip was in PYTHONPATH, but all 3 worked when pip was not in PYTHONPATH.

I’ll see if I can extract a minimal patch from https://github.com/pypa/build/commit/38099ad07636311d5ca8e810baf12b952b8f2568 and add it to our build recipe.

I don’t have Spack, but here’s a simple repro using Nix, which manipulates the Python path in the same way:

$ nix-shell -p python39.pkgs.pip --command fish
$ python -m pip install build --target foo
Collecting build
  Using cached build-0.7.0-py3-none-any.whl (16 kB)
Collecting pep517>=0.9.1
  Using cached pep517-0.12.0-py2.py3-none-any.whl (19 kB)
Collecting tomli>=1.0.0
  Using cached tomli-2.0.0-py3-none-any.whl (12 kB)
Collecting packaging>=19.0
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Using cached pyparsing-3.0.7-py3-none-any.whl (98 kB)
Installing collected packages: tomli, pyparsing, pep517, packaging, build
Successfully installed build-0.7.0 packaging-21.3 pep517-0.12.0 pyparsing-3.0.7 tomli-2.0.0
$ touch setup.py
$ PYTHONPATH="foo:$PYTHONPATH" python -m build
* Creating venv isolated environment...

Traceback (most recent call last):
  File "/[...]/foo/build/__main__.py", line 372, in main
    built = build_call(
  File "/[...]/foo/build/__main__.py", line 229, in build_package_via_sdist
    sdist = _build(isolation, builder, outdir, 'sdist', config_settings, skip_dependency_check)
  File "/[...]/foo/build/__main__.py", line 140, in _build
    return _build_in_isolated_env(builder, outdir, distribution, config_settings)
  File "/[...]/foo/build/__main__.py", line 104, in _build_in_isolated_env
    with _IsolatedEnvBuilder() as env:
  File "/[...]/foo/build/env.py", line 104, in __enter__
    executable, scripts_dir = _create_isolated_env_venv(self._path)
  File "/[...]/foo/build/env.py", line 261, in _create_isolated_env_venv
    pip_distribution = next(iter(metadata.distributions(name='pip', path=[purelib])))  # type: ignore[no-untyped-call]
StopIteration

ERROR
$ echo $PYTHONPATH
/nix/store/fkygdjhm2raz32xrzwnq8b89hkkdc9vv-python3-3.9.9/lib/python3.9/site-packages /nix/store/zs9mh2h826rgh2w39bv8gb1xc2nr77lk-python3.9-pip-21.3.1/lib/python3.9/site-packages

Looks like CentOS patches the venv module then 😕

You can install build with the virtualenv extra, build[virtualenv], which should fix this. We should make virrtualenv a hard requirement on CentOS though.