validate-pyproject: Many tests fails with `generator raised StopIteration`

I’m seeing 131 tests to fail like this:

FAILED tests/test_pre_compile.py::test_invalid_examples_api[cibuildwheel/overrides-noselect.toml-cli_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_examples_api[setuptools/07-pyproject.toml-cli_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[cibuildwheel/overrides-noselect.toml-api_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[setuptools/package-dir/invalid-name.toml-api_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[localtool/fail2.toml-cli_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_examples_api[atoml/pyproject.toml-cli_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[pep621/missing-fields/missing-version.toml-cli_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[setuptools/dependencies/invalid-extra-name.toml-api_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_examples_api[localtool/working.toml-cli_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[setuptools/packages/missing-find-arguments.toml-api_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_examples_api[ruff/modern.toml-cli_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_examples_api[setuptools/06-pyproject.toml-cli_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[store/ruff-unknown.toml-api_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[setuptools/pep621/license/both-text-and-file.toml-api_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[cibuildwheel/unknown-option.toml-api_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[setuptools/cmdclass/invalid-value.toml-api_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_examples_api[cibuildwheel/default.toml-cli_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_invalid_examples_api[setuptools/dynamic/readme-too-many.toml-cli_pre_compile] - RuntimeError: generator raised StopIteration
FAILED tests/test_pre_compile.py::test_examples_api[pep_text/pyproject.toml-api_pre_compile] - RuntimeError: generator raised StopIteration
=========================================== 131 failed, 352 passed, 1 skipped in 361.60s (0:06:01) ===========================================

I tested both the 0.19 sdist and the latest main git branch using tox -e py39.

I found that when I do this: touch src/LICENSE src/LICENSE.txt then all tests pass.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 19 (9 by maintainers)

Most upvoted comments

This is probably related to the change in behaviour?

pytest 7.1.3 (2022-08-31) #3396 (https://github.com/pytest-dev/pytest/issues/3396): Doctests now respect the --import-mode flag.

Let me think how we can better change that. Ideally I would not like to exclude the src directory from doctests, and remove the separate pytest --doctest-modules src command.

(The recommended import mode for new projects, i.e. “importlib” also seem to have problems https://github.com/pytest-dev/pytest/issues/7652, which I am hitting when working on this).

Isn’t validate-pyproject expecting newer importlib.metadata than the one bundled with Python 3.9?

In principle no…

According to the docs in importlib_metadata, Python 3.8 stdlib bundles what would be equivalent to importlib_metadata==1.4. The files() was introduced in version 0.9. So in theory the version in stdlib should match the needs of validate-pyproject. (Unless there is a bug that never received a fix).


Thank you for all the information, I will need to have a deeper look at this problem.

I test using these steps:

git clone https://github.com/abravalheri/validate-pyproject.git
cd validate-pyproject/
tox -e py39

AFAIK, tox installs the validate-pyproject so I do not need to do that manually:

$ tox -e py39
.pkg: install_requires> python -I -m pip install 'setuptools>=61.2' 'setuptools_scm[toml]>=7.1'
.pkg: _optional_hooks> python /usr/lib/python3.9/vendor-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_sdist> python /usr/lib/python3.9/vendor-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: build_sdist> python /usr/lib/python3.9/vendor-packages/pyproject_api/_backend.py True setuptools.build_meta
py39: install_package_deps> python -I -m pip install 'fastjsonschema<=3,>=2.16.2' 'packaging>=20.4' pytest pytest-cov pytest-randomly pytest-xdist 'repo-review; python_version >= "3.10"' setuptools 'tomli>=1.2.1; python_version < "3.11"' 'trove-classifiers>=2021.10.20'
py39: install_package> python -I -m pip install --force-reinstall --no-deps /data/builds/validate-pyproject/.tox/.tmp/package/1/validate_pyproject-0.19.post1.dev8+g84aa9e5.tar.gz
py39: commands[0]> pytest --doctest-modules src
============================================================================================================= test session starts =============================================================================================================
platform sunos5 -- Python 3.9.20, pytest-8.3.2, pluggy-1.5.0 -- /data/builds/validate-pyproject/.tox/py39/bin/python
...
...
...