pytest: Main and 8.0.x (not released yet): cannot find fixture when using --pyargs

a detailed description of the bug or problem you are having
output of pip list from the virtual environment you are using
pytest and operating system versions
minimal example if possible

While adding a comment to https://github.com/pytest-dev/pytest/issues/9765#issuecomment-1891056936 to reproduce that issue, I noticed another problem (not released yet, only the main and 8.0.x branches).

Reproduce the issue

python -m pip install "ewokscore[test]==0.7.1" git+https://github.com/pytest-dev/pytest.git@main
python -m pytest -v --pyargs ewokscore.tests

or also

python -m pip install "ewokscore[test]==0.7.1" git+https://github.com/pytest-dev/pytest.git@8.0.x
python -m pytest -v --pyargs ewokscore.tests

Released pytest versions do not show the problem. For example this works find:

python -m pip install "ewokscore[test]==0.7.1" "pytest==7.4.4"
python -m pytest -v --pyargs ewokscore.tests

Os and python version

Ubuntu 20.04, python 3.10.12: works
Windows 11, python 3.8.10: fails

Error message

The error says it cannot find the varinfo dependency which is defined in the module ewokscore.tests.conftest.

  @pytest.mark.parametrize("value", VALUES)
  def test_variable_references(value, varinfo):
E       fixture 'varinfo' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

About this issue

Original URL
State: closed
Created 6 months ago
Comments: 15 (15 by maintainers)

Most upvoted comments

The issue is here:

https://github.com/pytest-dev/pytest/blob/348e6de102c5a5ccdaab88c860d57b1f49f7b1d2/src/_pytest/fixtures.py#L1486-L1506

A bit dense technical explanation, hopefully you manage to follow 😃

When a conftest plugin (and any other plugin) is registered, pytest calls the pytest_plugin_registered hook. The fixtures code implements this hook to register the fixtures defined in the plugin.

When you register a fixture, you need to define its “visibility” AKA baseid. The baseid is a nodeid, and a test can only “see” a fixture if the baseid is the nodeid of one of its parents (see matchfactories function).

When a fixture is defined in e.g. some test file, the visibility is the specific node it’s defined in, e.g. the Module node (run pytest --collect-only to see the collection tree).

When a fixture is defined in a conftest plugin (the relevant case for this issue), the visibility logically is supposed to be the conftest’s directory. In practice this is done in the code above – the baseid (the nodeid variable in the code above) is synthesized from the plugin.__file__.

As we’ve already learned from #9765, relying on both __file__ and user-input paths is problematic. @woutdenolf solution in PR #11821 is to go back to relying on __file__ for registration and normalizing it more. But in #9765 the solution we’ve decided on is the opposite, that is, avoid relying on __file__ as much as possible.

Therefore, the PR I would like to see instead is to change the fixtures.py pytest_plugin_registered hook impl to not rely on plugin.__file__. Some ways I can think to do this:

Instead of using plugin.__file__, use the registration name, which for conftest plugins is the original conftest path as we registered it. The hookimpl can do plugin_name = manager.get_name(plugin) then use that instead of __file__.
Internally in pluggy (pytest’s plugin system), get_name(plugin) is currently a linear search, which might get somewhat slow with many plugins. Maybe we don’t care about it, but if we do, can do one of these instead:
- Make the lookup faster in pluggy
- Add a new plugin_name parameter to the pytest_plugin_registered hook.
Longer term, I really think we should tie conftests to Directory nodes (new in pytest 8.0), this is something I’m working toward but it’s speculative and not applicable for the short term.

@woutdenolf @lesteve It’d be great if one of you could try this and see if it indeed fixes the issue.

bluetech on Jan 16, 2024