vscode-python: [Pytest] Tests not being discovered due to pytest_report hook

Type: Bug

Behaviour

After updating VS Code, VS Code Pytest is no longer discovering tests. Running Pytest on the terminal works as expected and tests are found correctly.

Expected vs. Actual

When running on ssh terminal, within Conda environment:

(kedro_dataiku) xxxxxx@xxxxxx:/ace/src/project_ace$ pytest --disable-warnings --collect-only
================================================= test session starts ==================================================
platform linux -- Python 3.7.16, pytest-3.10.1, py-1.10.0, pluggy-0.13.1
rootdir: /ace/src, inifile:
plugins: dash-1.19.0, anyio-2.2.0, typeguard-2.13.3, mock-1.13.0, cov-2.9.0
collected 1071 items

Output of VS Code after failing to discover tests:

2023-12-13 23:32:55.024 [info] > conda run -n kedro_dataiku --no-capture-output python ~/.vscode-server/extensions/ms-python.python-2023.22.0/pythonFiles/get_output_via_markers.py -m pytest -p vscode_pytest --collect-only
2023-12-13 23:32:55.024 [info] cwd: ./src/project_ace
2023-12-13 23:32:55.907 [info] >>>PYTHON-EXEC-OUTPUT

2023-12-13 23:32:56.172 [info] <<<PYTHON-EXEC-OUTPUT

2023-12-13 23:32:56.172 [error] Traceback (most recent call last):
  File "/home/xxxxx/.vscode-server/extensions/ms-python.python-2023.22.0/pythonFiles/get_output_via_markers.py", line 23, in <module>
    runpy.run_module(moduleName, run_name="__main__", alter_sys=True)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/runpy.py", line 96, in _run_module_code

2023-12-13 23:32:56.172 [error]     mod_name, mod_spec, pkg_name, script_name)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/pytest.py", line 91, in <module>
    raise SystemExit(pytest.main())
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/_pytest/config/__init__.py", line 58, in main

2023-12-13 23:32:56.172 [error]     config = _prepareconfig(args, plugins)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/_pytest/config/__init__.py", line 196, in _prepareconfig

2023-12-13 23:32:56.172 [error]     pluginmanager=pluginmanager, args=args
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__

2023-12-13 23:32:56.172 [error]     return self._hookexec(self, self.get_hookimpls(), kwargs)

2023-12-13 23:32:56.172 [error]   File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec

2023-12-13 23:32:56.172 [error]     return self._inner_hookexec(hook, methods, kwargs)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>

2023-12-13 23:32:56.172 [error]     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall

2023-12-13 23:32:56.172 [error]     gen.send(outcome)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/_pytest/helpconfig.py", line 93, in pytest_cmdline_parse

2023-12-13 23:32:56.172 [error]     config = outcome.get_result()
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result

2023-12-13 23:32:56.172 [error]     raise ex[1].with_traceback(ex[2])
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall

2023-12-13 23:32:56.172 [error]     res = hook_impl.function(*args)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/_pytest/config/__init__.py", line 675, in pytest_cmdline_parse

2023-12-13 23:32:56.173 [error]     self.parse(args)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/_pytest/config/__init__.py", line 845, in parse

2023-12-13 23:32:56.173 [error]     self._preparse(args, addopts=addopts)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/_pytest/config/__init__.py", line 795, in _preparse

2023-12-13 23:32:56.173 [error]     self.pluginmanager.consider_preparse(args)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/_pytest/config/__init__.py", line 475, in consider_preparse

2023-12-13 23:32:56.173 [error]     self.consider_pluginarg(opt2)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/_pytest/config/__init__.py", line 489, in consider_pluginarg

2023-12-13 23:32:56.173 [error]     self.import_plugin(arg)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/_pytest/config/__init__.py", line 537, in import_plugin

2023-12-13 23:32:56.173 [error]     self.register(mod, modname)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/_pytest/config/__init__.py", line 306, in register
    ret = super(PytestPluginManager, self).register(plugin, name)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/pluggy/manager.py", line 126, in register

2023-12-13 23:32:56.173 [error]     self._verify_hook(hook, hookimpl)
  File "/home/xxxxx/.conda/envs/kedro_dataiku/lib/python3.7/site-packages/pluggy/manager.py", line 261, in _verify_hook

2023-12-13 23:32:56.173 [error]     notinspec,
pluggy.manager.PluginValidationError: Plugin 'vscode_pytest' for hook 'pytest_report_teststatus'
hookimpl definition: pytest_report_teststatus(report, config)
Argument(s) {'config'} are declared in the hookimpl but can not be found in the hookspec

Steps to reproduce:

  1. Configure tests with Pytest
  2. Refresh test discovery

Diagnostic data

  • Python version (& distribution if applicable, e.g. Anaconda): 3.7.16
  • Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
  • Value of the python.languageServer setting: Pylance
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python) 

XXX

User Settings 


condaPath: "<placeholder>"

languageServer: "Pylance"

testing
• cwd: "<placeholder>"
• pytestEnabled: true

Extension version: 2023.22.1 VS Code version: Code 1.85.1 (0ee08df0cf4527e40edc9aa28f4b5bd38bbff2b2, 2023-12-13T09:49:37.021Z) OS version: Windows_NT x64 10.0.19045 Modes: Remote OS version: Linux x64 3.10.0-1160.99.1.el7.x86_64 Remote OS version: Linux x64 3.10.0-1160.99.1.el7.x86_64

System Info
Item Value
CPUs 12th Gen Intel® Core™ i7-12850HX (24 x 2419)
GPU Status 2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled
Load (avg) undefined
Memory (System) 63.67GB (40.83GB free)
Process Argv –crash-reporter-id d642995a-15db-49b9-871b-7574d0bace4e
Screen Reader no
VM 0%
Item Value
Remote SSH: ace
OS Linux x64 3.10.0-1160.99.1.el7.x86_64
CPUs Intel® Xeon® Platinum 8375C CPU @ 2.90GHz (48 x 2899)
Memory (System) 371.81GB (297.91GB free)
VM 0%
Item Value
Remote Container dataiku-base-ace-lbarcelo (reverent_saha) @ ace
OS Linux x64 3.10.0-1160.99.1.el7.x86_64
CPUs Intel® Xeon® Platinum 8375C CPU @ 2.90GHz (48 x 2899)
Memory (System) 371.81GB (297.91GB free)
VM 0%
A/B Experiments 
vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vsdfh931:30280409
vshan820:30294714
vstes263:30335439
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
pythonvs932:30410667
py29gd2263:30899288
vsclangdf:30486550
c4g48928:30535728
dsvsc012:30540252
azure-dev_surveyone:30548225
282f8724:30602487
89544117:30613380
vscrp:30673768
showlangstatbar:30737416
0bi6i642:30917235
fixshowwlkth:30771522
showindicator:30805244
pythongtdpath:30769146
i26e3531:30792625
welcomedialog:30910333
pythonnosmt12:30797651
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
dsvsc013:30795093
dsvsc014:30804076
dsvsc015:30845448
pythontestfixt:30902429
pyreplss1:30897532
pythonmypyd1:30879173
pythoncet0:30885854
pythontbext0:30879054
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
aa_t_chat:30882232

About this issue

  • Original URL
  • State: closed
  • Created 7 months ago
  • Reactions: 1
  • Comments: 23 (11 by maintainers)

Most upvoted comments

I can also confirm that setting "python.experiments.optOutFrom": [ "pythonTestAdapter" ] fixes the issue

+2
vanackere on Dec 15, 2023

@brettcannon version of vscode is quite recent (Version: 1.85.1 Commit: 0ee08df0cf4527e40edc9aa28f4b5bd38bbff2b2)

This must be a bug in devcontainer extension then, i’ll investigate and report

+1
vanackere on Dec 20, 2023

Are you able to get it working still in the experiment? We will soon switch to fully using the rewrite as our testing infrastructure which will disable the use of the experiment opt out as a work around.

+1
eleanorjboyd on Dec 18, 2023

I can also confirm that setting "python.experiments.optOutFrom": [ "pythonTestAdapter" ] fixes the issue

same here(with no obvious error in python output, though). for platform darwin -- Python 3.7.17, pytest-7.4.3, pluggy-1.2.0 environment opting out from pythonTestAdapter helped to discover tests in Vscode Testing section. (make sure to put that in user settings, not workspace settings and reload the VSCode to respect new settings)

Before (and after) that running pytest --collect-only in the terminal had the expected output, so it was the testing UI only that did not pick up discovered tests.

+1
peacefulseeker on Dec 18, 2023
Read more comments on GitHub
← vscode-python: Pytest test discovery over SSH does not work
vscode-python: Pytest: Unable to run tests for non-installed modules →