vscode-python: Pytest discovery fails if something is printed to stderr

Can I get any help with pytest discovery failure?

> /usr/local/bin/python ~/.vscode-server/extensions/ms-python.python-2022.0.1786462952/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
cwd: .
[ERROR 2022-1-3 23:36:9.283]: Error discovering pytest tests:
 [r [Error]: Graceful shutdown completed in 0.000 seconds.

	at ChildProcess.<anonymous> (/root/.vscode-server/extensions/ms-python.python-2022.0.1786462952/out/client/extension.js:32:39235)
	at Object.onceWrapper (events.js:422:26)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)]

Tried many different options with “python.testing.cwd” and “python.testing.pytestArgs” but without luck.

I used unit tests in the past and they work fine in VS Code.

Also I use VS Code with “Remote Container” and work inside the Docker container and run tests inside it.

Hope that you can solve this issue soon, because with pytest it doesn’t work for me. And I have no clear vision why. Thanks.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 8
  • Comments: 31 (1 by maintainers)

Most upvoted comments

Getting a similar error, however it seems like it’s because the environment location is messed up

> conda run -n onedevex-dev --no-capture-output python ~/.vscode-server/extensions/ms-python.python-2022.0.1786462952/pythonFiles/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2022.0.1786462952/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear .
cwd: .
[ERROR 2022-1-4 19:17:3.378]: Error discovering pytest tests:
 [r [Error]: 
EnvironmentLocationNotFound: Not a conda environment: /private/home/calebh/miniconda3/envs/onedevex-dev/envs/onedevex-dev


	at ChildProcess.<anonymous> (/private/home/calebh/.vscode-server/extensions/ms-python.python-2022.0.1786462952/out/client/extension.js:32:39235)
	at Object.onceWrapper (events.js:422:26)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)

I’m getting a very similar issue - pytest discovery not working in vscode while pytest from cmd line works fine. From the code Python output I see:

> f:\anaconda\condabin\conda.bat run -p F:\anaconda\envs\msticpy --no-capture-output python ~\.vscode\extensions\ms-python.python-2022.1.1779589815-dev\pythonFiles\get_output_via_markers.py ~\.vscode\extensions\ms-python.python-2022.1.1779589815-dev\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir . -s --cache-clear tests

Not sure if something in conda is able to interpret ~\.vscode\extens... as %userprofile%\.vscode\extens... (I’m running on Windows where “~” doesn’t resolve to anything).

If I try to copy that command (and replace the “~” with userprofile path), it fails as follows

(I’ve added -v flags to conda run to try to get more info):

(msticpy) E:\src\microsoft\msticpy>f:\anaconda\condabin\conda.bat run -v -v -v -p F:\anaconda\envs\msticpy --no-capture-output python %userprofile%\.vscode\extensions\ms-python.python-2022.1.1779589815-dev\pythonFiles\get_output_via_markers.py %userprofile%\.vscode\extensions\ms-python.python-2022.1.1779589815-dev\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir . -s --cache-clear tests 
DEBUG conda.gateways.logging:set_verbosity(231): verbosity set to 3
DEBUG conda.gateways.subprocess:subprocess_call(71): executing>> C:\WINDOWS\system32\cmd.exe /d /c F:\anaconda\envs\msticpy\.tmpzij4q0j9.bat
>>>PYTHON-EXEC-OUTPUT
usage: C:\Users\Ian\.vscode\extensions\ms-python.python-2022.1.1779589815-dev\pythonFiles\testing_tools\run_adapter.py [-h] {discover} ...
C:\Users\Ian\.vscode\extensions\ms-python.python-2022.1.1779589815-dev\pythonFiles\testing_tools\run_adapter.py: error: argument cmd: invalid choice: 'C:\\Users\\Ian\\.vscode\\extensions\\ms-python.python-2022.1.1779589815-dev\\pythonFiles\\testing_tools\\run_adapter.py' (choose from 'discover')
TRACE conda.gateways.subprocess:subprocess_call(108): $ C:\WINDOWS\system32\cmd.exe /d /c F:\anaconda\envs\msticpy\.tmpzij4q0j9.bat
==> cwd: E:\src\microsoft\msticpy <==
==> exit code: 2 <==
==> stdout <==
None
==> stderr <==
None

ERROR conda.cli.main_run:execute(33): Subprocess for 'conda run ['python', 'C:\\Users\\Ian\\.vscode\\extensions\\ms-python.python-2022.1.1779589815-dev\\pythonFiles\\get_output_via_markers.py', 'C:\\Users\\Ian\\.vscode\\extensions\\ms-python.python-2022.1.1779589815-dev\\pythonFiles\\testing_tools\\run_adapter.py', 'discover', 'pytest', '--', '--rootdir', '.', '-s', '--cache-clear', 'tests']' command failed.  (See above for error)
TRACE conda.gateways.disk.delete:rm_rf(165): rm_rf F:\anaconda\envs\msticpy\.tmpzij4q0j9.bat

I also tried removing the get_output_via_markers.py component of the command - still no success. But I noticed that it seemed to be trying to execute pytest from the conda base environment rather than the current conda env.

(msticpy) E:\src\microsoft\msticpy>f:\anaconda\condabin\conda.bat run -v -v -v -p F:\anaconda\envs\msticpy --no-capture-output python %userprofile%\.vscode\extensions\ms-python.python-2022.1.1779589815-dev\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir . -s --cache-clear tests
DEBUG conda.gateways.logging:set_verbosity(231): verbosity set to 3
DEBUG conda.gateways.subprocess:subprocess_call(71): executing>> C:\WINDOWS\system32\cmd.exe /d /c F:\anaconda\envs\msticpy\.tmp39rluiwl.bat
equivalent command: f:\anaconda\python.exe -m pytest --collect-only --rootdir . -s --cache-clear tests
TRACE conda.gateways.subprocess:subprocess_call(108): $ C:\WINDOWS\system32\cmd.exe /d /c F:\anaconda\envs\msticpy\.tmp39rluiwl.bat
==> cwd: E:\src\microsoft\msticpy <==
==> exit code: 120 <==
==> stdout <==
None
==> stderr <==
None

ERROR conda.cli.main_run:execute(33): Subprocess for 'conda run ['python', 'C:\\Users\\Ian\\.vscode\\extensions\\ms-python.python-2022.1.1779589815-dev\\pythonFiles\\testing_tools\\run_adapter.py', 'discover', 'pytest', '--', '--rootdir', '.', '-s', '--cache-clear', 'tests']' command failed.  (See above for error)
TRACE conda.gateways.disk.delete:rm_rf(165): rm_rf F:\anaconda\envs\msticpy\.tmp39rluiwl.bat

This is the line I’m referring to - although not sure if this is a anything to do with the problem - it should be f:\anaconda\envs\msticpy\python.exe

equivalent command: f:\anaconda\python.exe -m pytest --collect-only --rootdir . -s --cache-clear tests

The result is the same with or without the -v flags.

I digged in this problem because I was using the environment.yml to set env variables and I discoverd the following:

the problem semms to be to me that that conda puts a message on the stderr:

Then vscode-python everytime founds a new message on the stderr, forwards it

In this case, I guess is conda’s fault since it writes to the stderr this WARNING

Btw, if vscode would activate the environment before, instead of running:

conda run -n env-name --no-capture-output python ~/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear

The problem would be fixed

Mine is working again seemingly… I’ll open up a new issue if it returns

Edit: ah ok, I think I’ve discovered the source of the error, will open a new issue. Edit2: Opened as #19090

Update:

  • The original issue reported by @1st (discovery failing if something is printed to stderr) should be addressed as part of the refactoring work done in #17242.
  • @pstjohn and @1vecera, it looks like your discovery problems have a different root cause, please open separate issues with a sample repo, your interpreter info, and repro steps to make it easier for us to track and debug them. Feel free to reference this issue as well.
  • Anybody else, same as above: if your discovery problems are not caused by print statements, please open a separate issue.

Thanks!

@1vecera For now I “fixed” the issue by reverting to v2021.12.1559732655

There’s a fundamental issue here in that the extension is always trying to do full discovery before anything is usable/runnable.

In large repositories this is impractical, unhelpful, time-consuming, and unnecessary. I appreciate this might be due to our repository structure choices, but in a repository such as ours with a 10-year history and many, many sub-folders this is hugely impractical and almost unusable.

With e.g. Eclipse and PyDev I can always select a test file or folder and “just run it” regardless of the state of the rest of the repo. This doesn’t seem that unreasonable, or that unachievable. With a large repo, I largely don’t care about the state of the rest of the repo, I just want to run the test/tests I’m currently working on.

Ultimately, if the the file I’m viewing is called test.py and the function I’m looking at is called test I should be able to run it, without caring about anything else, shouldn’t I?

Hi everyone, a couple of things here:

  • Issues with conda are most likely related to #18436, since we recently changed how we run tools when in a conda environment. We hope to make a hotfix release today or tomorrow, please try it out whenever it comes out, and if it still doesn’t work please open a separate issue;
  • When creating a conda environment, make sure you create it with the python=3.XXX argument, or the extension won’t recognize it as a conda environment;
  • We currently do not support setting environment variables in the test runner, we have an open feature request for it: https://github.com/microsoft/vscode-python/issues/17171. Please upvote it to show your interest and help us prioritize it! A workaround for now would be to specify your environment variables in a .env file.

@1st and @brianoflondon, please provide a sample repo that I can use to try and replicate your issues.

Perhaps I’m getting something similar which seems to stem from the discovery process trying to run my module itself with bad command line options. I’m really new to setting up a new project here so very unsure if this is my problem or similar to what others have reported here.

This bit puzzles me, why is discovery trying to run podping-hivewatcher (my project) with pytest command line options.

usage: hivewatcher [options]
hivewatcher: error: unrecognized arguments: discover pytest -- --rootdir /Users/gbishko/Documents/Python-iMac/PodcastIndex/podping-hivewatcher -s --cache-clear tests

Running pytest on the command line works fine.

> ~/Library/Caches/pypoetry/virtualenvs/podping-hivewatcher-YKYAFVom-py3.8/bin/python ~/.vscode/extensions/ms-python.python-2022.0.1786462952/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear tests
cwd: .
[ERROR 2022-1-5 14:29:48.180]: Error discovering pytest tests:
 [r [Error]: ============================= test session starts ==============================
platform darwin -- Python 3.8.9, pytest-5.4.3, py-1.11.0, pluggy-0.13.1
rootdir: /Users/gbishko/Documents/Python-iMac/PodcastIndex/podping-hivewatcher
usage: hivewatcher [options]
hivewatcher: error: unrecognized arguments: discover pytest -- --rootdir /Users/gbishko/Documents/Python-iMac/PodcastIndex/podping-hivewatcher -s --cache-clear tests
collected 0 items / 1 error

==================================== ERRORS ====================================
________ ERROR collecting tests/integration/test_podping_hivewatcher.py ________
tests/integration/test_podping_hivewatcher.py:2: in <module>
    from podping_hivewatcher.hivewatcher import allowed_op_id, get_allowed_accounts
src/podping_hivewatcher/hivewatcher.py:10: in <module>
    from podping_hivewatcher.config import Config
src/podping_hivewatcher/config/__init__.py:164: in <module>
    args = my_parser.parse_args()
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/argparse.py:1771: in parse_args
    self.error(msg % ' '.join(argv))
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/argparse.py:2521: in error
    self.exit(2, _('%(prog)s: error: %(message)s\n') % args)
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/argparse.py:2508: in exit
    _sys.exit(status)
E   SystemExit: 2
=========================== short test summary info ============================
ERROR tests/integration/test_podping_hivewatcher.py - SystemExit: 2
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.46s ===============================

Traceback (most recent call last):
  File "/Users/gbishko/.vscode/extensions/ms-python.python-2022.0.1786462952/pythonFiles/testing_tools/run_adapter.py", line 22, in <module>
    main(tool, cmd, subargs, toolargs)
  File "/Users/gbishko/.vscode/extensions/ms-python.python-2022.0.1786462952/pythonFiles/testing_tools/adapter/__main__.py", line 100, in main
    parents, result = run(toolargs, **subargs)
  File "/Users/gbishko/.vscode/extensions/ms-python.python-2022.0.1786462952/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 44, in discover
    raise Exception("pytest discovery failed (exit code {})".format(ec))
Exception: pytest discovery failed (exit code 2)

	at ChildProcess.<anonymous> (/Users/gbishko/.vscode/extensions/ms-python.python-2022.0.1786462952/out/client/extension.js:32:39235)
	at Object.onceWrapper (events.js:422:26)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)]