I wanted to give some transparency about this whole situation and how we got here as I know some people are a bit frustrated at the moment with our pytest support has been fluctuating a bit over the last few releases.

We used to use a hack to discover tests in pytest and the pytest team specifically asked us to stop (and rightfully so) 😁 . Pytest didn’t have any stability guarantees for the hack we were using, hence why the last few releases there has been skew for what tests we discovered in the extension (totally our fault for using the hack BTW, it’s just the usual software issue of having the time to do the proper fix versus patching what we had).

And so in this release we finally got around to using pytest’s plug-in system for test discovery which is a more proper way. Unfortunately there seems to be a an issue yet again in terms of what pytest reports and what we used to discover using the hack (and I’m not placing blame on pytest, just that discovery is not universally broken, just that it’s hard to pin down). Add to that the fact that pytest is very flexible in its discovery and it makes finding reproducers a bit hard without repos we can clone (thanks to those providing such repos!). We did increase our testing of test discovery as well, so I would like to think this isn’t due to sloppiness on our part as much as the complexity of the whole situation as people’s testing setups are not always simple.

Regardless of everything I have said above, we’re obviously sorry this has caused trouble for folks and we are actively trying to figure out what’s going on.

Hi everyone:

• It looks like the problems are mostly related to conflicting pytest plugins. So for those that reported a problem, please reply with which pytest plugins you are using. Thanks!
• We will be publishing a new version of the extension tomorrow with the old behavior. Hopefully this should resolve the issues you’ve experienced. In the interim, you can download & install the pre-release version extension from here and try it now
• We have updated the development version of the extension to log the details of the failure. Please could you download & install the development version of the extension from here and try it now. Test discovery will still fail. However this time, the details of the failure will be logged in the Python Test Log output panel. Please provide the details of the failure.

Once again, we apologize for the inconvenience caused by this issue.

Seems like a bug in the latest version of VS Code Python extension. I had the same issue, then I downgraded the Python extension to 2019.3.6558 and then it works again. So we should go to our VS Code extensions list, select the Python extension and “Install another version…” from the setting of that extension.

@chkpnt when you look in the settings.json in the repo i pointed to, i specified the root directory as test directory --> python.testing.pyTestArgs = [ "." ].

Anyway, i think it should not be such a hassle to discover tests, which are somewhere in the workspace folder tree - with or without __init__.py. I am wondering that this is somehow broken, because the same setup worked a few days before (maybe it has to do with python extension update?!)…

As i said, when running pytest from the commandline from the root folder with the same commands from the pyTestArgs the tests are discovered without any problem - with and without __init__.py. Within vscode tests are only discovered when i leave out the __init__.py. For replication check my repo: https://github.com/xibalba01/vscode_python_ext_bug

@r-beer i second that. More clear error messages would be helpful.

Strangely when i set up a clean repo i get not problems… since my repo/filebase where the error occured is largely crap i dont like to make it official 😃 But maybe it helps:

https://github.com/xibalba01/vscode_python_ext_bug

Some interesting thing happened while playing around with my repo: I figured out that the error did NOT occur in vscode when i leave out the __init__.py file! Tests are discovered successfully. I have to add, that an existing __init__.py has no effect on manually running pytest! Hope it helps discovering the bug.

I’m hitting this issue as well with my previously testing & working reference repo: https://github.com/noelbundick/azure-python-boilerplate

Repro steps on MacOS

• python 3.7.1
• pytest 4.4.1
• vscode-python 2019.4.11987

# setup
git clone https://github.com/noelbundick/azure-python-boilerplate
cd azure-python-boilerplate
python3 -m venv .venv
source .venv/bin/activate

# verify it's working in cli
boilerplate
pytest

code .

• Open a Python file to activate the extension
• Allow update from python.unitTest.* to python.testing.* in settings.json
• Note the Test discovery failed error

Remediate

• Downgrade vscode-python to 2019.3.6558
• Reload window
• Revert settings.json to use python.unitTest.* again
• Python: Run All Unit Tests

Note that tests are discovered and tests pass

@brettcannon @ericsnowcurrently

My work around was to run “python -m pytest” instead. The fix is probably ensuring that PYTHONPATH includes the project

I believe that’s one of the problems. We’ve been using -m which adds current DIR into path, whereas the current speech doesn’t. Solution is to just add the workspace into the path.

Tried the debug extension, the abbreviated error message is: “E ModuleNotFoundError: No module named ‘mylib’” ./mylib is one of module folder in the project root dir.

This is the same error I get when I run “pytest” in my root project dir from command line. My work around was to run the following instead: “PYTHONPATH=. pytest”

The fix is probably ensuring that PYTHONPATH includes the project root dir when you launch ‘pytest’.

I have found that removing all args, and leaving it as a blank list appears to work.

after update pytest and vscode are latest version,I met same issue with exit code 2

Test Discovery failed: Error: Traceback (most recent call last): File “c:\Users\wuguilin.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\run_adapter.py”, line 16, in <module> main(tool, cmd, subargs, toolargs) File “c:\Users\wuguilin.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\adapter_main_.py”, line 90, in main parents, result = run(toolargs, **subargs) File “c:\Users\wuguilin.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\adapter\pytest.py”, line 43, in discover raise Exception(‘pytest discovery failed (exit code {})’.format(ec)) Exception: pytest discovery failed (exit code 1)

my code

import pytest

def test_a():
    assert 1

@DonJayamanne, unfortunately I can’t provide my repo. I tried to drag some modules and their companion tests into a fresh repo but VSCode/Python were well-behaved.

From Python output window after running VSCode in the original repo:

Starting Jedi Python language engine.
Diagnostic Code: InvalidTestSettingDiagnostic, Message: Your settings needs to be updated to change the setting "python.unitTest." to "python.testing.", otherwise testing Python code using the extension may not work. Would you like to automatically update your settings now?

I have the same problem with pytest! I didn’t change anything to the test configuration and it worked fine yesterday. Environment data (vscode and python extension version) is more or less the same as from the OP except that i use Miniconda python v3.7.1.

Here are my workspace settings:

{
    "python.testing.pyTestArgs": [
        ".",
        "-v",
        "-s"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pyTestEnabled": true
}

Additionally i get a large error message in the console, which could help you guys debug this error:

Python Extension: Failed to parse discovered Test Error: Traceback (most recent call last):
  File "C:\Users\USER\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\run_adapter.py", line 16, in <module>
    main(tool, cmd, subargs, toolargs)
  File "C:\Users\USER\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\adapter\__main__.py", line 90, in main
    parents, result = run(toolargs, **subargs)
  File "C:\Users\USER\.vscode\extensions\ms-python.python-2019.4.11987\pythonFiles\testing_tools\adapter\pytest.py", line 43, in discover
    raise Exception('pytest discovery failed (exit code {})'.format(ec))
Exception: pytest discovery failed (exit code 2)

	at ChildProcess.module.exports.exec.c.once (C:\Users\USER\.vscode\extensions\ms-python.python-2019.4.11987\out\client\extension.js:9:35776)
	at Object.onceWrapper (events.js:273:13)
	at ChildProcess.emit (events.js:182:13)
	at maybeClose (internal/child_process.js:961:16)
	at Process.ChildProcess._handle.onexit (internal/child_process.js:248:5)

I would like to add that i love vscode and its capabilities - you are doing great work! Please fix this error, because i also need vscode and testing at work 😃