vscode-python: python.testing.cwd ignored when discovering or running tests

Disclaimer: I have about zero days experience with Typescript, and the vscode-python extension… however I think the issue might have stemmed from this commit: “Force the pytest root dir to always be the workspace root.” https://github.com/microsoft/vscode-python/commit/9fe2a76322236bf752fe36e4e4230b2d2036ddff

Hi @pbotsman 👋 Thank you for reaching out and thoroughly filling out the issue template 👍

I can confirm that I could reproduce your issue in the latest release of the extension (2019.11.49689), and that it doesn’t happen in the previous release (2019.10.44104).

Seems like we don’t take python.testing.cwd into account anymore when discovering tests, or running single tests using pytest.

A possible workaround while waiting for a fix would be to move the content of python.testing.cwd in the list of pytest arguments, let me know if the following works for you:

{
    "python.pythonPath": "/path/to/python",
    "python.testing.pytestArgs": [        
        "./tests/py",
        "-v",
        "-s",
        "--log-cli-level", "INFO",
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true
}

@luabud

As @danstreeter and I pointed out before, the issue is that this extension forces the --rootdir parameter to pytest to be set to the workspaceFolder.

The fix is to perhaps have this as the default, but allow users to override the argument in their settings.

The problem was introduced here https://github.com/microsoft/vscode-python/commit/9fe2a76322236bf752fe36e4e4230b2d2036ddff#

FWIW, I just stumbled upon this, and would love to see a fix that changes the working directory to python.testing.cwd before running pytest.

Hey folks, sorry for the frustration with this issue. We’re planning to tackle this and many other testing bugs once we migrate into VS Code’s testing API (https://github.com/microsoft/vscode-python/issues/15750), but we don’t have an ETA for that yet.

In the mean time, adding "--rootdir","path_to_desired_cwd" to pytestArgs should give a workaround for this issue. So for the sample in the opening comment of this issue, it would look like this:

    "python.testing.pytestArgs": ["--rootdir","./tests/py"],

@flyte you had mentioned setting this up didn’t work for you, would you mind sharing your project structure so we can see what’s going on?

Hi, I can confirm that this issue does not appear to have been resolved and I have similar behaviour.

Any progress or workarounds for this issue? Setting option "python.testing.pytestArgs": ["--rootdir=path/to/root"] doesn’t help.

The only relevant settings are this:

"python.testing.pytestEnabled": true,
"python.testing.pytestArgs": ["--rootdir","${workspaceFolder}/unit_tests"]

In my project all my tests are under a unit_tests folder. It reads files relative to that folder but the command used in Test Log discover pytest -- --rootdir f:\code\cloverleaf\web -s unit_tests, not --rootdir f:\code\cloverleaf\web\unit_tests. I’ve also tried without ${workspaceFolder} but have had no luck.

Merged a fix for this just now to main. Fix is essentially what https://github.com/microsoft/vscode-python/issues/8678#issuecomment-808210852 said. If there is --rootdir in python.testing.pytestArgs, we won’t override it. so for the sample repo, the settings change would be:

{
    "python.testing.pytestArgs": [
        "-v",
        "-s",
        "--log-cli-level",
        "INFO",
        "--rootdir",
        "tests/py"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true
}

That should work:

Try this out with this build https://github.com/microsoft/vscode-python/suites/3858667759/artifacts/95863817 if that doesn’t work please provide repro steps.

This looks like a bug. It also seems to affect the unittest tests. Setting python.testing.cwd has zero effect on them.

I can confirm that it affects unittest tests as well. Interestingly, when running the whole test-suite (“Run All Tests”) cwd is correctly set to the folder containing the tests files (even without setting python.testing.cwd at all). When I run a single test, cwd is set to the workspace-root, regardless of the python.testing.cwd setting.

I have also noticed after setting rootdir in python.testing.pytestArgs in either

"python.testing.pytestArgs": ["--rootdir","${workspaceFolder}/unit_tests"]

or

"python.testing.pytestArgs": ["--rootdir=${workspaceFolder}/unit_tests"]

The unit test results (pass or fail) are no longer updated in the TEST explorer.

Environments

This bug actually forced me to switch to Pycharm. I experienced the bug using unittest. I hope it get’s resolved soon.

Hi @kimadeline,

Thanks for the workaround. This is exactly what I did 👍

I see. Thanks for the input. This should work for most case with this:

    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--rootdir",
        "${workspaceFolder}",
        "tests"
    ],

I will tweak it, it seems like for some reason, the cwd gets set to `` (empty) string instead of . which seems to be the problem here. But that is unrelated to this.

@danstreeter can you try:

"python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--rootdir",
        "./tests"
    ],

@danstreeter You can install this and uninstall to restore your VS Code to stable extension or any extension version.

Here is how you would do it:

1. Install the VSIX using Extension tab > … > Install from vsix… option from the “…” menu.
2. You may have to Reload VS Code (command pallet Reload Window command). Test out to see if this works for you.
3. To revert back, Go to extensions tab, find the python extension and Uninstall it.
4. Reload VS Code (command pallet Reload Window command), go to extensions tab and install from market place. This should get you back to stable build of the extension.

Note, you can also click on the ‘Gear’ icon next to the extension in the extensions tab to install a specific version using “Install Another version…”

If you want to try this out is complete isolation, you could install VS Code Insiders (which can install side by side with your stable one) and install this vsix there and test it out. then uninstall the whole thing. This way it won’t touch your day-to-day VS Code instance.

Works for me

Had to restart VS Code in order tests to appear in the sidebar without being nested inside tests directory