vstest: Can't list full test names from a DLL.
Description
IMPORTANT: if the defect is reproduced only in a workflow from within the Visual Studio IDE then do not report the issue here - instead, please report it using Visual Studio’s “Send Feedback” option that can be accessed from the Help menu OR using this link https://developercommunity.visualstudio.com.
For a defect reproducable from the vstest command line, describe the issue you’ve observed. There is no way to list all tests with their fully qualified name in the command line.
Steps to reproduce
What steps can reproduce the defect? Please share the setup, commandline for vstest.console, sample project, target framework etc. For a C++ test DLL (Microsoft Unit Test Framework for C++), run:
vstest.console.exe <target DLL> --ListFullyQualifiedTests
This will fail with message:
Target Path should be specified for listing FQDN tests!
Expected behavior
Share the expected output (List of FQDN tests in the DLL).
Actual behavior
What is the behavior observed? The command forces to print the output to a file using switch
--ListTestsTargetPath. This seems enforced by this source line:https://github.com/Microsoft/vstest/blob/master/src/vstest.console/Processors/ListFullyQualifiedTestsArgumentProcessor.cs#L225.
Diagnostic logs
Please share test platform diagnostics logs. Instructions to collect logs are here.
The logs may contain test assembly paths, kindly review and mask those before sharing.
Environment
Please share additional details about the test environment. Operating system, Build version of vstest.console Windows 10, x64 Version 10.0.17134 Build 17134 VSTest, version 15.7.2
About this issue
- Original URL
- State: open
- Created 6 years ago
- Comments: 18 (9 by maintainers)
It’s not a matter of “live with”. It’s a feature available in virtually every test framework, missing on this one.
Would it be too hard to implement for VSTest? Does this pose security issues? If it were truly by design, why does Visual Studio not have a problem displaying the tests with their fully qualified name?
I’m using this test-runner to verify versions of our product with a set of tests that runs against a dedicated environment. Like others have reported we have multiple tests with similar or identical names.
We run the test from the command line on a dedicated machine and the report is presented in a web interface.
Having the ability to have the fully qualified name reported to the command line seems like a logical option. The fact that most other test frameworks has this option verifies this in my opinion.
@cltshivash We are using both NUnit3 for dotnet and GoogleTest for C++ code.
I would like the ability to cluster the tests under assembly + namespace + class as per Visual Studio, I’m not sure what you mean by container, I can see options for Test Suite, but we are not using that so we get something like this…
I know this should be under the vsts-task but without the FQN being emitted from vstest I don’t see how they would have enough information to be able to produce anything better