nunit3-vs-adapter: An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Error when using Real Time Discovery

When reporting a bug, please provide the following information to speed up triage:

NUnit 3.13.2, NUnit3TestAdapter 4.1.0 (also happens on 4.2.0-beta.4)
Visual Studio 2019 (16.11.8) and 2022 (17.0.4)
.NET Core 3.1

I have a C# test project with a TestCaseSource that uses an F# type (discriminated union). When I try to run all tests using the VS test runner no tests get run and the Tests output window shows

========== Starting test discovery ==========
NUnit Adapter 4.1.0.0: Test discovery starting
NUnit Adapter 4.1.0.0: Test discovery complete
========== Test discovery finished: 1 Tests found in 1.3 sec ==========
========== Starting test run ==========
NUnit Adapter 4.1.0.0: Test execution started
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Error: Invalid Condition 'FullyQualifiedName=CSharpTest.MyTests.RunSomeTests
  BofDU3 = DU3Member "bbb" }'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.
Stack trace:
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Discovery.DiscoveryContext.GetTestCaseFilter(IEnumerable`1 supportedProperties, Func`2 propertyProvider)
   at NUnit.VisualStudio.TestAdapter.VsTestFilter.get_TfsTestCaseFilterExpression() in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\VsTestFilter.cs:line 100
   at NUnit.VisualStudio.TestAdapter.VsTestFilter.get_IsEmpty() in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\VsTestFilter.cs:line 102
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.InitializeForExecution(IRunContext runContext, IFrameworkHandle frameworkHandle) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 248
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 105
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.RunTestsWithSources.InvokeExecutor(LazyExtension`2 executor, Tuple`2 executorUriExtensionTuple, RunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestInternalWithExecutors(IEnumerable`1 executorUriExtensionMap, Int64 totalTests)

========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in 41 ms ==========

I can run the test using dotnet test from the command line, however.

This looks like bug #691 which was closed a while ago.

About this issue

Original URL
State: open
Created 3 years ago
Comments: 17 (10 by maintainers)

Most upvoted comments

The extra node you see there, RunSomeTests, comes from the Real Time Test Discovery, and is not a valid result. The RTD can’t figure out the underlying tests since it just does source discovery. I would turn this option off, as it is just confusing in your case,
It should not be a single test that causes this. If you add the dbg version of the adapter to your real test code, you should be able to single step through it the same way. That method is being used from different places in the adapter code, but since it stopped on the InitializationForExecution code in the repro, it should stop in the same place for your real code.
I have asked MSPG about which situations where the sources interface method could be called.

OsirisTerje on Dec 29, 2021

Thanks a lot for your persistence! It can confirm this, it breaks on my site too when I turn on the “Discover tests in real time”.

I’ll report this to the MSPG too, and you’re right, it’s quite a “gotcha” !

Thanks again and a Happy New Year !

OsirisTerje on Dec 29, 2021