nunit3-vs-adapter: TestFilter does not support test cases with spaces and special characters
Running “dotnet test” with a test filter for a test case with spaces the following exception occurs:
dotnet test --settings …/…/…/build/.runsettings --no-build --logger “trx;LogFileName=C:\Temp\0.trx” --filter “FullyQualifiedName="BackLook.Apps.WorkInProgress.Specs.Computing work in progress Project.No Workpackages exist"” --no-build
Test run for C:\ws\BackLook\system\App.WorkInProgress\BackLook.Apps.WorkInProgress.Specs\bin\Debug\net48\BackLook.Apps.WorkInProgress.Specs.dll (.NETFramework,Version=v4.8) Microsoft ® Test Execution Command Line Tool Version 16.10.0 Copyright © Microsoft Corporation. All rights reserved.
Starting test execution, please wait…
A total of 1 test files matched the specified pattern.
An exception occurred while invoking executor ‘executor://nunit3testexecutor/’: Unexpected Word ‘work’ at position 64 in selection expression.
Stack trace:
at NUnit.VisualStudio.TestAdapter.TestFilterConverter.TestFilterParser.Expect(Token[] valid) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\TestFilterConverter\TestFilterParser.cs:line 252
at NUnit.VisualStudio.TestAdapter.TestFilterConverter.TestFilterParser.Parse(String input) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\TestFilterConverter\TestFilterParser.cs:line 66
at NUnit.VisualStudio.TestAdapter.NUnitTestFilterBuilder.ConvertVsTestFilterToNUnitFilter(IVsTestFilter vsFilter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitTestFilterBuilder.cs:line 72
at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 116 at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.RunTestsWithSources.InvokeExecutor(LazyExtension2 executor, Tuple2 executorUriExtensionTuple, RunContext runContext, IFrameworkHandle frameworkHandle) at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.<>c__DisplayClass48_0.<RunTestInternalWithExecutors>b__0() at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformThread.<>c__DisplayClass0_0.<Run>b__0() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformThread.Run(Action action, PlatformApartmentState apartmentState, Boolean waitForCompletion) at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.TryToRunInSTAThread(Action action, Boolean waitForCompletion) at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestInternalWithExecutors(IEnumerable1 executorUriExtensionMap, Int64 totalTests)
according to https://docs.microsoft.com/de-de/dotnet/core/testing/selective-unit-tests?pivots=mstest spaces and special characters can be encoded using “url style encoding” but that causes the same exception.
example: --filter “FullyQualifiedName=BackLook.Apps.WorkInProgress.Specs.Computing%20work%20in%20progress%20Project.No%20Workpackages%20exist”
i am using
<PackageReference Include="nunit" Version="3.12.0" />
<PackageReference Include="NUnit3TestAdapter" Version="4.0.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.3.0" />
and “dotnet cli” version 5.0.301
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 4
- Comments: 29 (13 by maintainers)
as far as i can see in the code the repro would probably sufficient to reproduce the issue, at least i see spaces in the test cases. the question is: how do u execute it. if we call it from “dotnet” CLI with a filter like
“–filter “FullyQualifiedName=<put some test name with spaces here>””
then we would probably get the same issue.
dotnet test --filter “FullyQualifiedName=Unit To Test Input A equals itself”
results in
putting the test name in quotes gives similar issues
@OsirisTerje I just updated VS to 16.11.5 and the problem still exists. I also doubt that the issue is related to VS but to unit3-vs-adapter. as described above my use case is NOT running tests with spaces in VS itself but on the console using dotnet CLI. The error message clearly indicates that the filter parser is not supporting my use case:
An exception occurred while invoking executor ‘executor://nunit3testexecutor/’: Unexpected Word ‘work’ at position 64 in selection expression. Stack trace: at NUnit.VisualStudio.TestAdapter.TestFilterConverter.TestFilterParser.Expect(Token[] valid) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\TestFilterConverter\TestFilterParser.cs:line 252 at NUnit.VisualStudio.TestAdapter.TestFilterConverter.TestFilterParser.Parse(String input) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\TestFilterConverter\TestFilterParser.cs:line 66 at NUnit.VisualStudio.TestAdapter.NUnitTestFilterBuilder.ConvertVsTestFilterToNUnitFilter(IVsTestFilter vsFilter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitTestFilterBuilder.cs:line 72
==> pls reopen the issue
thx
The problem seems to be in the
NUnitTestAdapter/TestFilterConverter/Tokenizer.cs:Tokenizer.GetNextToken()method, which does not support any quoting or escape characters (eg. backslash).Till then, a workaround is to AND together all the valid parts of the rhs value, like this:
(reported also in this SO)
@OsirisTerje i think the missing support is in the parser of the filter at least as far as i understood the callstack of the error i posted at the beginning of this thread
Tried to use new libraries again, but it still hasn’t been fixed So I’m back to good old NUnit 3.12.0 NUnit3TestAdapter 3.17.0 Whitespaces work there
@NastassiaDzianisava A small tip: When I create categories, I normally make them as typesafe classes, as shown in the figure below:
That way I avoid misspelling when they’re being used too.
Hi,
we have the same problem with version 4.1.0.