WindowsAppSDK: MSTest Discovery Broken with 1.1.x

Describe the bug

After updating to 1.1.1/1.1.2 from 1.0.3 MSTest discovery is now broken. This is a blocking issue for us.

We couldn’t test in our CI before because we were blocked by #2631 as well. I tried in the CI, but it seems there may have been something missed with the fix for #2631 as it’s trying to install 1.1.1 still for MSTest, see comment here on that issue.

Locally I see the same behavior still I did with 1.1.1:

========== Starting test discovery ==========
[MSTest][Discovery][H:\...\tests\SizerBase.UnitTests.WinAppSdk\bin\x64\Debug\net6.0-windows10.0.19041.0\SizerBase.UnitTests.WinAppSdk.dll] Failed to discover tests from assembly H:\...\tests\SizerBase.UnitTests.WinAppSdk\bin\x64\Debug\net6.0-windows10.0.19041.0\SizerBase.UnitTests.WinAppSdk.dll. Reason:File does not exist: H:\...\tests\SizerBase.UnitTests.WinAppSdk\bin\x64\Debug\net6.0-windows10.0.19041.0\SizerBase.UnitTests.WinAppSdk.dll
No test is available in H:\...\tests\SizerBase.UnitTests.WinAppSdk\bin\x64\Debug\net6.0-windows10.0.19041.0\SizerBase.UnitTests.WinAppSdk.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
========== Test discovery finished: 0 Tests found in 5.1 sec ==========

The file exists on disk, everything built. Only change is updating NuGet reference from 1.0.3 to 1.1.x causes tests now to not be discovered properly/run.

Steps to reproduce the bug

Create a MSTest based unit test project for WinAppSDK with 1.0.3
Run tests.
Close VS
Clean environment
Update to 1.1.x
Re-run tests (fail)

Note, you have to close VS/clean environment as VS caches test discovery, didn’t detect this issue at first on initial update as VS used prior cache to run tests. Issue seems to be with test discovery itself.

Expected behavior

Able to run MSTest projects as was able to with 1.0.3.

Screenshots

No response

NuGet package version

1.1.1

Packaging type

No response

Windows version

Windows 10 version 21H2 (19044, November 2021 Update)

IDE

Visual Studio 2022

Additional context

No response

About this issue

Original URL
State: open
Created 2 years ago
Reactions: 2
Comments: 17 (6 by maintainers)

Commits related to this issue

Update MSTest to 2.2.10 See https://github.com/microsoft/WindowsAppSDK/issues/2706 Seems like discovery works, but now `Window.Current` fails to activate. — committed to CommunityToolkit/Labs-Windows by michael-hawker 2 years ago
Update MSTest to 2.2.10 See https://github.com/microsoft/WindowsAppSDK/issues/2706 Seems like discovery works, but now `Window.Current` fails to activate. — committed to CommunityToolkit/Labs-Windows by michael-hawker 2 years ago
Update MSTest to 2.2.10 See https://github.com/microsoft/WindowsAppSDK/issues/2706 Seems like discovery works, but now `Window.Current` fails to activate. — committed to CommunityToolkit/Labs-Windows by michael-hawker 2 years ago

Most upvoted comments

Can reproduce. Here’s log output from my test project w/ verbosity cranked up. Not very helpful.

[7/5/2022 2:45:33.994 PM]  Operation Dequeue : 'RunFromCodeLensOperation'
[7/5/2022 2:45:33.999 PM]  Building Test Projects
[7/5/2022 2:45:34.070 PM]  Start building 1 containers.
[7/5/2022 2:45:34.070 PM]  Triggering build for 1 IProjectBasedTestContainers.
[7/5/2022 2:45:35.599 PM]  No IBuildableTestContainers were found.
[7/5/2022 2:45:35.599 PM]  Completed building containers.
[7/5/2022 2:45:35.599 PM]  Start updating 1 containers.
[7/5/2022 2:45:35.599 PM]  Updating container C:\Sources\was-repros\was_issue_2706\tests\testproject\bin\Debug\net6.0-windows10.0.19041.0\testproject.dll.
[7/5/2022 2:45:35.615 PM]  Successfully refreshed container: C:\Sources\was-repros\was_issue_2706\tests\testproject\bin\Debug\net6.0-windows10.0.19041.0\testproject.dll
[7/5/2022 2:45:35.615 PM]  Completed updating containers.
[7/5/2022 2:45:35.615 PM]  DiscoveryOperation<RunFromCodeLensOperation> Changed container count is 1
[7/5/2022 2:45:35.654 PM]  Test run animation has started.
[7/5/2022 2:45:35.663 PM]  Test run progress has started.
[7/5/2022 2:45:35.663 PM]  Discovering the following containers :
[7/5/2022 2:45:35.663 PM]  	C:\Sources\was-repros\was_issue_2706\tests\testproject\bin\Debug\net6.0-windows10.0.19041.0\testproject.dll
[7/5/2022 2:45:35.765 PM]  Starting test discovery for requested test run
[7/5/2022 2:45:35.921 PM]  Project testproject references test adapter: MSTest.TestAdapter, version 2.2.8
[7/5/2022 2:45:36.123 PM]  ========== Starting test discovery ==========
[7/5/2022 2:45:36.124 PM]  File timestamp changed from 7/5/2022 2:34:56 PM to 7/5/2022 2:45:35 PM for C:\Sources\was-repros\was_issue_2706\tests\testproject\bin\Debug\net6.0-windows10.0.19041.0\testproject.dll
[7/5/2022 2:45:36.126 PM]  Grouped C:\Sources\was-repros\was_issue_2706\tests\testproject\bin\Debug\net6.0-windows10.0.19041.0\testproject.dll : (AnyCPU, FrameworkCore10, net6.0, )
[7/5/2022 2:45:36.128 PM]  Test discovery settings for C:\Sources\was-repros\was_issue_2706\tests\testproject\bin\Debug\net6.0-windows10.0.19041.0\testproject.dll:
 <RunSettings>
  <BoostTestInternalSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <VSProcessId>28340</VSProcessId>
  </BoostTestInternalSettings>
  <RunConfiguration>
    <ResultsDirectory>C:\Sources\was-repros\was_issue_2706\TestResults</ResultsDirectory>
    <SolutionDirectory>C:\Sources\was-repros\was_issue_2706\</SolutionDirectory>
    <CollectSourceInformation>False</CollectSourceInformation>
  </RunConfiguration>
  <GoogleTestAdapterSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <SolutionSettings>
      <Settings />
    </SolutionSettings>
    <ProjectSettings />
  </GoogleTestAdapterSettings>
</RunSettings>.
[7/5/2022 2:45:36.565 PM] [Warning] [MSTest][Discovery][C:\Sources\was-repros\was_issue_2706\tests\testproject\bin\Debug\net6.0-windows10.0.19041.0\testproject.dll] Failed to discover tests from assembly C:\Sources\was-repros\was_issue_2706\tests\testproject\bin\Debug\net6.0-windows10.0.19041.0\testproject.dll. Reason:File does not exist: C:\Sources\was-repros\was_issue_2706\tests\testproject\bin\Debug\net6.0-windows10.0.19041.0\testproject.dll
[7/5/2022 2:45:36.566 PM] [Warning] No test is available in C:\Sources\was-repros\was_issue_2706\tests\testproject\bin\Debug\net6.0-windows10.0.19041.0\testproject.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[7/5/2022 2:45:36.590 PM]  ========== Test discovery finished: 0 Tests found in 466.1 ms ==========
[7/5/2022 2:45:36.902 PM]  ========== Starting test run ==========
[7/5/2022 2:45:36.904 PM]  Attachments processing not needed
[7/5/2022 2:45:36.906 PM]  ========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========
[7/5/2022 2:45:36.989 PM]  Test run animation has stopped.
[7/5/2022 2:45:37.021 PM]  Test run progress has stopped.

riverar on Jul 5, 2022