azure-pipelines-tasks: VSTest@2: Unable to test .NET 6, vstest.console not found

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: VsTestV2

Environment

Azure Pipelines
Agent - Hosted: vmImage: ‘windows-2022’

Issue Description

I want to upgrade my .NET 5 solution to .NET 6. After every thing works local (VS 2022 Enterprise) I tried to run our Azure Pipeline which builds, tests and publishes the application. After switching the host from “windows-latest” to “windows-2022” the solution could be build with the pipeline, but the tests could not be executed:

… unable to find vstest.console from a visual studio installation …

I just found a commit “VsTest task changes to support 2022”, but it seems, that this update is no available in the pipelines yet?

- task: VSTest@2
  displayName: 'Run tests & publish results"'
  inputs:
    testRunTitle: 'Hardware Independent'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'
    testAssemblyVer2: '$(testDll)'
    codeCoverageEnabled: true
    runInParallel: true
    rerunFailedTests: true
    rerunMaxAttempts: 3
    testFiltercriteria: 'TestCategory~hardware independent'

Task logs

Starting: Run tests & publish results"
==============================================================================
Task         : Visual Studio Test
Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
Version      : 2.170.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/vstest
==============================================================================
SystemVssConnection exists true
SystemVssConnection exists true
Running tests using vstest.console.exe runner.
======================================================
Test selector : Test assemblies
Test filter criteria : TestCategory~hardware independent
Search folder : D:\a\1\s
Action when minimum tests threshold not met : donothing
Minimum tests expected to be run: 0
VisualStudio version selected for test execution : 16.0
Attempting to find vstest.console from a visual studio installation with version [15.0,17.0).
Attempting to find vstest.console from a visual studio build tools installation with version [15.0,17.0).
##[error]Error: Visual Studio NaN is not found. Try again with a version that exists on your build agent machine.
Finishing: Run tests & publish results"

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 6
Comments: 20 (4 by maintainers)

Most upvoted comments

I didn’t do anything. The vstest task version just changed:

2021-12-01T08:00:19.5451609Z ##[section]Starting: Test Assemblies
2021-12-01T08:00:19.5633250Z ==============================================================================
2021-12-01T08:00:19.5633525Z Task         : Visual Studio Test
2021-12-01T08:00:19.5634038Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2021-12-01T08:00:19.5634654Z Version      : 2.195.0
2021-12-01T08:00:19.5634936Z Author       : Microsoft Corporation
2021-12-01T08:00:19.5636235Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/vstest
2021-12-01T08:00:19.5636810Z ==============================================================================

But now, I have other problem:

Test run detected DLL(s) which were built for different framework and platform versions. Following DLL(s) do not match current settings, which are .NETFramework,Version=v4.0 framework and X86 platform.
......Test.dll is built for Framework .NETCoreApp,Version=v6.0 and Platform AnyCPU.

Obviously, there aren’t any netframework x86 dll in my test project. 🥲

kindlaj on Dec 1, 2021

I just kicked a new pipeline run again and we’re still getting 2.170.1. Our task is just requiring VSTest@2, so not sure why we seem to be getting a different version that you are 🤔

Sergio0694 on Dec 3, 2021

Our builds are still getting task version 2.170.1 and the task fails if I try to specify 2.195.0 . @ejszi , how did you get the new version of the task to run?

aarondandy on Nov 30, 2021

With 2.195.0 task version, it’s resolved. Thanks!

kindlaj on Nov 23, 2021

@MatthewSteeples wasn’t resolved. Version of the image is 20211109.2 but version of the task is still Version : 2.170.1

MrImpossibru on Nov 18, 2021