vstest: Microsoft.TestPlatform (17.4.0-preview-20220726-02) not working

Description

Since updating to the latest version of Microsoft.TestPlatform, unittests don’t run anymore in Azure DevOps pipelines. It seems as if there are some assembly reference mismatches, and test attributes are no longer recognized.

https://www.nuget.org/packages/Microsoft.TestPlatform 17.4.0-preview-20220726-02 <== error 17.2.0 (latestStable) <== works 17.4.0-preview-20220707-01 <== works

Steps to reproduce

Build pipeline step:

- task: VisualStudioTestPlatformInstaller@1
  displayName: 'Install Test Platform'

Expected behavior

Tests running fine.

Actual behavior

example 1:

  Failed TestFileArchiveOptions
  Error Message:
   Unable to set TestContext property for the class xxx.FileArchiveTest. Error: System.ArgumentException: Object of type 'Microsoft.VisualStudio.TestPlatform.MSTestFramework.TestContextImplementation' cannot be converted to type 'Microsoft.VisualStudio.TestTools.UnitTesting.TestContext'..
  Stack Trace:
      at System.RuntimeType.TryChangeType(Object value, Binder binder, CultureInfo culture, Boolean needsSpecialCast)
   at System.Reflection.MethodBase.CheckArguments(Object[] parameters, Binder binder, BindingFlags invokeAttr, CultureInfo culture, Signature sig)
   at System.Reflection.RuntimeMethodInfo.InvokeArgumentsCheck(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.RuntimePropertyInfo.SetValue(Object obj, Object value, Object[] index)
   at Microsoft.VisualStudio.TestPlatform.MSTestFramework.TestMethodRunner.SetTestContext()

The test class in question is fine:

    [TestClass]
    public class FileArchiveTest 
    {
        public TestContext TestContext { get; set; }

example 2:

  Failed RunActivity_SingleActivityWithBaggage_BaggageIsSet
  Error Message:
   Method xxx.ClassInitialize has wrong signature. The method must be static, public, does not return a value and should take a single parameter of type TestContext.

the method looks fine:

        [ClassInitialize]
        public static void ClassInitialize(TestContext context)

example 3:

  Failed TarjanTest5 [1 ms]
  Error Message:
   Test method xxx.TopoSortTest.TarjanTest5 threw exception: 
System.InvalidOperationException: Graph is not complete. Missing node 'v7' in adjacency list. Check if node is defined as key in the dictionary.

the ExpectedException attibute is not honored:

        [TestMethod]
        [ExpectedException(typeof(InvalidOperationException))]
        public void TarjanTest5()
        {

Diagnostic logs

failing output:

##[section]Starting: Install Test Platform
==============================================================================
Task         : Visual Studio test platform installer
Description  : Acquire the test platform from nuget.org or the tool cache. Satisfies the ‘vstest’ demand and can be used for running tests and collecting diagnostic data using the Visual Studio Test task.
Version      : 1.151.3
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/tool/vstest-platform-tool-installer
==============================================================================
Starting VsTest platform tools installer task.
==============================================================================
Looking for the latest pre-release version of the Microsoft.Testplatform.
[command]C:\CI\7\_work\_tasks\VisualStudioTestPlatformInstaller_2c65196a-54fd-4a02-9be8-d9d1837b7111\1.151.3\nuget.exe list packageid:Microsoft.TestPlatform -PreRelease -NonInteractive -Source https://api.nuget.org/v3/index.json
Microsoft.TestPlatform 17.4.0-preview-20220726-02
Found tool in cache: VsTest 17.4.0-preview-20220726-02 x64
VsTest will use the Test Platform package found in C:\CI\7\_work\_tool\VsTest\17.4.0-preview-20220726-02\x64
##[section]Finishing: Install Test Platform

working output:

##[section]Starting: Install Test Platform
==============================================================================
Task         : Visual Studio test platform installer
Description  : Acquire the test platform from nuget.org or the tool cache. Satisfies the ‘vstest’ demand and can be used for running tests and collecting diagnostic data using the Visual Studio Test task.
Version      : 1.151.3
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/tool/vstest-platform-tool-installer
==============================================================================
Starting VsTest platform tools installer task.
==============================================================================
Looking for the latest pre-release version of the Microsoft.Testplatform.
[command]C:\CI\1\_work\_tasks\VisualStudioTestPlatformInstaller_2c65196a-54fd-4a02-9be8-d9d1837b7111\1.151.3\nuget.exe list packageid:Microsoft.TestPlatform -PreRelease -NonInteractive -Source https://api.nuget.org/v3/index.json
Microsoft.TestPlatform 17.4.0-preview-20220707-01
Found tool in cache: VsTest 17.4.0-preview-20220707-01 x64
VsTest will use the Test Platform package found in C:\CI\1\_work\_tool\VsTest\17.4.0-preview-20220707-01\x64
##[section]Finishing: Install Test Platform

Environment

Azure Devops pipelines

Workarouds:

use earlier versions:

- task: VisualStudioTestPlatformInstaller@1
  displayName: 'Install Test Platform (latestStable!)'
  inputs:  
    packageFeedSelector: 'nugetOrg'
    versionSelector: 'latestStable'

or

- task: VisualStudioTestPlatformInstaller@1
  displayName: 'Install Test Platform (pinned version!)'
  inputs:
    packageFeedSelector: 'nugetOrg'
    versionSelector: 'specificVersion'
    testPlatformVersion: '17.4.0-preview-20220707-01'