testfx: Unable to run unit tests remotely on Test Agent using MSTest V2

Description

I am trying to run my unit tests with UI automation on a Visual Studio Test Controller and Visual Studio Test Agent but I get the following warning Warning : A testsettings file or a runsettings with a ForcedLegacyMode set to true is not supported with the MSTest V2 Adapter. and none of my tests load.

Steps to reproduce

• Add a .testsettings file to your solution
• Add a .runsettings file to your solution
• Add a SettingsFile element to the MSTest element in the .runsettings
• Add a ForceLegacyMode element to the MSTest element in the .runsettings
• Select the .runsettings as the active test settings (Test -> Test Settings -> Select Test Settings File)
• Rebuild Solution

Expected behavior

Tests are loaded and I am able to run my unit tests on the Test Controller. I understand the legacy mode not being supported but it seems like the .runsettings should also have a means for me to specify the host for the test controller.

Actual behavior

Tests are not loaded and I get the following warnings. When I remove loading the SettingsFile and the ForceLegacyMode elements my tests load but then I won’t be able to connect to the controller.

[11/22/2017 9:54:12 AM Warning] Index was outside the bounds of the array.
[11/22/2017 9:54:13 AM Warning] Index was outside the bounds of the array.
[11/22/2017 9:54:13 AM Warning] Index was outside the bounds of the array.
[11/22/2017 9:54:14 AM Warning] Warning : A testsettings file or a runsettings with a ForcedLegacyMode set to true is not supported with the MSTest V2 Adapter.

Environment

Just to clarify I am not doing coded UI tests, I am running unit tests that are using Microsoft’s UIA through a open source project called FlaUI. I am trying to use the test controller as part of our CD CI pipeline to run out UI automation on producing builds.

Here are my .runsettings and .testsettings

.runsettings

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <!-- Configurations that affect the Test Framework -->
  <RunConfiguration>
    <MaxCpuCount>1</MaxCpuCount>
    <!-- Path relative to solution directory -->
    <ResultsDirectory>.\TestResults</ResultsDirectory>

    <!-- [x86] | x64    
      - You can also change it from menu Test, Test Settings, Default Processor Architecture -->
    <TargetPlatform>x86</TargetPlatform>

    <!-- Framework35 | [Framework40] | Framework45 -->
    <TargetFrameworkVersion>Framework45</TargetFrameworkVersion>
  </RunConfiguration>

  <!-- Configurations for data collectors -->
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <Configuration>
          <CodeCoverage>
            <ModulePaths>
              <Exclude>
                <ModulePath>.*CPPUnitTestFramework.*</ModulePath>
              </Exclude>
            </ModulePaths>

            <!-- We recommend you do not change the following values: -->
            <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
            <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
            <CollectFromChildProcesses>True</CollectFromChildProcesses>
            <CollectAspDotNet>False</CollectAspDotNet>

          </CodeCoverage>
        </Configuration>
      </DataCollector>

    </DataCollectors>
  </DataCollectionRunSettings>

  <!-- Parameters used by tests at runtime -->
  <TestRunParameters>
  </TestRunParameters>

  <!-- Adapter Specific sections -->

  <!-- MSTest adapter -->
  <MSTest>
    <MapInconclusiveToFailed>True</MapInconclusiveToFailed>
    <CaptureTraceOutput>false</CaptureTraceOutput>
    <DeleteDeploymentDirectoryAfterTestRunIsComplete>False</DeleteDeploymentDirectoryAfterTestRunIsComplete>
    <DeploymentEnabled>False</DeploymentEnabled>
    <SettingsFile>test.testsettings</SettingsFile>
    <ForcedLegacyMode>true</ForcedLegacyMode>
    <AssemblyResolution>
      <Directory Path=".\Tests" includeSubDirectories="true"/>
    </AssemblyResolution>
  </MSTest>

</RunSettings>

.testsettings

<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="rmoqa01" id="076be28c-d18b-46bf-ad20-4d43ec821ea4" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Description>These are default test settings for a local test run.</Description>
  <RemoteController name="10.2.0.101" />
  <Execution location="Remote">
    <Hosts skipUnhostableTests="false">
      <VSSDKTestHostRunConfig name="VS IDE" HiveKind="DevEnv" HiveName="15.0_c9b36733" xmlns="http://microsoft.com/schemas/VisualStudio/SDK/Tools/IdeHostAdapter/2006/06" />
    </Hosts>
    <TestTypeSpecific>
      <UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">
        <AssemblyResolution>
          <TestDirectory useLoadContext="true" />
        </AssemblyResolution>
      </UnitTestRunConfig>
    </TestTypeSpecific>
    <AgentRule name="AllAgentsDefaultRole">
    </AgentRule>
  </Execution>
  <Properties />
</TestSettings>