azure-pipelines-tasks: Release Definition Visual Studio Test task v2.* does not find test assemblies

Description

When trying to run Visual Studio Test task v. 2.* within some Release Definition (after the deployment task) it fails - test assemblies not found.


2018-03-11T13:45:58.9299389Z ##[section]Starting: VsTest - testAssemblies
2018-03-11T13:45:58.9308588Z ==============================================================================
2018-03-11T13:45:58.9308898Z Task         : Visual Studio Test
2018-03-11T13:45:58.9309304Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test runner. Test frameworks that have a Visual Studio test adapter such as xUnit, NUnit, Chutzpah, etc. can also be run. Tests can be distributed on multiple agents using this task (version 2).
2018-03-11T13:45:58.9309987Z Version      : 2.3.22
2018-03-11T13:45:58.9310218Z Author       : Microsoft Corporation
2018-03-11T13:45:58.9310499Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=835764)
2018-03-11T13:45:58.9310806Z ==============================================================================
2018-03-11T13:46:00.2276627Z Run the tests locally using vstest.console.exe
2018-03-11T13:46:00.2277087Z ========================================================
2018-03-11T13:46:00.2277775Z Test selector : Test assemblies
2018-03-11T13:46:00.2278155Z Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
2018-03-11T13:46:00.2278744Z Test filter criteria : TestCategory=GUITests
2018-03-11T13:46:00.2279050Z Search folder : D:\a\r1\a
2018-03-11T13:46:00.2279394Z Run settings file : D:\a\r1\a
2018-03-11T13:46:00.2279964Z Run in parallel : false
2018-03-11T13:46:00.2280285Z Run in isolation : false
2018-03-11T13:46:00.2281981Z Path to custom adapters : null
2018-03-11T13:46:00.2282309Z Other console options : null
2018-03-11T13:46:00.2282694Z Code coverage enabled : false
2018-03-11T13:46:00.2283486Z Rerun failed tests: false
2018-03-11T13:46:00.2283876Z VisualStudio version selected for test execution : 14.0
2018-03-11T13:46:00.3538282Z ========================================================
2018-03-11T13:46:00.3606879Z ##[warning]No test assemblies found matching the pattern: **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**.
2018-03-11T13:46:00.4897315Z ##[section]Finishing: VsTest - testAssemblies

The same task in a Build Definition runs just fine, thus tried to add a path to custom test assemblies from that successful Build Definition task (F:\VSTSBuild\workspace\8\s\VS$PROJECTNAME\bin\Release) but it does not help:

2018-03-11T11:24:14.1592030Z ##[section]Starting: VsTest - testAssemblies
2018-03-11T11:24:14.1599679Z ==============================================================================
2018-03-11T11:24:14.1600003Z Task         : Visual Studio Test
2018-03-11T11:24:14.1600437Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test runner. Test frameworks that have a Visual Studio test adapter such as xUnit, NUnit, Chutzpah, etc. can also be run. Tests can be distributed on multiple agents using this task (version 2).
2018-03-11T11:24:14.1600997Z Version      : 2.3.22
2018-03-11T11:24:14.1601205Z Author       : Microsoft Corporation
2018-03-11T11:24:14.1601467Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=835764)
2018-03-11T11:24:14.1601755Z ==============================================================================
2018-03-11T11:24:14.8054390Z Run the tests locally using vstest.console.exe
2018-03-11T11:24:14.8054796Z ========================================================
2018-03-11T11:24:14.8055468Z Test selector : Test assemblies
2018-03-11T11:24:14.8055818Z Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
2018-03-11T11:24:14.8056150Z Test filter criteria : TestCategory=GUITests
2018-03-11T11:24:14.8056455Z Search folder : D:\a\r1\a
2018-03-11T11:24:14.8056858Z Run settings file : D:\a\r1\a
2018-03-11T11:24:14.8058062Z Run in parallel : false
2018-03-11T11:24:14.8058411Z Run in isolation : false
2018-03-11T11:24:15.0560015Z ##[error]Error: Path to custom adapters '**F:\VSTSBuild\workspace\8\s\VS\$PROJECTNAME\bin\Release**' should be a directory and it should exist.
2018-03-11T11:24:15.1786711Z ##[section]Finishing: VsTest - testAssemblies

The ‘F:\VSTSBuild\workspace\8\s\VS$PROJECTNAME\bin\Release’ path taken from the logs of the Build Definition test task which runs properly.

It looks like “D:\a\r1\a” is the root directory and the path to the custom assembles is calculated related to the root directory - because ‘F:\VSTSBuild\workspace\8\s\VS$PROJECTNAME\bin\Release’ does exist on file system.

Is that possible to set the root directory in some way? Because considering even drive letters are different I cannot calculate the relative path properly.

Steps to reproduce

Create Visual Studio Test task v. 2.* within some Build Definition Agent Phase with criteria (TestCategory!=GUITests) and default assemblies names ($(BuildConfiguration)*test*.dll;-:\obj**) Create Visual Studio Test task v. 2.* within some Release Definition Agent Phase with criteria (TestCategory=GUITests) and default values for test assemblies (*test.dl, !*TestAdapter.dll, !\obj**)** Make Build Definition run Release Definition Run Build Definition

Expected behavior

Build Definition tests run Release definition test run

Actual behavior

Build Definition tests run Release definition test fails while not finding the test assemblies

Environment

VSTS 2017 Release Definition Visual Studio Test task v. 2.*

Account, project and definition names can be provided upon request

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 35 (20 by maintainers)

Most upvoted comments

@Koshak118 Can you run your release with release variable “system.debug=true” and provide us the logs at devops_tools[at]microsoft[dot]com. Thanks.