azure-pipelines-tasks: Test run fails but it doesn't fail the task - Task keeps running forever unless manual timeout is configured

Environment

• Server - Azure Pipelines or TFS on-premises? Azure Pipelinens

  • If using Azure Pipelines, provide the account name, team project name, build definition name/build number:

• Agent - Hosted or Private:

  • If using private agent, provide the OS of the machine running the agent and the agent version: [AGENT_VERSION] --> [2.123.0]

Windows Server 2016- Azure IaaS VM

Issue Description

Test run fails but it doesn’t fails the task, so unless we specify a manual timeout control for the Task itself it runs forever, setting up manual timeout for the task is not ideal either.

This behaviour was different on v1 of this task. When all tests have failed it would fail the task as well and the release would keep continuing.

Task logs

ReleaseLogs_4739.zip

Error logs

[Insert error from the logs here for a quick overview]

2018-10-30T01:30:12.0126231Z Microsoft ® Test Execution Command Line Tool Version 15.8.0 2018-10-30T01:30:12.0126231Z Copyright © Microsoft Corporation. All rights reserved. 2018-10-30T01:30:12.0126231Z 2018-10-30T01:30:12.0276223Z vstest.console.exe 2018-10-30T01:30:12.0276223Z “C:\agent_work\r1\a\Well - Application Build-(Dev Branch)\TEST DLL\bin\Release\Flexstar.AcceptanceTests.dll” 2018-10-30T01:30:12.0276223Z /TestCaseFilter:“Name=UISmokeTest” 2018-10-30T01:30:12.0276223Z /Settings:“C:\agent_work_temp\5754e240-dbe3-11e8-84c9-7d084e19c408.runsettings” 2018-10-30T01:30:12.0276223Z /logger:“trx” 2018-10-30T01:30:12.0276223Z /TestAdapterPath:“C:\agent_work\r1\a\Well - Application Build-(Dev Branch)\TEST DLL\bin\Release” 2018-10-30T01:30:12.0276223Z /diag:“C:\agent_work_temp\57463c40-dbe3-11e8-84c9-7d084e19c408.txt” 2018-10-30T01:30:12.7776367Z Starting test execution, please wait… 2018-10-30T01:30:12.7776367Z Logging Vstest Diagnostics in file: C:\agent_work_temp\57463c40-dbe3-11e8-84c9-7d084e19c408.txt 2018-10-30T01:30:16.4966816Z Starting ChromeDriver 2.40.565498 (ea082db3280dd6843ebfb08a625e3eb905c4f5ab) on port 57178 2018-10-30T01:30:16.4966816Z Only local connections are allowed. 2018-10-30T01:32:23.6242154Z Failed UISmokeTest 2018-10-30T01:32:23.6436250Z Error Message: 2018-10-30T01:32:23.6436250Z Class Initialization method Flexstar.AcceptanceTests.UITests.SmokeTests.UISmokeTests.SetupTests threw exception. OpenQA.Selenium.WebDriverException: OpenQA.Selenium.WebDriverException: The HTTP request to the remote WebDriver server for URL http://localhost:57178/session/6b6655e577621055885369047d074fde/execute_async timed out after 120 seconds. —> System.Net.WebException: The operation has timed out. 2018-10-30T01:32:23.6436250Z Stack Trace: 2018-10-30T01:32:23.6436250Z at OpenQA.Selenium.Remote.HttpCommandExecutor.MakeHttpRequest(HttpRequestInfo requestInfo) 2018-10-30T01:32:23.6436250Z at OpenQA.Selenium.Remote.HttpCommandExecutor.Execute(Command commandToExecute) 2018-10-30T01:32:23.6436250Z at OpenQA.Selenium.Remote.DriverServiceCommandExecutor.Execute(Command commandToExecute) 2018-10-30T01:32:23.6436250Z at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters) 2018-10-30T01:32:23.6436250Z at OpenQA.Selenium.Remote.RemoteWebDriver.ExecuteScriptCommand(String script, String commandName, Object[] args) 2018-10-30T01:32:23.6436250Z at OpenQA.Selenium.Remote.RemoteWebDriver.ExecuteAsyncScript(String script, Object[] args) 2018-10-30T01:32:23.6436250Z at Protractor.NgWebDriver.set_Url(String value) in c:\Users\Bruno\Projets\GitHub\bbaia\protractor-net\src\Protractor\NgWebDriver.cs:line 163 2018-10-30T01:32:23.6436250Z at Flexstar.AcceptanceTests.WellFunctions.initializeDriverAndNavigate(String url, Int32 timeoutSeconds) in C:\agent_work\3\s\Development-branch\Flexstar.AcceptanceTests\WellFunctions.cs:line 103 2018-10-30T01:32:23.6436250Z at Flexstar.AcceptanceTests.UITests.SmokeTests.UISmokeTests.SetupTests(TestContext testContext) in C:\agent_work\3\s\Development-branch\Flexstar.AcceptanceTests\UITests\UISmokeTests.cs:line 37 2018-10-30T01:32:23.6436250Z 2018-10-30T01:32:23.6436250Z Standard Output Messages: 2018-10-30T01:32:23.6436250Z Navigating to: https://*****************/ 2018-10-30T01:32:23.6436250Z timeoutSeconds: 120 2018-10-30T01:32:23.6436250Z 2018-10-30T01:32:23.6436250Z 2018-10-30T01:32:23.6711037Z Results File: C:\agent_work\r1\a\TestResults\BuildSvcAcct_DEV-TEST-UAT-VM_2018-10-30_12_32_23.trx 2018-10-30T01:32:23.6711037Z 2018-10-30T01:32:23.6711037Z Total tests: 1. Passed: 0. Failed: 1. Skipped: 0. 2018-10-30T01:32:23.6711037Z Test Run Failed. 2018-10-30T01:32:23.6711037Z Test execution time: 2.1709 Minutes 2018-10-30T01:34:04.8322974Z ##[error]The task has timed out. 2018-10-30T01:34:04.8322974Z ##[section]Finishing: UISmokeTest - Task v2