azure-pipelines-tasks: Selenium Tests are not running anymore

Required Information

Question, Bug, or Feature?
Type: Bug

Enter Task Name: VSTestV2 Version: 2.149.6

Environment

• Server - Azure Pipelines using agents on self-hosted
• Chrome Version: 73.0.3683.86 (64bits)
• chromedriver: 2.45.615291

Issue Description

We have a Visual Studio test suite running Selenium tests on Chrome (MSTest).

Starting two weeks ago, all our automated tests ran from Azure Devops stopped to pass. It seems something doesn’t succeed anymore to communicate between Chromedriver <=> Chrome and/or the “tests runner” <=> chromedriver. Error is Assert.Fail failed. Unexpected Error : The HTTP request to the remote WebDriver server for URL http://localhost:3723/session/c121c86b25ece8424dbb8771c2ef16b3/window/current/maximize timed out after 60 seconds.

If I remote connect to the host, and run the command in CLI via vstest.console.exe MyTestSuite.dll everything is working and passing well (so not issue about chromedriver and Chrome version apparently).

If I run tests from Azure Pipelines DTAExecutionHost is run and tests are timing out (see logs below).

Task logs

2019-04-02T07:02:51.4448821Z ##[section]Starting: Run UI Tests
2019-04-02T07:02:51.4466958Z ==============================================================================
2019-04-02T07:02:51.4467132Z Task         : Visual Studio Test
2019-04-02T07:02:51.4467246Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2019-04-02T07:02:51.4467491Z Version      : 2.149.6
2019-04-02T07:02:51.4467612Z Author       : Microsoft Corporation
2019-04-02T07:02:51.4467734Z Help         : [More information](https://go.microsoft.com/fwlink/?LinkId=835764)
2019-04-02T07:02:51.4467916Z ==============================================================================
2019-04-02T07:02:53.0105040Z SystemVssConnection exists true
2019-04-02T07:02:53.2177272Z SystemVssConnection exists true
2019-04-02T07:02:53.3607476Z SystemVssConnection exists true
2019-04-02T07:02:53.5279941Z In distributed testing flow
2019-04-02T07:02:53.5280227Z ======================================================
2019-04-02T07:02:53.5281101Z Test selector : Test plan
2019-04-02T07:02:53.5281452Z Test plan Id : 120
2019-04-02T07:02:53.5281784Z Test plan configuration Id : 1
2019-04-02T07:02:53.5282079Z Test suite Id selected: 121
2019-04-02T07:02:53.5282458Z Search folder : D:\agent-04\_work\r1\a
2019-04-02T07:02:53.5284903Z VisualStudio version selected for test execution : latest
2019-04-02T07:02:53.7884430Z Distributed test execution, number of agents in job : 1
2019-04-02T07:02:53.7922853Z Run in parallel : false
2019-04-02T07:02:53.7924129Z Run in isolation : false
2019-04-02T07:02:53.7927143Z Path to custom adapters : null
2019-04-02T07:02:53.7935703Z Other console options : null
2019-04-02T07:02:53.7939578Z Code coverage enabled : false
2019-04-02T07:02:53.7946849Z Diagnostics enabled : true
2019-04-02T07:02:53.8509935Z ======================================================
2019-04-02T07:02:53.8511227Z Source filter: **\Awesome.UI.Tests.dll
2019-04-02T07:02:53.9824089Z SystemVssConnection exists true
2019-04-02T07:02:53.9880445Z [command]D:\agent-04\_work\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.149.6\Modules\DTAExecutionHost.exe --inputFile D:\agent-04\_work\_temp\input_56299271-5515-11e9-b71c-292706c41bc5.json
2019-04-02T07:02:54.0719110Z ##########################################################################
2019-04-02T07:02:54.7242268Z ##[warning]Advanced diagnostics data collection requires test platform version 15.9 or higher.
2019-04-02T07:02:56.0021950Z ===========================================
2019-04-02T07:02:56.0022623Z AgentName: WIN-AFFCJLIC91K-r2d2-agent01-agent04-30
2019-04-02T07:02:56.0022887Z ServiceUrl: https://dev.azure.com/fake-name-cicd/
2019-04-02T07:02:56.0023055Z TestPlatformVersion: 15.8.0
2019-04-02T07:02:56.0023232Z EnvironmentUri: vstest://env/fake-name/_apis/release/1/1608/6123/1
2019-04-02T07:02:56.0023352Z QueryForTaskIntervalInMilliseconds: 3000
2019-04-02T07:02:56.0023469Z MaxQueryForTaskIntervalInMilliseconds: 10000
2019-04-02T07:02:56.0023602Z QueueNotFoundDelayTimeInMilliseconds: 3000
2019-04-02T07:02:56.0023722Z MaxQueueNotFoundDelayTimeInMilliseconds: 50000
2019-04-02T07:02:56.0023840Z ===========================================
2019-04-02T07:02:56.7024754Z TestExecutionHost.Execute: Registered TestAgent : 40 : WIN-AFFCJLIC91K-r2d2-agent01-agent04-30
2019-04-02T07:02:56.7739688Z IsValidServiceResponse: Received None command..Service Workflow is not active
2019-04-02T07:02:56.9372820Z Updated Run Settings:
2019-04-02T07:02:56.9406486Z <RunSettings>
2019-04-02T07:02:56.9407328Z   
2019-04-02T07:02:56.9407645Z   <RunConfiguration>
2019-04-02T07:02:56.9407924Z     <CustomSlicing enabled="true" maxagents="1" numberOfTestCasesPerSlice="0" isTimeBasedSlicing="False" sliceTime="0" />
2019-04-02T07:02:56.9409954Z     <BatchSize>1000</BatchSize>
2019-04-02T07:02:56.9410128Z     <ResultsDirectory>D:\agent-04\_work\_temp\TR_e9891b1e-51f9-4d63-9f1d-46e028f46084</ResultsDirectory>
2019-04-02T07:02:56.9410275Z   </RunConfiguration>
2019-04-02T07:02:56.9410416Z </RunSettings>
2019-04-02T07:02:56.9899216Z Run Settings File Path : C:\Windows\ServiceProfiles\NetworkService\AppData\Local\Temp\tmpF133.runsettings
2019-04-02T07:02:57.2364530Z Creating run for selected test plan with following parameters
2019-04-02T07:02:57.2368545Z Test plan ID: 120
2019-04-02T07:02:57.2368952Z Test suite ID: 121
2019-04-02T07:02:57.2370025Z Test configuration ID: 1
2019-04-02T07:02:58.1331909Z test configuration mapping: 
2019-04-02T07:02:58.1392831Z Run title: Tests UI
2019-04-02T07:02:58.1396801Z test settings id : 1000048
2019-04-02T07:02:58.1397364Z Build location: D:\agent-04\_work\r1\a
2019-04-02T07:02:58.1400193Z Build Id: 2322
2019-04-02T07:02:59.5081787Z Test run with Id 1000244 associated
2019-04-02T07:03:07.8123453Z Received the command : Start
2019-04-02T07:03:07.8154171Z TestExecutionHost.ProcessCommand. Start Command handled
2019-04-02T07:03:07.9677000Z Slice with id = 39, of type = 'Execution' received.
2019-04-02T07:03:08.2210822Z Count of test sources found: 1
2019-04-02T07:03:08.2651509Z =================================================================
2019-04-02T07:03:08.2657691Z Discovering tests from sources
2019-04-02T07:03:08.2740040Z Using new test platform for test execution
2019-04-02T07:03:11.3599956Z Number of testcases discovered : 11
2019-04-02T07:03:11.4083093Z Discovered tests 11 from sources
2019-04-02T07:03:11.4083984Z =================================================================
2019-04-02T07:03:11.4955691Z [RunStatistics]This execution slice with id '39', received '11' testcases to execute out of which '11' is discovered.
2019-04-02T07:03:11.5043401Z **************** Starting test execution *********************
2019-04-02T07:03:11.5208597Z Using new test platform for test execution
2019-04-02T07:06:44.1894038Z Failed    Awesome.UI.Tests.Connection.ConectionTests.ConnectionViaForm_UserWithLoginAndMdp_LogOnTheWall    ConnectionViaForm_UserWithLoginAndMdp_LogOnTheWall
2019-04-02T07:10:10.2223856Z Failed    Awesome.UI.Tests.Connection.ConectionTests.ConnectionViaO365_UserWithLoginAndMdp_LogOnTheWall    ConnectionViaO365_UserWithLoginAndMdp_LogOnTheWall
2019-04-02T07:13:36.3480819Z Failed    Awesome.UI.Tests.Connection.ConectionTests.ConnectionViaForm_InactiveUser_ErrorMessageDisplayed    ConnectionViaForm_InactiveUser_ErrorMessageDisplayed
2019-04-02T07:17:02.4611029Z Failed    Awesome.UI.Tests.Connection.ConectionTests.ConnectionViaForm_UserWithoutCompany_ErrorMessageDisplayed    ConnectionViaForm_UserWithoutCompany_ErrorMessageDisplayed
2019-04-02T07:20:28.4278358Z Failed    Awesome.UI.Tests.Sharing.SharingTests.Sharing_ConnectTwitterAccount_TwitterAccountIsConnected    Sharing_ConnectTwitterAccount_TwitterAccountIsConnected
2019-04-02T07:23:54.3835581Z Failed    Awesome.UI.Tests.Sharing.SharingTests.Sharing_ShareContentWithoutComment_ShareSucceded    Sharing_ShareContentWithoutComment_ShareSucceded
2019-04-02T07:27:20.3192832Z Failed    Awesome.UI.Tests.Sharing.SharingTests.Sharing_ShareContentWithComment_ShareSucceded    Sharing_ShareContentWithComment_ShareSucceded
2019-04-02T07:29:46.9786970Z ##[error]The operation was canceled.
2019-04-02T07:29:46.9844684Z ##[section]Finishing: Run UI Tests

For information, when logged in to the host, I can see in task manager all the 3 processes (DTAExecutionHost, chromedriver and Chrome, but no “activity” like CPU or memory moving):

Troubleshooting

Using system.debug flag, there is no interesting additional log.

Please, advice what could be wrong 😃