runtime: Environment.Exit hanging in RemoteExecutor

https://dev.azure.com/dnceng/public/_build/results?buildId=380511&view=ms.vss-test-web.build-test-results-tab&runId=11763518&paneView=debug&resultId=147963

Configuration: netcoreapp-Windows_NT-Debug-x64-Windows.81.Amd64.Open

System.Net.Tests.AuthenticationManagerTest.CustomTargetNameDictionary_ValidCollection

Timed out at 10/7/2019 8:39:55 PM after 60000ms waiting for remote process.\r\n\tProcess ID: 5404\r\n\tHandle: 1096\r\n\tName: dotnet\r\n\tMainModule: C:\\dotnetbuild\\work\\ddad0c4a-3d2b-409b-b8b2-09e253a66230\\Payload\\dotnet.exe\r\n\tStartTime: 10/7/2019 8:38:55 PM\r\n\tTotalProcessorTime: 00:00:00.1875000\r\n\tThreads:\r\n\t\tThread dotnet/corefx#1 (OS 0x152C) \r\n\t\t\t[InlinedCallFrame] (System.Environment._Exit)\r\n\t\t\t[InlinedCallFrame] (System.Environment._Exit)\r\n\t\t\tSystem.Environment.Exit(Int32)\r\n\t\t\tMicrosoft.DotNet.RemoteExecutor.Program.Main(System.String[])\r\n\t\t\t[GCFrame]\r\n\t\t\t[GCFrame]\r\n\t\tThread dotnet/corefx#2 (OS 0x1538) [Finalizer]\r\n\t\t\t[DebuggerU2MCatchHandlerFrame]\r\n\t\tThread dotnet/corefx#3 (OS 0x1528) [Thread pool worker]\r\n
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 131
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose() in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 55
   at System.Net.Tests.AuthenticationManagerTest.CustomTargetNameDictionary_ValidCollection() in /_/src/System.Net.Requests/tests/AuthenticationManagerTest.cs:line 93

cc @davidsh, @wfurt, @scalablecory, @eiriktsarpalis, @karelz cc @stephentoub for another hang

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 28 (28 by maintainers)

Most upvoted comments

Or we can call MiniDumpWriteDump to create the dump

That’s a good idea.

If @MattGal can tell us where we would need to save the dump for it to be discovered, I can make the relevant change.

stephentoub on Oct 9, 2019

My guess is that it is hanging due to bugs(?) with RemoteExecutor or perhaps the CI machine is way too busy for dispatch a new process to run the RemoteExecutor.

I’m fairly sure it’s neither. It looks like a hang in Environment.Exit. This has been happening with some frequency in CI over the past week at least.

      		Thread dotnet/corefx#1 (OS 0x152C) 
      			[InlinedCallFrame] (System.Environment._Exit)
      			[InlinedCallFrame] (System.Environment._Exit)
      			System.Environment.Exit(Int32)
      			Microsoft.DotNet.RemoteExecutor.Program.Main(System.String[])
      			[GCFrame]
      			[GCFrame]
      		Thread dotnet/corefx#2 (OS 0x1538) [Finalizer]
      			[DebuggerU2MCatchHandlerFrame]
      		Thread dotnet/corefx#3 (OS 0x1528) [Thread pool worker]

stephentoub on Oct 7, 2019