vstest: The active test run was aborted. Reason: Test host process crashed

Description

For a defect reproducable from the vstest command line, describe the issue you’ve observed.

Steps to reproduce

git clone https://github.com/xuzhg/WebApi.git git checkout -b RoutingBaseForNEtCore30 origin/RoutingBaseForNEtCore30
dotnet --version
run the following command (please replace the following Folder): dotnet test [YOURLocalClonedFolder]\test\E2ETest\Microsoft.Test.E2E.AspNet.OData\Build.AspNetCore3x\Microsoft.Test.E2E.AspNetCore3x.OData.csproj --logger trx --results-directory C:\BuildAgent\_work\_temp --configuration release --no-build -v diag
result:

Expected behavior

Shall pass all test cases running

Actual behavior

Crash with the error message:

    <RunInfos>
      <RunInfo computerName="ODataBuild50" outcome="Error" timestamp="2019-12-02T20:25:22.4166882+00:00">
        <Text>The active test run was aborted. Reason: Test host process crashed : Unhandled exception. Unhandled exception. System.Threading.Tasks.TaskCanceledException: The operation was canceled.&#xD;
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)&#xD;
   at Microsoft.Test.E2E.AspNet.OData.QueryComposition.SecurityTests.TestDosAttack(String filter) in C:\BuildAgent\_work\1\s\test\E2ETest\Microsoft.Test.E2E.AspNet.OData\QueryComposition\SecurityTests.cs:line 85&#xD;
   at Microsoft.Test.E2E.AspNet.OData.QueryComposition.SecurityTests.&lt;&gt;c__DisplayClass9_0.&lt;&lt;TestDosAttackWithMultipleThreads&gt;b__0&gt;d.MoveNext() in C:\BuildAgent\_work\1\s\test\E2ETest\Microsoft.Test.E2E.AspNet.OData\QueryComposition\SecurityTests.cs:line 95&#xD;
--- End of stack trace from previous location where exception was thrown ---&#xD;
   at System.Threading.Tasks.Task.&lt;&gt;c.&lt;ThrowAsync&gt;b__139_1(Object state)&#xD;
   at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()&#xD;
   at System.Threading.ThreadPoolWorkQueue.Dispatch()&#xD;
System.Threading.Tasks.TaskCanceledException: The operation was canceled.&#xD;
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)&#xD;
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)&#xD;
   at Microsoft.Test.E2E.AspNet.OData.QueryComposition.SecurityTests.TestDosAttack(String filter) in C:\BuildAgent\_work\1\s\test\E2ETest\Microsoft.Test.E2E.AspNet.OData\QueryComposition\SecurityTests.cs:line 85&#xD;
   at Microsoft.Test.E2E.AspNet.OData.QueryComposition.SecurityTests.&lt;&gt;c__DisplayClass9_0.&lt;&lt;TestDosAttackWithMultipleThreads&gt;b__0&gt;d.MoveNext() in C:\BuildAgent\_work\1\s\test\E2ETest\Microsoft.Test.E2E.AspNet.OData\QueryComposition\SecurityTests.cs:line 95&#xD;
--- End of stack trace from previous location where exception was thrown ---&#xD;
   at System.Threading.Tasks.Task.&lt;&gt;c.&lt;ThrowAsync&gt;b__139_1(Object state)&#xD;
   at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()&#xD;
   at System.Threading.ThreadPoolWorkQueue.Dispatch()&#xD;
</Text>
      </RunInfo>
    </RunInfos>

Environment

Please share additional details about the test environment.

only installed VS2019 Enterprise.

Additional Info

If i run the failure test case individually, it can pass:

as:

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 9
Comments: 49 (3 by maintainers)

Most upvoted comments

We are getting this issue from time to time on our build server, too. It’s not consistent and sometimes it works just fine. We didn’t see before when we were running core 2.2, but now see when we are running 3.1

+19

MarioAtTrimble on Feb 11, 2020

We are using vstest indirectly (via Microsoft.NET.Test.Sdk), and I have upgraded it to 16.5.0, and yet the error persists.

> A total of 1 test files matched the specified pattern.
> The active test run was aborted. Reason: Test host process crashed : Fatal error. Internal CLR error. (0x80131506)
>    at System.Reflection.RuntimeModule.GetTypes(System.Reflection.RuntimeModule)
>    at System.Reflection.RuntimeAssembly.get_DefinedTypes()
>    at Xunit.Runner.VisualStudio.VsTestRunner.GetAvailableRunnerReporters(System.Collections.Generic.IEnumerable`1<System.String>)
>    at Xunit.Runner.VisualStudio.VsTestRunner+<>c__DisplayClass23_0.<GetRunnerReporter>b__0()
>    at System.Lazy`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ViaFactory(System.Threading.LazyThreadSafetyMode)
>    at System.Lazy`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ExecutionAndPublication(System.LazyHelper, Boolean)
>    at System.Lazy`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].CreateValue()
>    at System.Lazy`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].get_Value()
>    at Xunit.Runner.VisualStudio.VsTestRunner.GetRunnerReporter(LoggerHelper, Xunit.Runner.VisualStudio.RunSettings, System.Collections.Generic.IEnumerable`1<System.String>)
>    at Xunit.Runner.VisualStudio.VsTestRunner.RunTests(Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IRunContext, Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IFrameworkHandle, LoggerHelper, Xunit.Runner.VisualStudio.TestPlatformContext, Xunit.Runner.VisualStudio.RunSettings, System.Func`1<System.Collections.Generic.List`1<Xunit.Runner.VisualStudio.AssemblyRunInfo>>)
>    at Xunit.Runner.VisualStudio.VsTestRunner.Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.ITestExecutor.RunTests(System.Collections.Generic.IEnumerable`1<System.String>, Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IRunContext, Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IFrameworkHandle)
>    at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.RunTestsWithSources.InvokeExecutor(Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.Utilities.LazyExtension`2<Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.ITestExecutor,Microsoft.VisualStudio.TestPlatform.Common.Interfaces.ITestExecutorCapabilities>, System.Tuple`2<System.Uri,System.String>, Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Adapter.RunContext, Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IFrameworkHandle)
>    at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestInternalWithExecutors(System.Collections.Generic.IEnumerable`1<System.Tuple`2<System.Uri,System.String>>, Int64)
>    at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestsInternal()
>    at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTests()
>    at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.ExecutionManager.StartTestRun(System.Collections.Generic.Dictionary`2<System.String,System.Collections.Generic.IEnumerable`1<System.String>>, System.String, System.String, Microsoft.VisualStudio.TestPlatform.ObjectModel.Engine.ClientProtocol.TestExecutionContext, Microsoft.VisualStudio.TestPlatform.ObjectModel.Engine.ITestCaseEventsHandler, Microsoft.VisualStudio.TestPlatform.ObjectModel.Client.ITestRunEventsHandler)
>    at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TestRequestHandler+<>c__DisplayClass30_3.<OnMessageReceived>b__3()
>    at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TestRequestHandler+<>c.<.ctor>b__18_1(System.Action)
>    at Microsoft.VisualStudio.TestPlatform.Utilities.JobQueue`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SafeProcessJob(System.__Canon)
>    at Microsoft.VisualStudio.TestPlatform.Utilities.JobQueue`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].BackgroundJobProcessor()
>    at Microsoft.VisualStudio.TestPlatform.Utilities.JobQueue`1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<.ctor>b__12_0()
>    at System.Threading.Tasks.Task.InnerInvoke()
>    at System.Threading.Tasks.Task+<>c.<.cctor>b__274_0(System.Object)
>    at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
>    at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
>    at System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread)
>    at System.Threading.Tasks.ThreadPoolTaskScheduler+<>c.<.cctor>b__10_0(System.Object)
>    at System.Threading.ThreadHelper.ThreadStart_Context(System.Object)
>    at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
>    at System.Threading.ThreadHelper.ThreadStart(System.Object)

Are there any other recommendations?

adaughterson on Mar 5, 2020

Okay, I now have a reproduction for the problem, and it’s thread related.

This is the error I get:

========== Starting test run ==========
NUnit Adapter 4.2.0.0: Test execution started
Running selected tests in C:\Users\itaib\source\repos\TestHostCrashTest\TestHostCrashTest\bin\Debug\net5.0\TestHostCrashTest.dll
   NUnit3TestExecutor discovered 1 of 1 NUnit test cases using Current Discovery mode, Non-Explicit run
The active test run was aborted. Reason: Test host process crashed : Unhandled exception. System.Exception: some value
   at TestHostCrashTest.CrashTest.<>c.<CrashTestHost>b__0_0() in C:\Users\itaib\source\repos\TestHostCrashTest\TestHostCrashTest\CrashTest.cs:line 14
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

========== Test run aborted: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========

and this is the code:

using NUnit.Framework;
using System.Threading;

namespace TestHostCrashTest
{
    public class CrashTest
    {
        [Test]
        public void CrashTestHost()
        {
            var t = new Thread(() =>
                throw new Exception("some value")
            );

            t.IsBackground = true;
            t.Start();
            t.Join();
        }
    }
}

itaibh on Jan 6, 2022

I had this issue. Here is how I resolved it:

I read the test logs (output | Tests tab).
I looked at the call-stack where the logs were saying that the process crashed.
It was an async void method. Obviously my code was causing the crash. Switched to async Task. Issue resolved.
But that was in my specific case. It should be something as simple as that in your specific case - in other words - your code is causing it.
If unable to resolve - post your Output | Tests somewhere and publish a link to it here.
We will help you figure it out. 😃

brah-mcdude on Feb 9, 2022

@juwens Yes, of course it breaks a PROGRAM, but we’re discussing a UNIT TEST

itaibh on Jan 18, 2022

@dbtfsbre you can install 5.0.101 dotnet SDK, and also install ProcDump so it is available in your PATH (if you open command line and type procdump, you should see output of the procdump tool). Then you do just:

dotnet test MyMsTestProject.dll --filter OneSpecificTest -v diag --blame-crash

And it should create a dump for you, open that in VS and you should see where that exception is coming from. Most likely it is your code calling some method recursively and forgetting to terminate.

nohwnd on Jan 14, 2021

I have got here due to use of wrong function. Assert.ThrowsException<MyException>(async () => await myFunction()) Fixed by replacing to async version: await Assert.ThrowsExceptionAsync<MyException>(async () => await myFunction())

plachta11b on Oct 30, 2020

Our builds have started failing with this error. Is there a way to get some decent logs out? We are using azure hosted pipelines.

We have been getting this error for months but it’s been very intermittent, so hasn’t bothered us. But over the past 2 days it has spiked hugely.

gregpakes on Jun 16, 2020

After the most recent Visual Studio 2022 update I’ve started seeing this nearly constantly in one of our solutions (with 421 tests using xUnit). Using .NET 6. I’ve gone over this entire thread as well as some other discussions I’ve found. None lead to fixes. When I do get it to run without this error, there’s only 1 failure (and that’s a not implemented exception for a test I’ve yet to write).

try dotnet test —-blame it will reveal the source of the issue. In our case it were unhandled exceptions in async void methods and more tricky async void lambdas.

That is literally what I wrote above as the cause of crash in my case (#2261 (comment)). @xuzhg - please search for “async void” in your code.

Holy moly, that was it. Thanks dude.

jakubkrzysztofsikora on Jan 19, 2023

After the most recent Visual Studio 2022 update I’ve started seeing this nearly constantly in one of our solutions (with 421 tests using xUnit). Using .NET 6. I’ve gone over this entire thread as well as some other discussions I’ve found. None lead to fixes. When I do get it to run without this error, there’s only 1 failure (and that’s a not implemented exception for a test I’ve yet to write).

try dotnet test —-blame it will reveal the source of the issue. In our case it were unhandled exceptions in async void methods and more tricky async void lambdas.

That is literally what I wrote above as the cause of crash in my case (https://github.com/microsoft/vstest/issues/2261#issuecomment-1034018644). @xuzhg - please search for “async void” in your code.

brah-mcdude on Oct 15, 2022

After the most recent Visual Studio 2022 update I’ve started seeing this nearly constantly in one of our solutions (with 421 tests using xUnit). Using .NET 6. I’ve gone over this entire thread as well as some other discussions I’ve found. None lead to fixes. When I do get it to run without this error, there’s only 1 failure (and that’s a not implemented exception for a test I’ve yet to write).

try dotnet test —-blame it will reveal the source of the issue. In our case it were unhandled exceptions in async void methods and more tricky: async void lambdas.

juwens on Oct 25, 2022

I’ve been getting this same issue, but it seemingly occurs only when running a whole test project at once; if I try to test each individual test methods in their own session/run then I never encounter the problem.

I also never get a stack trace or any meaningful debug output just before the Stack overflow occurs.

This is the most I’ve been able to extract out of the test/debug output:

========== Starting test run ==========
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.3+1b45f5407b (64-bit .NET Core 3.1.26)
[xUnit.net 00:00:02.50]   Starting:    McrApi.Tests.Integration
The active test run was aborted. Reason: Test host process crashed : Stack overflow.

Running/debugging the test project seems to make no difference. I’ve never been able to get VS to even break on the Stack overflow; this is just bizarre.

Using Xunit/.NET Core 3.1.

mamift on Jul 12, 2022

I just got a similar problem with my application in .NET 6

The active test run was aborted. Reason: Test host process crashed : Stack overflow.
Repeat 19163 times:
--------------------------------
   at Studip.NET.EntityHandlerBase`1[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].get_BaseUrl()
--------------------------------

Edit: I just removed some lines of code including attributes/reflection and now the test runs perfectly. I don’t know why that caused it and how to fix it. This is the line of code I removed:

typeof(TEntity)
        .GetCustomAttributes(typeof(SchemaNameAttribute), false)
        .OfType<SchemaNameAttribute>()
        .FirstOrDefault()?.Type
        ?? throw new InvalidProgramException($"Type '{typeof(TEntity).FullName}' has no '{nameof(SchemaNameAttribute)}' to be used in type handler base!");

cn-ml on Feb 10, 2022

@brah-mcdude - if the application/test code itself caused the crash, this should be much more obvious/different from the other scenarios presented “The active test run was aborted. Reason: Test host process crashed” <-- too generic.

mcallaghan-geotab on Feb 9, 2022

@juwens Then how do you explain it works on .net 4.6? It’s a different runtime… There’s a specific problem introduced in .net core and above.

itaibh on Jan 17, 2022

FWIW: the solution we found was actually due to OOM kills from kubernetes gitlab-runner setup. The nodes were stretching their RAM too tightly, and eventually digging showed events where k8s was killing the dotnet process.

Looked like this:

Memory cgroup out of memory: Killed process 3010546 (dotnet) total-vm:5186748kB, anon-rss:378116kB, file-rss:67156kB, shmem-rss:0kB, UID:0 pgtables:1336kB oom_score_adj:936

This might not be the problem or solution for everyone, but wanted to share back as it is definitely something to consider!

mcallaghan-geotab on Mar 6, 2021

@nohwnd Thanks. Something went wrong when copying files to the test machine for an investigation. The test is running fine in Visual Studio using Test Explorer and during our regular Release test runs. After compiling the test again directly on the test machine, I was able to run it with dotnet test as well as vstest.console.exe.

ghost on Jan 14, 2021

We had the exact same issue with our build. It turned out to be misconfigured settings for code coverage. In our CodeCoverage.runsettings-file we were not properly excluding test projects from coverage.

jesperoastrom on Oct 20, 2020

We were seeing similar issues in VS, there was a problem with cancellation of run that produced a cancellation error and was not caught in the correct place. It resulted in the same error, but probably not the same stack. That error is kind of a catch all for all errors when the other process disconnects in the middle of an operation so it is hard to tell. I would suggest upgrading to 16.5.0 to see if that was fixes.

nohwnd on Feb 13, 2020