vstest: Since SDK 2.2.300 test performance is magnitudes slower

Description

Tests take a long time to run on detached hosts build servers (CircleCI)

Similar issue was closed, but the issue still happens. https://github.com/microsoft/vstest/issues/2078

Steps to reproduce

Repo which can reproduce the issue https://github.com/davidruhmann/vstest2080

Expected behavior

Tests to complete < 1s

Actual behavior

Tests take over > 20s

Each line of Test run in progress. output takes several seconds while the test takes < 2 ms.

Diagnostic logs

https://circleci.com/gh/davidruhmann/vstest2080/3

Environment

.NET Core SDK (reflecting any global.json):
 Version:   2.2.401
 Commit:    729b316c13

Runtime Environment:
 OS Name:     debian
 OS Version:  9
 OS Platform: Linux
 RID:         debian.9-x64
 Base Path:   /usr/share/dotnet/sdk/2.2.401/

Host (useful for support):
  Version: 2.2.6
  Commit:  7dac9b1b51

.NET Core SDKs installed:
  2.2.401 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 23 (13 by maintainers)

Commits related to this issue

for #2120 DisableProgress when environment CI true — committed to davidruhmann/vstest by davidruhmann 5 years ago
ci(hanging): adds --logger:Console;noprogress=true to dotnet test commands as per https://github.com/microsoft/vstest/issues/2120 — committed to specklesystems/speckle-sharp by didimitrie 4 years ago

Most upvoted comments

Can workaround the issue by specifying the following parameter to dotnet test:

'--logger:Console;noprogress=true'

martincostello on Aug 24, 2019

@davidruhmann Try separating the package restore and build from the test steps. you’re doing all three in a single step that takes 27 seconds; I usually do something like the following:

dotnet restore
dotnet build --no-restore
dotnet test --no-restore --no-build

If you do this in distinct CirlceCI steps it will give you a more accurate reflection of where the time is being taken.

JSkimming on Aug 9, 2019

I was thinking that the convention of setting CI environment variable may not be followed well, say CI=NameOfCIService though it’s an edge case.

JSkimming on Aug 9, 2019

Personally I lean towards having invalid boolean value ignored. CI can be 0|1|true|false/i

davidruhmann on Aug 9, 2019

@davidruhmann I think you should consider submitting a PR as I did 😃

JSkimming on Aug 9, 2019

However, I would not categorize that as a resolution.

Agreed, but it narrows down the potential causes, and I suspect the progress indicator is the culprit.

So you think, not showing a progress indicator might fix the issue?

Yup, but I don’t think it can be disabled using cli options.

JSkimming on Aug 9, 2019