azure-pipelines-tasks: VSTest task fails to execute tests in .NET Core 2.0 test project

Today I’ve found that coverage is now displayed correctly for .net core project

This is what I had to do to make it work

any plan to support .NETCore 2.0 unit tests?

Because if I run vstest.console /? I get these info below:

--Framework|/Framework:<Framework Version> Target .Net Framework version to be used for test execution. Valid values are ".NETFramework,Version=v4.5.1", ".NETCoreApp,Version=v1.0" etc. Other supported values are Framework35, Framework40, Framework45 and FrameworkCore10.

And it says it only supports FrameworkCore10. Will there be FrameworkCore20 or related TFM support for .NET Core 2.0 unit tests available build-in the vstest.console.exe?

cc @PBoraMSFT

@PBoraMSFT it is woking in VS Enterprise locally, and documentation mentions it should work from console https://github.com/Microsoft/vstest-docs/blob/master/docs/analyze.md#working-with-code-coverage

As you can see in logs posted above I was able to use TPv2 by specifying correct path and have coverage report created, but failed to upload because task is looking for it in different path.

And there is clearly no coverage support for dotnet test now https://github.com/Microsoft/vstest/issues/981

Can you please update VsTest task to use TPv2 and support .net core coverage. This is a real show stopper for us moving to .net core.

/Framework:“.NETCoreApp,Version=v2.0” fixed the issue for me with .NETCore 2.0

@eriawan for .NETCore 2.0 I use /Framework:“.NETCoreApp,Version=v2.0” which is picking up the tests and running them locally but having issues on VSTS hosted agent.

Huge props to @IvanAlekseev for the work-around. We’ve got code coverage up and running using these settings.

Here’s the exact strings required (for easy copy/paste-ability) for anyone that needs them:

Path to vstest.console.exe: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform

Other console options: /Framework:FrameworkCore10 /logger:console;verbosity="normal"

@tiagorosendo it is working fine with xunit. you may want to set the test assemblies to only the assemblies of your test project, then the testadapter.dll will be excluded.

@IvanAlekseev i’ve replicated your settings and still have this error, have any idea what that might be?

Params: 2017-10-06T06:06:32.7478934Z Test selector : Test assemblies 2017-10-06T06:06:32.7478934Z Test assemblies : *test.dll,!*\obj** 2017-10-06T06:06:32.7478934Z Test filter criteria : null 2017-10-06T06:06:32.7478934Z Search folder : d:\a\1\s 2017-10-06T06:06:32.7478934Z Run settings file : d:\a\1\s\Test.runsettings 2017-10-06T06:06:32.7478934Z Run in parallel : false 2017-10-06T06:06:32.7478934Z Run in isolation : false 2017-10-06T06:06:32.7478934Z Path to custom adapters : null 2017-10-06T06:06:32.7478934Z Other console options : /Framework:FrameworkCore10 2017-10-06T06:06:32.7478934Z Code coverage enabled : true 2017-10-06T06:06:32.7478934Z vstest.console.exe, specified location : C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\TestPlatform\

Error: 2017-10-06T06:06:46.7345547Z No test is available in d:\a\1\s\UnitTests\bin\Debug\netcoreapp2.0\xunit.runner.visualstudio.dotnetcore.testadapter.dll. Make sure test project has a nuget reference of package “Microsoft.NET.Test.Sdk” and framework version settings are appropriate and try again.

We will update on this thread.

@acesiddhu Regarding your answer to @joshbouganim “…Having said that rest of the things remains as is, like showing line coverage data, ability to download coverage file.”

We are experiencing the same issue as @joshbouganim and i’m a bit confused about the “things remain the same” part. Is there something that should be done so the coverage results would be shown on the coverage tab? Currently there’s just the option to download .coverage file.

Same here! I was expecting some awesome graphs =(

@acesiddhu Regarding your answer to @joshbouganim “…Having said that rest of the things remains as is, like showing line coverage data, ability to download coverage file.”

We are experiencing the same issue as @joshbouganim and i’m a bit confused about the “things remain the same” part. Is there something that should be done so the coverage results would be shown on the coverage tab? Currently there’s just the option to download .coverage file.

@joshbouganim the old UI used to show Block level data as well. we have removed that because we feel user generally needs and interpret Line level data. Dealing with Block is what we think is hard (specially for large code base). Having said that rest of the things remains as is, like showing line coverage data, ability to download coverage file.

@Antebios happy to help but following your instructions, this is all i see:

and just for reference, this is my tests tab:

and code coverage tab:

100% of my code is definitely not covered. i’m looking for a more detailed view like in @IvanAlekseev’s screenshot that shows blocks and lines covered.

I actually get VSTS to run tests (.net core 2.0) but it fails with following error:

2018-06-01T11:06:36.1777961Z Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Failed to initialize client proxy: could not connect to test process. 2018-06-01T11:06:36.1779658Z at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, CancellationToken cancellationToken) 2018-06-01T11:06:36.1780886Z at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)

Any ideas what is causing this?

@IvanAlekseev i tried your solution and have the same problem as @tiagorosendo . We are also using xUnit and we need .net Core 2.0 (which currently not supported). Thanks for the hint though

@chunliu thx, its works! but sonar qube throw one exception when i turn on code coverage =/ i will wait the sonarqube team fix this issue.

@PBoraMSFT thanks for you reply. We are currently using hosted agents, so we wait for them to update. Is there any timeline for it and is there any specific place where you notify about updates?

@IvanAlekseev - The VSTest task uses the test platform version 1. .NET core tests require the test platform version 2, which at this time does not support all the scenarios that are provided by Test platform version 1. The platform team is working on the convergence and once that is done, the task will also be able to consume the converged platform.

If using the vstest.console.exe of the Test platform v2 works for your scenarios, you may use it - but please note that unless the platform convergence happens, our ability to address any issues that may arise here will be limited.

As for this specific issue related to code coverage, we are addressing it. As Nitin indicated earlier, this requires a fix on the agent and will require you to update the agent as the fix becomes available.

@IvanAlekseev, @sandersaares - VSTest task cannot run .NET core tests as it uses the Test platform version 1. To run .NET core tests, we recommend using the .NET core task with the test command.

@IvanAlekseev - Code coverage and other data collection is not supported in test platform v2 yet.