vstest: unable to discover netcoreapp1.1 tests.

Moving https://github.com/dotnet/cli/issues/5548#issuecomment-277313556 on behalf of @niemyjski

I am trying to upgrade my project from project.json to the new csproj format and hitting roadblocks left and right… (https://github.com/exceptionless/foundatio/tree/feature/dotnet)

It appears that VS2017 RC3 (Latest) is unable to discover or run any tests that have either netcoreapp1.0 or netcoreapp1.1.

When I try to run it from the command line I get

Test run for C:\Code\Foundatio\test\Foundatio.Tests\bin\Debug\netcoreapp1.1\Foundatio.Tests.dll(.NETCoreApp,Version=v1.1)
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Warning: No test is available in C:\Code\Foundatio\test\Foundatio.Tests\bin\Debug\netcoreapp1.1\Foundatio.Tests.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.

Even more when I try to run with diagnostics

dotnet test .\test\Foundatio.Tests\Foundatio.Tests.csproj -v
Index was outside the bounds of the array.

but

dotnet test .\test\Foundatio.Tests\Foundatio.Tests.csproj --no-build /v
:diag

works but doesn’t give me any diagnostics on why it couldn’t connect to test host / run tests.

It seems to be failing to discover net standard 1.0 tests as well 😦

I looked at some of your csproj’s and versions + refs look ok. Do you perhaps have an old CLI version?

Regardless, @dotnet/vstest needs to take a look to help us see how we can shore up the UX.

I shouldn’t as I am inside of a fresh vm with a freshly installed VS2017RC3 .03

If you pull down the repo and switch to the dotnet branch, you may need to add back the netcoreapp1.0 to one or more test projects ( I need to get this working on a ci ASAP)

fyi i checked out the feature/dotnet branch today and added back netcoreapp1.0 to Foundatio.Tests.csproj and just changed the xunit.assert package reference to xunit and then it worked… it then also worked for netcoreapp1.1 (only tested in command line, not VS)

Thanks for this, shouldn’t xunit be picked up as a transitive package?

I couldn’t find any noticeable difference in the built output except for the xunit metapackage appearing in deps.json. Even tough xunit appears to just be a meta-package, it seems to be required in order for test discovery to work. Since you split xunit.core and xunit.assert packages across the logger project (p2p referenced) and the test project itself, the resulting dependencies are identical to the following test project which also doesn’t discover tests:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp1.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0-preview-20170123-02" />
    <PackageReference Include="xunit.assert" Version="2.2.0-beta5-build3474" />
    <PackageReference Include="xunit.core" Version="2.2.0-beta5-build3474" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.2.0-beta5-build1225" />
  </ItemGroup>

</Project>

I’m unsure if this is expected or an issue with either vstest or the xunit test runner.

Yeah I added xunit and now it works so some kind of issue here

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 29 (15 by maintainers)

Commits related to this issue

Added sdk package to all test projects to make the tests discoverable... the sdk ref isn't transitive... https://github.com/Microsoft/vstest/issues/446#issuecomment-280414567 — committed to FoundatioFx/Foundatio by niemyjski 7 years ago

Most upvoted comments

@dasMulli Update for tests failing on MacOS, looks like issue related Swashbuckle.AspNetCore 1.0.0-rc1 reference. Removing same reference makes tests passed. Raised issue https://github.com/Microsoft/vstest/issues/516.

smadala on Feb 17, 2017

@dasMulli Able to repo tests not discovered for provided project. Looks like testhost failing with error

'/Users/vstest/Downloads/testrepro/tests/bin/Debug/netcoreapp1.1/Microsoft.DiaSymReader.Native.amd64.dll'. A device attached to the system is not functioning.
 (Exception from HRESULT: 0x8007001F)
File name: '/Users/vstest/Downloads/testrepro/tests/bin/Debug/netcoreapp1.1/Microsoft.DiaSymReader.Native.amd64.dll' ---> System.Runtime.InteropServices.COMException (0x8007001F): A device attached to the system is not functioning.
 (Exception from HRESULT: 0x8007001F)
   at System.Runtime.Loader.AssemblyLoadContext.nGetFileInformation(String s)
   at Microsoft.VisualStudio.TestPlatform.Common.Utilities.AssemblyResolver.OnResolve(AssemblyLoadContext loadContext, AssemblyName args) 
TpTrace Information: 0 : 2596, 3, 2017/02/17, 00:37:19.419, 2192148601555, testhost.dll, AssemblyResolver: Microsoft.DiaSymReader.Native.x86: Resolving assembly.
TpTrace Information: 0 : 2596, 3, 2017/02/17, 00:37:19.420, 2192149147009, testhost.dll, AssemblyResolver: Microsoft.DiaSymReader.Native.x86: Failed to load assembly. Reason:System.IO.FileLoadException: Could not load file or assembly '/Users/vstest/Downloads/testrepro/tests/bin/Debug/netcoreapp1.1/Microsoft.DiaSymReader.Native.x86.dll'. A device attached to the system is not functioning.
 (Exception from HRESULT: 0x8007001F)
File name: '/Users/vstest/Downloads/testrepro/tests/bin/Debug/netcoreapp1.1/Microsoft.DiaSymReader.Native.x86.dll' ---> System.Runtime.InteropServices.COMException (0x8007001F): A device attached to the system is not functioning.
 (Exception from HRESULT: 0x8007001F)
   at System.Runtime.Loader.AssemblyLoadContext.nGetFileInformation(String s)
   at Microsoft.VisualStudio.TestPlatform.Common.Utilities.AssemblyResolver.OnResolve(AssemblyLoadContext loadContext, AssemblyName args) 
TpTrace Information: 0 : 2596, 3, 2017/02/17, 00:37:19.420, 2192149428634, testhost.dll, AssemblyResolver: xunit.runner.reporters.netstandard11: Resolving assembly.
TpTrace Information: 0 : 2596, 3, 2017/02/17, 00:37:19.420, 2192149943982, testhost.dll, AssemblyResolver: xunit.runner.reporters.netstandard11: Failed to load assembly. Reason:System.IO.FileLoadException: Could not load file or assembly '/Users/vstest/Downloads/testrepro/tests/bin/Debug/netcoreapp1.1/xunit.runner.reporters.netstandard11.dll'. A device attached to the system is not functioning.
 (Exception from HRESULT: 0x8007001F)
File name: '/Users/vstest/Downloads/testrepro/tests/bin/Debug/netcoreapp1.1/xunit.runner.reporters.netstandard11.dll' ---> System.Runtime.InteropServices.COMException (0x8007001F): A device attached to the system is not functioning.
 (Exception from HRESULT: 0x8007001F)
   at System.Runtime.Loader.AssemblyLoadContext.nGetFileInformation(String s)
   at Microsoft.VisualStudio.TestPlatform.Common.Utilities.AssemblyResolver.OnResolve(AssemblyLoadContext loadContext, AssemblyName args)

log.txt log.host.17-02-17_00-37-18_20929_5.txt

Will try to find root cause.

smadala on Feb 16, 2017

We are already using those package versions in Foundatio @niemyjski and the test runner is still very flakey for us.

ejsmith on Feb 15, 2017

@jimmystridh xunit < 2.2.0-beta5 not compatible with netcoreapp1.1, and Microsoft.NET.Test.Sdk < 15.0.0-preview-20170210-02 doesn’t provide error message from xunit runner to end user.

Please update PackageReferences to following:

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0-preview-20170210-02" />
    <PackageReference Include="xunit" Version="2.2.0-rc2-build3523" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.2.0-rc2-build1249" />
  </ItemGroup>

smadala on Feb 15, 2017