fixie: Rider VSTest plugin *should* find Fixie tests, but doesn't

Some nice progress over here: https://github.com/fixie/fixie/pull/258

I’ve got things working with an alpha build in ReSharper, though Rider’s not reacting to it yet.

One possible difference between this and your successful proofs of concept above is that in the case of my alpha build, the Fixie.TestAdapter package itself is the one bringing in the important Microsoft.NET.Test.Sdk package, and I’m bringing in version 16.10.0 which is pretty recent.

Edit: Worked for me the next day. Maybe I just needed a restart after enabling the Test Adapter checkbox.

So It does work, you folks may be missing a reference to Microsoft.NET.Test.Sdk, and also be sure to enable VSTest adapters support in the settings of Rider.

Here are the results of the tests after I run them.

Much appreciated!

Leaving a comment here letting you know I’ve read the issue. I’ll try to find folks inside JetBrains that can give answers and possibly help.

Our understanding is the JetBrains presentation of the feature here is misleading, that all one would need is a TestAdapter.

As background: any integration with VS Test Explorer comes in the form of of a “TestAdapter” dll (which we do provide), and “dotnet test” works by invoking that same integration. Think of “dotnet test” as a headless invocation of the Test Explorer experience. So, that’s two votes in favor of “Fixie implemented its TestAdapter correctly”. We also include a URI like those in your screenshot.

In the early days of .NET Core, there was a blog post and some documentation from JetBrains suggesting that anything that integrates with dotnet test would Just Work here too, but that was right before Microsoft made a very large pivot in the way dotnet test would work.

More recent documentation hints at an extra hurdle to cross: https://www.jetbrains.com/help/resharper/sdk/Features/UnitTest/DotNetCore.html describes the idea that ReSharper (and I presume Rider) can lean on “dotnet test” as originally promised, but (emphasis added):

Map dotnet test IDs to ReSharper elements. When a test is discovered, it is reported with an ID that is relevant only to the test runner from the NuGet package. A ReSharper test provider must map between this ID and a ReSharper unit test element.

The test ID in question is probably the method-full-name string we are providing to VSTest, which helps Test Explorer show the tree of tests. It’s not so much an arbitrary ID in fact as it is just a method name, so it’s a little surprising that the docs here describe it as being opaque.

That makes it sound like:

1. Our VSTest plugin is important, but not enough.
2. We still have to build a whole JetBrains-style plugin ourselves.
3. That plugin’s job is to take one string per discovered test and map that to JetBrains’ understanding of type/method.

At least that was my understanding before your screen shot. Your screen shot and the listing of found-URIs makes me wonder why we’re not even showing up in that list. Are we missing some small thing that would make us finally visible? Or do we have to make a whole plugin for one string.Split()-and-map.

I’d like to hear from someone in the know about this “Enable VSTest adapters support” option for the real answer. @khalidabuhakmeh we were talking about this recently on twitter. Is there someone very close to the Rider + TestAdapter detection who could shed some light on the shortest path to success here? If Rider is meant to just work with any TestAdapter, I’d prefer the shorter path to getting that going than to make a whole plugin.