vstest: The provided manager was not found in any slot.

The currently released version of VS 17.6.3 does not have the fix yet unfortunately, it will be in 17.6.4 that should ship in few days I hope.

A (Re-)Discover button would help. Or is there a way to trigger the discovery manually?

In Visual Studio Log/Output Window you can switch to the “Tests” Tab. You can see that it is always running the test discovery when you build a project. It even does the test discovery for all projects, not only the one that you build. So, triggering the test-discovery can be simply done by building (even if the build does nothing because all is build and just finishes right away)

That also means that we are now spammed with these errors and exceptions (mentioned above) in the background on each build…

Try adding this item group to your project.

  <ItemGroup>
    <ProjectCapability Remove="TestContainer" />
  </ItemGroup>

Uuuf. 😃 I saw just the first part of your message in preview on my phone, and was hoping that this is not another " Can confirm, this is broken for me as well"…

The worst thing is, that it ends with “Test discovery aborted” instead of continuing with the other .dlls or other assemblies i.e. you have randomly missing tests even though maybe only one .dll has an issue

@PeterHevesi 17.6.1 was released today with the fix for the other problem, which can under some circumstances fix this problem as well. This problem is only happening after crash of testhost (based on the provided repro), the testhost crash needs to be solved, as this is just a by-product of that crash.

Why specifically it crashes in your case with 17.6.0 but not 17.5.0 I don’t know, because I don’t have a repro from you unfortunately.

@PeterHevesi I added a reference to Microsoft.NET.Test.SDK.

I’ve also finally found the underlying issue with not finding the manager in slot.

    private void StartTestRunOnConcurrentManager(
        IProxyExecutionManager proxyExecutionManager,
        IInternalTestRunEventsHandler eventHandler,
        TestRunCriteria testRunCriteria,
        bool initialized,
        Task? initTask)
    {
        if (testRunCriteria != null)
        {
            Task.Run(() =>
                {
                    if (!initialized)
                    {
                        if (!proxyExecutionManager.IsInitialized)
                        {
                            proxyExecutionManager.Initialize(_skipDefaultAdapters);
                        }

                        Interlocked.Increment(ref _runStartedClients);
                        proxyExecutionManager.InitializeTestRun(testRunCriteria, eventHandler);
                    }
                    else
                    {
                        initTask!.Wait();
                    }

                    EqtTrace.Verbose("ParallelProxyExecutionManager: Execution started. Started clients: " + _runStartedClients);

                    proxyExecutionManager.StartTestRun(testRunCriteria, eventHandler);
                })

This code kicks off work on each testhost that we start in parallel (which we also use when we pre-start testhost).

proxyExecutionManager.InitializeTestRun(testRunCriteria, eventHandler); attempts to start testhost process, and fails. Of course it does not throw an exception but instead it emits RunCompletedEvent, which is handled by the rest of the code and propagates back to the parallel handler, which grabs the slot that is holding the failed manager and clears it.

The method returns void, and because it does not throw we dont’t know that the initialization failed, and continue executing. We again attempt to start testhost, because proxyExecutionManager.StartTestRun(testRunCriteria, eventHandler); again checks if testhost is initialized and if not it tries to start it up. We fail again, and again trigger the error handler that tries to clean up the slot. This time it won’t find it of course because the slot is already cleared and possible re-purposed for a different manager.

On thing that threw me off was that the stacks look very similar in both places, so I did not notice that in one case we go directly into Initialize and in another we go into Start and then Initialize, even thought I was specifically thinking about that possibility, but did not compare the stacks in a diff tool unfortunately. That could have saved me half a day.

If you ask why it worked in the past, that is because previously we did not use those slots and just ignored any double event notifications because the “testhost is dead anyway”.

In any case this is not a serious issue, as it can be reached only when the testhost already failed to start up.

@nohwnd will give a more detailed explanation but the fix is already merged (see https://github.com/microsoft/vstest/pull/4474) and we are working on rolling it out to NuGet, VS and .NET.

Okay, thanks, looks like it is in fact related to https://github.com/microsoft/vstest/issues/4467 and a workaround for you would be to provide dll filter that is specific to your test project naming pattern. It seems that most your projects are named Tests..dll something, that should restrict your pattern better than using test.dll:

So I would try replacing this line, with bin/**/*Tests*.dll, and keeping the rest of the exclude rules in place for now to see if it helped.

https://github.com/unoplatform/uno/blob/master/build/ci/.azure-devops-unit-tests.yml#L77