nunit3-vs-adapter: Test Explorer is finding tests, but not running them after upgrading to NUnit3TestAdapter v4.3.0
NUnit: v3.13.3 NUnit3TestAdapter: v4.3.0 Visual Studio Professional 2019: v16.11.20 Target Framework Version: v4.5
I’m running Visual Studio Professional 2019 and have many NUnit tests in my project. With v4.2.1 of the NUnit3TesetAdapter installed, the Test Explorer is able to find and run all my tests as expected.
However, a few days ago I updated my NUnit3TestAdapter nuget package from 4.2.1 to 4.3.0 and now my Test Explorer does not run any of the tests. It throws no errors and has no problem finding/displaying the tests, but whenever I try to run them, it says:
Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in 172.1 ms
I can confirm that rolling back to 4.2.1, cleaning, and rebuilding the project fixes the problem so maybe the issue is a bug in the 4.3.0 package, however I wanted to check if there is any configuration I need to do in Visual Studio to fix this.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 47 (26 by maintainers)
Released here https://www.nuget.org/packages/NUnit3TestAdapter/4.3.1
I have a similar issue with some interesting Test log output:
The test project is a multi-targeted C# project with net48, netcoreapp3.1 and net6.0. Executing the test for netcoreapp3.1 and net6.0 works fine.
@ojintoad Thanks, really appreciate the feedback!
@OsirisTerje great work! i installed it, deleted the .vs/bin/obj folders, did a fresh rebuild and the tests are running right off the bat which is very promising. Internal.Trace.###.log files are no longer appearing in the root folder, too which is great.
I’ll keep poking around and seeing if I can get the original behavior to repro but so far it’s looking good
In addition, I suggest setting the engine
WorkDirectoryproperty to some safe value immediately upon creating the instance. Otherwise, you would be relying on the engine never trying to write any trace entries until you create a runner, which is not actually the case. In fact, there are other calls you could make to the engine (which you don’t now use) that could result in a log entry being written. I’d also turn off the tracing at that point as well as a sort of “belt and suspenders” approach.I’m having the same issue as @Guitrum describes, with a fairly similar setup.
NUnit: v3.13.3 NUnit3TestAdapter: v4.3.0 Visual Studio Professional 2022: v17.3.6 Target Framework Version: v4.7.2
I managed to trigger this on a very simple repro solution - NUnitVSAdapterRepro.zip - but after closing and opening Visual Studio it then started working again as normal.
Given the log that @smkanadl and I both got from the Tests Output window that they posted in https://github.com/nunit/nunit3-vs-adapter/issues/1027#issuecomment-1299991747 , I figured that it was to do with the current working directory of VS itself, so I made sure to launch it the same way (the default start menu shortcut for VS2022 opens the IDE in the
C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\IDE\directory which is consistent with the logs referenced above). However, despite closing and reopening it maybe 20-30 times, I managed to get it to fail once more only, with every other try working successfully.Recreating the same repro did not reproduce the issue either, nor can I now reproduce the issue on the work project that I hit the issue on originally, so I can only determine that the issue is nondeterministic, and that the majority of the time VS decides to change its working directory to the solution directory (evidenced by the output of the snippet of code provided by @OsirisTerje in https://github.com/nunit/nunit3-vs-adapter/issues/1027#issuecomment-1300084065 ), but sometimes it leaves it unset and then test discovery fails.
All that being said, I’m still unsure why test discovery fails even when the CWD is not changed to the solution directory - the code in https://github.com/nunit/nunit3-vs-adapter/blob/V4.3.0/src/NUnitTestAdapter/NUnit3TestDiscoverer.cs#L137 suggests that this should be being picked up - the
ProgramW6432environment variable is set toC:\Program Files, so the check at https://github.com/nunit/nunit3-vs-adapter/blob/V4.3.0/src/NUnitTestAdapter/NUnit3TestDiscoverer.cs#L176 should catch this and set the CWD to a temporary directory inside the NUnit adapter code. I didn’t see any way of debugging the test discovery from within Visual Studio either. I’d be tempted to add more logging there to try and catch this issue myself.I hope this helps, though I doubt it’ll bring you any joy that the issue seems nondeterministic. 😦
@OsirisTerje I was working on stripping my solution down to as basic as I could get it to try to repro the issue and somehow got the tests to run with 4.3.0! I went back to my original solution, cleaned, and rebuilt and now it’s working there as well!
EDIT: I spoke too soon. Closing and reopening visual studio broke it again… 😦
The attribute is embedded by convention from the
Directory.Build.props. But you are right. I tested it on a different machine and it all ran the net48 test case. And also after a fresh checkout on my machine it ran fine again. This is giving me a severe headache!@smkanadl Your experience equals mine. I can’t reproduce the write-protected directory. That’s why I am asking everyone for a repro 😃
The other issue with the empty logfiles is known (https://github.com/nunit/nunit-console/issues/1178) and will be included in NUnit.Engine 3.16, ETA sometime this November. It might be possible to get out an adapter using a dev version before that though, I would just like to get some more information on this directory thing.
Just noticed you had uploaded a repro, thanks!