nunit3-vs-adapter: Test Discovery slow for large TestCaseSource and TestCaseAttribute

I am currently using version 3.15.1 of the test adapter in VS 2019 Community edition (16.2.5). .NET Standard 4.5.2 for the code.

I have a solution with >80,000 tests. 2 test projects have about ~40,000 each. There are many tests that have several hundred test cases (either from TestCaseAttribute or TestCaseSource and a class building the test cases). When I go to run my tests, the actual test run is only a few minutes - but the discovery phase takes up to 20 minutes to find/build all test cases. While I understand that constructing large TestCaseSource and parsing hundreds of TestCaseAttributes will take some time, but 20 minutes seems excessive.

[9/23/2019 9:13:57.068 AM Informational] NUnit Adapter 3.15.1.0: Test execution started
[9/23/2019 9:13:57.108 AM Informational] Running selected tests in C:\Users\cidth\Documents\GitHub\CreatureGen\CreatureGen.Tests.Integration.Stress\bin\Debug\CreatureGen.Tests.Integration.Stress.dll
[9/23/2019 9:13:57.308 AM Informational] NUnit Adapter 3.15.1.0: Test execution started
[9/23/2019 9:13:57.351 AM Informational] Running selected tests in C:\Users\cidth\Documents\GitHub\CreatureGen\CreatureGen.Tests.Integration.IoC\bin\Debug\CreatureGen.Tests.Integration.IoC.dll
[9/23/2019 9:13:57.528 AM Informational]    NUnit3TestExecutor converted 3 of 3 NUnit test cases
[9/23/2019 9:13:57.863 AM Informational]    NUnit3TestExecutor converted 26 of 26 NUnit test cases
[9/23/2019 9:13:58.347 AM Informational] NUnit Adapter 3.15.1.0: Test execution started
[9/23/2019 9:13:58.404 AM Informational] Running selected tests in C:\Users\cidth\Documents\GitHub\CreatureGen\CreatureGen.Tests.Integration\bin\Debug\CreatureGen.Tests.Integration.dll
[9/23/2019 9:13:58.592 AM Informational] NUnit Adapter 3.15.1.0: Test execution complete
[9/23/2019 9:13:59.374 AM Informational]    NUnit3TestExecutor converted 757 of 757 NUnit test cases
[9/23/2019 9:13:59.589 AM Informational] NUnit Adapter 3.15.1.0: Test execution complete
[9/23/2019 9:14:02.972 AM Informational] NUnit Adapter 3.15.1.0: Test execution started
[9/23/2019 9:14:03.208 AM Informational] Running selected tests in C:\Users\cidth\Documents\GitHub\CreatureGen\CreatureGen.Tests.Unit\bin\Debug\CreatureGen.Tests.Unit.dll
[9/23/2019 9:14:03.758 AM Informational] NUnit Adapter 3.15.1.0: Test execution started
[9/23/2019 9:14:04.043 AM Informational] Running selected tests in C:\Users\cidth\Documents\GitHub\CreatureGen\CreatureGen.Tests.Integration.Tables\bin\Debug\CreatureGen.Tests.Integration.Tables.dll
[9/23/2019 9:14:11.565 AM Informational] NUnit Adapter 3.15.1.0: Test execution complete
[9/23/2019 9:20:06.993 AM Informational]    NUnit3TestExecutor converted 34144 of 34144 NUnit test cases
[9/23/2019 9:25:08.348 AM Informational]    NUnit3TestExecutor converted 45509 of 45509 NUnit test cases
[9/23/2019 9:32:16.820 AM Informational] NUnit Adapter 3.15.1.0: Test execution complete

I did read over in #448 that the adapter scans for things in serial, and twice - which would certainly be a source of the slowdown. If I set the test runner to use parallel runs (as the task suggests), that does help with the speed, but it is still very slow to do a “Run All”.

The repo where this occurs can be found here. Simple check out the branch, open the solution, build, and click “Run All”. There will be test failures, but the time it takes for test discovery is the concern.

I can run the tests in the console runner, and the test discovery is much faster, which also matches up with #448 .