nunit: Slow performance of nunit-console
I’ve noticed, that tests are being run in nunit-console performing much slower than running them in Visual Studio or in simple console application. nunit-console performs 4-5 times slower on some tests.
I’ve tested this with NUnit 2.6.4, NUnit 3.0.0-alpha4 (from nuget packages) and with NUnit compiled from master branch (with changes from 2.0 target framework to 4.0 target framework of nunit-console and other projects). I tried various settings of --domain and --process parameters, they didn’t help.
Here is the project which demonstrates the issue:
https://github.com/xplicit/Bond/tree/nunit
test UnitTest.GenericsTests.GenericInheritance takes 15-17 seconds on my machine if run it from Visual Studio tests or make a simple console application, which calls this method.
When I run it from nunit-console, it takes about 80 seconds.
Also I wrote a small app, which can be used to investigate the issue. When I run test from Visual Studio or console or with nunitlite it takes 22 seconds to run. If I run it using nunit-console it takes 36 seconds. However I can’t be sure for 100% that the app shows the same issue as the first one.
https://github.com/xplicit/nunitperf
The test case, I test: https://github.com/xplicit/nunitperf/blob/master/nunitPerf/Class1.cs
This is critical issue, because it does not allow to run unit-tests on build machines with limited build time (like appveyor service).
Also I am looking for a workaround how I can run unit tests on the build machine and get the results on time with current versions of nunit libraries. I tried to use nunitlite but seems that all tests must be located in the *.exe assembly, and nunitlite can’t load tests from other assemblies (no matter are they references or don’t referenced by nunitlited exe). If it were able to load assemblies I would have create the console app with references to current tests and run them from command line.
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Reactions: 1
- Comments: 42 (18 by maintainers)
Commits related to this issue
- Merge pull request #480 from nunit/jnm2/copy_pdb_to_output Copy adapter PDB to project build output — committed to johnmwright/nunit by OsirisTerje 6 years ago
I’ve got finally the same times for nunit-console and console application. It was successful only for 3a5, because for 3a4 the command without options does not work (it throws exception about .NET configuration System.ArgumentException: NUnit components for version 4.0.30319 of the CLR are not installed Имя параметра: targetRuntime в NUnit.Engine.Services.TestAgency.LaunchAgentProcess(RuntimeFramework targetRuntime, Boolean enableDebug, String agentArgs, Boolean useX86Agent))
So, this command shows the same time on my machine (18 sec):
nunit-console.exe nunitPerf.exeThese commands show other time (30 sec):nunit-console.exe --framework=net-4.0 nunitPerf.exenunit-sonsole.exe --process=Single nunitPerf.exe