nunit: Slow performance of nunit-console

I’ve noticed, that tests are being run in nunit-console performing much slower than running them in Visual Studio or in simple console application. nunit-console performs 4-5 times slower on some tests.
I’ve tested this with NUnit 2.6.4, NUnit 3.0.0-alpha4 (from nuget packages) and with NUnit compiled from master branch (with changes from 2.0 target framework to 4.0 target framework of nunit-console and other projects). I tried various settings of --domain and --process parameters, they didn’t help.

Here is the project which demonstrates the issue:

https://github.com/xplicit/Bond/tree/nunit

test UnitTest.GenericsTests.GenericInheritance takes 15-17 seconds on my machine if run it from Visual Studio tests or make a simple console application, which calls this method. When I run it from nunit-console, it takes about 80 seconds.

Also I wrote a small app, which can be used to investigate the issue. When I run test from Visual Studio or console or with nunitlite it takes 22 seconds to run. If I run it using nunit-console it takes 36 seconds. However I can’t be sure for 100% that the app shows the same issue as the first one.

https://github.com/xplicit/nunitperf

The test case, I test: https://github.com/xplicit/nunitperf/blob/master/nunitPerf/Class1.cs

This is critical issue, because it does not allow to run unit-tests on build machines with limited build time (like appveyor service).

Also I am looking for a workaround how I can run unit tests on the build machine and get the results on time with current versions of nunit libraries. I tried to use nunitlite but seems that all tests must be located in the *.exe assembly, and nunitlite can’t load tests from other assemblies (no matter are they references or don’t referenced by nunitlited exe). If it were able to load assemblies I would have create the console app with references to current tests and run them from command line.

About this issue

  • Original URL
  • State: closed
  • Created 9 years ago
  • Reactions: 1
  • Comments: 42 (18 by maintainers)

Commits related to this issue

Most upvoted comments

I’ve got finally the same times for nunit-console and console application. It was successful only for 3a5, because for 3a4 the command without options does not work (it throws exception about .NET configuration System.ArgumentException: NUnit components for version 4.0.30319 of the CLR are not installed Имя параметра: targetRuntime в NUnit.Engine.Services.TestAgency.LaunchAgentProcess(RuntimeFramework targetRuntime, Boolean enableDebug, String agentArgs, Boolean useX86Agent))

So, this command shows the same time on my machine (18 sec): nunit-console.exe nunitPerf.exe These commands show other time (30 sec): nunit-console.exe --framework=net-4.0 nunitPerf.exe nunit-sonsole.exe --process=Single nunitPerf.exe