stryker-net: Potential problem with coverage analysis on async tests

//First Scenario
[HttpPut("api/myendpoint01")]
public async Task<IActionResult> MyEndPoint01(string param)
{
 //Here the mutant is killed
 if(string.IsNullOrEmpty(param))
 {
   //Do something...
   return BadRequest();
 }
 //.......
}
//Second Scenario
[HttpPut("api/myendpoint02")]
public async Task<IActionResult> MyEndPoint02(string param)
{
 //Here the mutant is survived
 if(string.IsNullOrEmpty(param))
 {
   //Do something...
   return BadRequest();
 }

 //....... 
 
}

I’ve got the unit test for both endpoint covering the BadRequest result and Ok result.

My question is why in the first scenario the mutant is killed but in the second one it is survived?

Info:

Windows 10 Pro .NET Core 2.1.509 NUnit 3.12.0 Stryker 0.18.0

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 18 (9 by maintainers)

Most upvoted comments

Thanks @Mobrockers : I was about to say the same thing here. It sounds like a coverage issue. I lack experience with tests marked as ‘async’ but those could potentially result in invalid coverage data.

dupdob on Jul 24, 2020

Try it dotnet stryker -tr dotnettest

rodrigogurgel on Jul 23, 2020

Hi! Here is a procedure to help us identify where the problem is: Whenever you have an unexpected surviving mutant, you should:

Manually inject the mutant in your code and run your tests. In your case, I think it means replacing if (string.IsNullOrEmpty(...)) by if (!string.IsNullOrEmpty(...))
If the test run succeeds, it is indeed a surviving mutant and you need to add a new test or change an existing test that fails due to this mutation.
if the test run fails, it means StrykerMutator failed to see it as killed. Stryker should probably have run more tests against this mutation. Which means coverage analysis results where wrong. Please try again with coverage analysis turned off (see the Configuration.md file). This is going to be slow.
1. If the problem is fixed (i.e the mutant is killed), please perform another run with coverage analysis on and with debug log and provide the main log file (largest one) with the issue. It may be sufficient to fix the issue.
2. if the problem is not fixed, please open an issue, but we will probably need access to the project or to a project that can reproduce the issue.

Can you please try this and report you results please?

dupdob on Nov 2, 2021