stryker-net: Upgrading from 4.0.6 to 4.2.0 analysis no longer works for project

Describe the bug

Attempting to upgrade from 4.0.6 to 4.2.0, analysis is failing for one of our test projects, causing an error.

Specifying --dev-mode doesn’t appear to add any additional details.

4.0.6

Running mutation tests for '../src/Polly/Polly.csproj'. Test Project: 'Polly.Specs/Polly.Specs.csproj'

   _____ _              _               _   _ ______ _______  
  / ____| |            | |             | \ | |  ____|__   __| 
 | (___ | |_ _ __ _   _| | _____ _ __  |  \| | |__     | |    
  \___ \| __| '__| | | | |/ / _ \ '__| | . ` |  __|    | |    
  ____) | |_| |  | |_| |   <  __/ |    | |\  | |____   | |    
 |_____/ \__|_|   \__, |_|\_\___|_| (_)|_| \_|______|  |_|    
                   __/ |                                      
                  |___/                                       


Version: 4.0.6

A new version of Stryker.NET (4.2.0) is available. Please consider upgrading 
using `dotnet tool update -g dotnet-stryker`

[09:02:00 INF] Analysis starting.
[09:02:06 WRN] Could not find a project analysis for the chosen target framework net8.0. 
The available target frameworks are: net462,net472,net6.0,netstandard2.0. 
The first available framework will be selected, which is net462.
[09:02:06 INF] Found project D:\a\Polly\Polly\src\Polly\Polly.csproj to mutate.
[09:02:06 INF] Analysis complete.
[09:02:06 INF] Building test project D:\a\Polly\Polly\test\Polly.Specs\Polly.Specs.csproj (1/1)
[09:03:18 INF] Number of tests found: 1986 for project D:\a\Polly\Polly\src\Polly\Polly.csproj. Initial test run started.

4.2.0

Running mutation tests for '../src/Polly/Polly.csproj'. Test Project: 'Polly.Specs/Polly.Specs.csproj'

   _____ _              _               _   _ ______ _______  
  / ____| |            | |             | \ | |  ____|__   __| 
 | (___ | |_ _ __ _   _| | _____ _ __  |  \| | |__     | |    
  \___ \| __| '__| | | | |/ / _ \ '__| | . ` |  __|    | |    
  ____) | |_| |  | |_| |   <  __/ |    | |\  | |____   | |    
 |_____/ \__|_|   \__, |_|\_\___|_| (_)|_| \_|______|  |_|    
                   __/ |                                      
                  |___/                                       


Version: 4.2.0

[12:05:57 INF] Analysis starting.
[12:05:57 INF] Analyzing 1 test project(s).
[12:06:11 INF] Analysis complete.
[12:06:11 INF] Time Elapsed 00:00:13.4378512
Stryker.NET failed to mutate your project. For more information see the logs 
below:


No project references found. Please add a project reference to your test project
and retry.

Logs

See the mutation-report artifact here.

Expected behavior

Mutation tests work.

Desktop (please complete the following information):

  • OS: Windows
  • Type of project: framework and core
  • Framework Version: net462,net472,net6.0,netstandard2.0
  • Stryker Version: 4.2.0

Additional context

Failing upgrade: https://github.com/App-vNext/Polly/pull/2293

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 17 (17 by maintainers)

Most upvoted comments

Confirmed fixed with 4.3.0 - thanks.

I don’t think there is any feed available for that. No problem anyway

Disclaimer: I reproduce this issue using Stryker integration test projects. They normally target net 8.0, but I have modified the ‘tested project’ to target net7.0 instead (and using our XUnit test project as the starting point, to be specific). With this setup, I reproduce the issue. The fun part is that project analysis report a successful result when requesting for .net8.0 version of the project. Alas, the project is not actually built during a normal build, so the run fails (no assembly found).

I have been able to resolve this issue, by requesting .net8.0 only on the test project. And voila, it does work. I still have a warning about the being able to use .net8. I will look into this to decide if it is better to remove it or not.

Note that this fix works for single project mode, I sill have to look into solution mode, where it may be a bit trickier

The test project targets net8.0, but the library doesn’t because it doesn’t need to. We could add net8.0 but there’s no functional need for it

Thanks, that was the key missing information here. I should have though about this. This explains why I have a different result. And it looks like I do reproduce the issue as you experience it, so I should be able to fix it properly now.

Yes, Stryker should work seamlessly, as long as the test projects do support the selected target framework. The change introduced between V4.06 and V4.2 aimed at actually removing the random framework selection that happens for the tested project. To achieve that, Stryker picks the exact target project configuration resolved by the test project. It looks like we do something wrong here, probably by checking that selected configuration uses the configured target framework.

The problem you face when no longer picking a target framework is that Stryker may pick a net framework version (and not a core one), and then it needs the solution for building it.

I do have a problem when specifying a non supported target framewok, but it fails differently, and I have no idea why.

I am surprised that --def-mode does not provide extra information. Both hints toward an non anticipated failure mode. For the short time, I will fix the problem I reproduce.