stryker-net: Stryker failing with NullReferenceException

Describe the bug When running stryker in solution mode, mutation fails with a null reference exception. I ran stryker on another clone of stryker (master branch).

[10:49:37 ERR] Some mutations failed to be inserted, they are dropped.
[10:49:37 ERR] Some mutations failed to be inserted, they are dropped.
[10:49:37 ERR] Some mutations failed to be inserted, they are dropped.
[10:49:40 INF] Time Elapsed 00:01:44.7349597
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.CodeAnalysis.CSharp.SyntaxFactory.FindConditionalAccessNodeForBinding(CSharpSyntaxNode node)
   at Microsoft.CodeAnalysis.CSharp.Binder.GetReceiverForConditionalBinding(ExpressionSyntax binding, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindMemberBindingExpression(MemberBindingExpressionSyntax node, Boolean invoked, Boolean indexed, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindValue(ExpressionSyntax node, BindingDiagnosticBag diagnostics, BindValueKind valueKind)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindValueConditionalOperator(ConditionalExpressionSyntax node, ExpressionSyntax whenTrue, ExpressionSyntax whenFalse, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindConditionalOperator(ConditionalExpressionSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindParenthesizedExpression(ExpressionSyntax innerExpression, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindValue(ExpressionSyntax node, BindingDiagnosticBag diagnostics, BindValueKind valueKind)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindConditionalAccessExpression(ConditionalAccessExpressionSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindSimpleBinaryOperator(BinaryExpressionSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindValue(ExpressionSyntax node, BindingDiagnosticBag diagnostics, BindValueKind valueKind)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindValueConditionalOperator(ConditionalExpressionSyntax node, ExpressionSyntax whenTrue, ExpressionSyntax whenFalse, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindConditionalOperator(ConditionalExpressionSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindParenthesizedExpression(ExpressionSyntax innerExpression, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics, Boolean invoked, Boolean indexed)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindValue(ExpressionSyntax node, BindingDiagnosticBag diagnostics, BindValueKind valueKind)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindBooleanExpression(ExpressionSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindIfStatement(IfStatementSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindStatement(StatementSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindBlockParts(BlockSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindBlock(BlockSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindStatement(StatementSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindPossibleEmbeddedStatement(StatementSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindIfStatement(IfStatementSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindStatement(StatementSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindBlockParts(BlockSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindBlock(BlockSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindStatement(StatementSyntax node, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindMethodBody(CSharpSyntaxNode declaration, BlockSyntax blockBody, ArrowExpressionClauseSyntax expressionBody, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.Binder.BindMethodBody(CSharpSyntaxNode syntax, BindingDiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.MethodCompiler.BindMethodBody(MethodSymbol method, TypeCompilationState compilationState, BindingDiagnosticBag diagnostics, Boolean includeInitializersInBody, BoundNode initializersBody, Boolean reportNullableDiagnostics, ImportChain& importChain, Boolean& originalBodyNested, Boolean& prependedDefaultValueTypeConstructorInitializer, InitialState& forSemanticModel)
   at Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileMethod(MethodSymbol methodSymbol, Int32 methodOrdinal, ProcessedFieldInitializers& processedInitializers, SynthesizedSubmissionFields previousSubmissionFields, TypeCompilationState compilationState)
   at Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileNamedType(NamedTypeSymbol containingType)
   at Microsoft.CodeAnalysis.CSharp.MethodCompiler.<>c__DisplayClass25_0.<CompileNamedTypeAsync>b__0()
   at Roslyn.Utilities.UICultureUtilities.<>c__DisplayClass5_0.<WithCurrentUICulture>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Microsoft.CodeAnalysis.CSharp.MethodCompiler.WaitForWorkers()
   at Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileMethodBodies(CSharpCompilation compilation, PEModuleBuilder moduleBeingBuiltOpt, Boolean emittingPdb, Boolean hasDeclarationErrors, Boolean emitMethodBodies, BindingDiagnosticBag diagnostics, Predicate`1 filterOpt, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.CompileMethods(CommonPEModuleBuilder moduleBuilder, Boolean emittingPdb, DiagnosticBag diagnostics, Predicate`1 filterOpt, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.Emit(Stream peStream, Stream metadataPEStream, Stream pdbStream, Stream xmlDocumentationStream, Stream win32Resources, IEnumerable`1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, Stream sourceLinkStream, IEnumerable`1 embeddedTexts, RebuildData rebuildData, CompilationTestData testData, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.Emit(Stream peStream, Stream pdbStream, Stream xmlDocumentationStream, Stream win32Resources, IEnumerable`1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, Stream sourceLinkStream, IEnumerable`1 embeddedTexts, Stream metadataPEStream, RebuildData rebuildData, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.Emit(Stream peStream, Stream pdbStream, Stream xmlDocumentationStream, Stream win32Resources, IEnumerable`1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, Stream sourceLinkStream, IEnumerable`1 embeddedTexts, Stream metadataPEStream, CancellationToken cancellationToken)
   at Stryker.Core.Compiling.CsharpCompilingProcess.TryCompilation(Stream ms, Stream symbolStream, CSharpCompilation compilation, EmitResult previousEmitResult, Boolean lastAttempt, Int32 retryCount) in C:\Dev\Repos\stryker-net\src\Stryker.Core\Stryker.Core\Compiling\CsharpCompilingProcess.cs:line 172
   at Stryker.Core.Compiling.CsharpCompilingProcess.Compile(IEnumerable`1 syntaxTrees, Stream ilStream, Stream symbolStream) in C:\Dev\Repos\stryker-net\src\Stryker.Core\Stryker.Core\Compiling\CsharpCompilingProcess.cs:line 63
   at Stryker.Core.MutationTest.CsharpMutationProcess.CompileMutations(MutationTestInput input, CsharpCompilingProcess compilingProcess) in C:\Dev\Repos\stryker-net\src\Stryker.Core\Stryker.Core\MutationTest\CsharpMutationProcess.cs:line 89
   at Stryker.Core.MutationTest.CsharpMutationProcess.Mutate(MutationTestInput input) in C:\Dev\Repos\stryker-net\src\Stryker.Core\Stryker.Core\MutationTest\CsharpMutationProcess.cs:line 79
   at Stryker.Core.MutationTest.MutationTestProcess.Mutate() in C:\Dev\Repos\stryker-net\src\Stryker.Core\Stryker.Core\MutationTest\MutationTestProcess.cs:line 88
   at Stryker.Core.Initialisation.ProjectMutator.MutateProject(StrykerOptions options, MutationTestInput input, IReporter reporters) in C:\Dev\Repos\stryker-net\src\Stryker.Core\Stryker.Core\Initialisation\ProjectMutator.cs:line 38
   at Stryker.Core.Initialisation.ProjectOrchestrator.MutateProjects(StrykerOptions options, IReporter reporters, ITestRunner runner)+MoveNext() in C:\Dev\Repos\stryker-net\src\Stryker.Core\Stryker.Core\Initialisation\ProjectOrchestrator.cs:line 68
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Stryker.Core.StrykerRunner.RunMutationTest(IStrykerInputs inputs, ILoggerFactory loggerFactory, IProjectOrchestrator projectOrchestrator) in C:\Dev\Repos\stryker-net\src\Stryker.Core\Stryker.Core\StrykerRunner.cs:line 63
   at Stryker.CLI.StrykerCli.RunStryker(IStrykerInputs inputs) in C:\Dev\Repos\stryker-net\src\Stryker.CLI\Stryker.CLI\StrykerCLI.cs:line 102
   at Stryker.CLI.StrykerCli.<>c__DisplayClass11_0.<Run>b__0() in C:\Dev\Repos\stryker-net\src\Stryker.CLI\Stryker.CLI\StrykerCLI.cs:line 74
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass143_0.<OnExecute>b__0(CancellationToken _)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args)
   at Stryker.CLI.StrykerCli.Run(String[] args) in C:\Dev\Repos\stryker-net\src\Stryker.CLI\Stryker.CLI\StrykerCLI.cs:line 80
   at Stryker.CLI.Program.Main(String[] args) in C:\Dev\Repos\stryker-net\src\Stryker.CLI\Stryker.CLI\Program.cs:line 14

Logs log-20240315.txt

Desktop (please complete the following information):

  • OS: Windows
  • Type of project: Stryker latest master branch
  • Stryker Version: 4.0.0

Additional context I reverted some commits to locate that this behaviour was added during the refactor of the mutation orchestration by @dupdob

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 16 (14 by maintainers)

Most upvoted comments

update: the design I had in mind works. I had to add several orchestrators, which I do not like. But, I finally got my head around chained MemberAccesses (and variants). Still have to streamline the design a bit. But I can open a PR now if the fix is urgent

look what Copilot proposed me while working on a recovery strategy: _logger.LogError("Failed to compile due to a NullReferenceException. This is a known issue in Roslyn. Please report the issue to the Roslyn team."); So far, I did not spot anything surprising while reading the mutated code. But I am almost done having a work around strategy for his kind of issues

  • identify the problematic file(s) and logs its content
  • remove all mutations from this file and flag them as compilation error
  • restore the normal Stryker flows
  • spot what syntax construct triggered the error
  • [ ] fix Stryker accordingly
  • alt. devise a better workaround if this is a genuine Roslyn bug

the PR is ready for review. As said earlier, I will not attempt to reach expected coverage level as it requires wrapping C# compiler

I will see today how far I am for a good solution and decide afterward. One thing is sure: there is no quick fix, it needs to add a few orchestrators and maybe add features to MutationContext. So if I think this is too far away, I will open a PR with an interim solution. Here are the things I want to fix:

  • restore the normal Stryker flows (not looking into the resource problem yet
  • streamline the current fix: the new orchestrators may be reduced to a smaller code base via some shared base class, for example
  • increase mutation agility: as of now, Stryker can only mutate whole member access chains, such as x.y.z().t.u, at once, while it would be smarter to mutate only the beginning (which the master version does, sometimes).

The problem with member access chains is that they are described right from left, which may force some double scanning…

update: the root cause is a regression with ConditionalAccessExpressions (?.). I am trying to find a way to deal with these in some central fashion, and not by adding ifs everywhere. My last refactoring went too far; early design was too clunky, the current one is too streamlined. I may have a better design in mind. We will see about that.

I have been able to successfully compile 2 of the problematic files, the remaining three contains various constructions with conditional access expressions that force me to think about a better design.

Regarding restoring proper workflow in case of Roslyn exception, I have an almost working version, but I still have issues with resources: streams are finalized when the exception is thrown. I will see what can be done about it, but my focus is on the conditional access expression stuff.

Edited to correct usage of ConditionalExpressions (?:) instead of the proper ConditionalAccessExpression (?.) Stupid me

I reproduce, looking into it. Crashing the compiler is pretty rare…