pitest: Lot of random errors - "Minion exited abnormally due to RUN_ERROR", "Minion exited abnormally due to TIMEOUT_ERROR"

We are trying to bring Mutation testing using pitest. But we are facing random errors - “Minion exited abnormally due to RUN_ERROR”, “Minion exited abnormally due to TIMEOUT_ERROR” etc.,

We have added the plugin - id 'info.solidsoft.pitest' version '1.7.4'

The gradle task looks like -

pitest { junit5PluginVersion.set(“1.0.0”) pitestVersion.set(“1.9.0”) targetClasses.addAll(“com.abc.def.deliveries.configuration.","com.abc.def.deliveries.security.”) targetTests.addAll(“com.abc.def.deliveries.*”) // test packages to test coverage on mutators.addAll(“ALL”) threads.set(5) failWhenNoMutations.set(true) mutationThreshold.set(70) // minimum of 70% of mutations need to be caught in order for test to pass reportDir.set(file(“$buildDir/pitestHtml”)) // output of report }

On running the task, with the command - ./gradlew :deliveries:pitest -DtimeoutConstant=10000

The logs look like -

Task :deliveries:pitest 17:26:19 PIT >> INFO : Verbose logging is disabled. If you encounter a problem, please enable it before reporting an issue. 17:26:21 PIT >> INFO : Incremental analysis reduced number of mutations by 0 17:26:21 PIT >> INFO : Created 8 mutation test units in pre scan 17:26:23 PIT >> INFO : Sending 35 test classes to minion 17:26:23 PI| >> INFO : /ent tests\to minion / \ - / /17:27:56 PIT >> INFO : Calculated coverage in 95 seconds. - 17:27:56 PIT >> INFO : Incremental analysis reduced number of mutations by 0 17:27:56 PIT >> INFO : Created 8 mutation test units 17:28:10 PIT >> WARNING : Minion exited abnormally due to RUN_ERROR 17:28:27 PIT >> WARNING : Minion exited abnormally due to RUN_ERROR
17:28:50 PIT >> WARNING : Minion exited abnormally due to RUN_ERROR 17:29:09 PIT >> WARNING : Minion exited abnormally due to RUN_ERROR 17:29:28 PIT >> WARNING : Minion exited abnormally due to RUN_ERROR 17:30:22 PIT >> WARNING : Minion exited abnormally due to RUN_ERROR 1:31|01 PIT >> 17:32:36 PIT >> INFO : Completed in 376 secondsRROR
================================================================================

• Mutators ================================================================================

org.pitest.mutationtest.engine.gregor.mutators.RemoveConditionalMutator_ORDER_ELSE

Generated 1 Killed 0 (0%) KILLED 0 SURVIVED 1 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 0

---

org.pitest.mutationtest.engine.gregor.mutators.VoidMethodCallMutator

Generated 5 Killed 0 (0%) KILLED 0 SURVIVED 5 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 0

---

org.pitest.mutationtest.engine.gregor.mutators.experimental.MemberVariableMutator

Generated 20 Killed 8 (40%) KILLED 8 SURVIVED 7 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 5

---

org.pitest.mutationtest.engine.gregor.mutators.RemoveConditionalMutator_EQUAL_ELSE

Generated 24 Killed 3 (13%) KILLED 3 SURVIVED 4 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 17

---

org.pitest.mutationtest.engine.gregor.mutators.returns.NullReturnValsMutator

Generated 27 Killed 25 (93%) KILLED 10 SURVIVED 2 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 15 NO_COVERAGE 0

---

org.pitest.mutationtest.engine.gregor.mutators.RemoveConditionalMutator_ORDER_IF

Generated 1 Killed 0 (0%) KILLED 0 SURVIVED 1 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 0

---

org.pitest.mutationtest.engine.gregor.mutators.MathMutator

Generated 8 Killed 0 (0%) KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 8

---

org.pitest.mutationtest.engine.gregor.mutators.returns.EmptyObjectReturnValsMutator

Generated 13 Killed 11 (85%) KILLED 11 SURVIVED 1 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 1

---

org.pitest.mutationtest.engine.gregor.mutators.NegateConditionalsMutator

Generated 26 Killed 5 (19%) KILLED 5 SURVIVED 3 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 18

---

org.pitest.mutationtest.engine.gregor.mutators.returns.PrimitiveReturnsMutator

Generated 1 Killed 0 (0%) KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 1

---

org.pitest.mutationtest.engine.gregor.mutators.ConditionalsBoundaryMutator

Generated 1 Killed 0 (0%) KILLED 0 SURVIVED 1 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 0

---

org.pitest.mutationtest.engine.gregor.mutators.experimental.ReturnValuesMutator

Generated 50 Killed 39 (78%) KILLED 18 SURVIVED 2 TIMED_OUT 0 NON_VIABLE 2 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 19 NO_COVERAGE 9

---

org.pitest.mutationtest.engine.gregor.mutators.returns.BooleanTrueReturnValsMutator

Generated 5 Killed 0 (0%) KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 5

---

org.pitest.mutationtest.engine.gregor.mutators.InlineConstantMutator

Generated 28 Killed 4 (14%) KILLED 4 SURVIVED 7 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 17

---

org.pitest.mutationtest.engine.gregor.mutators.experimental.NakedReceiverMutator

Generated 12 Killed 8 (67%) KILLED 8 SURVIVED 4 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 0

---

org.pitest.mutationtest.engine.gregor.mutators.ReturnValsMutator

Generated 41 Killed 38 (93%) KILLED 23 SURVIVED 2 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 15 NO_COVERAGE 1

---

org.pitest.mutationtest.engine.gregor.mutators.experimental.ArgumentPropagationMutator

Generated 3 Killed 3 (100%) KILLED 3 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 0

---

org.pitest.mutationtest.engine.gregor.mutators.ConstructorCallMutator

Generated 4 Killed 1 (25%) KILLED 1 SURVIVED 3 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 0

---

org.pitest.mutationtest.engine.gregor.mutators.returns.BooleanFalseReturnValsMutator

Generated 3 Killed 1 (33%) KILLED 1 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 2

---

org.pitest.mutationtest.engine.gregor.mutators.NonVoidMethodCallMutator

Generated 98 Killed 56 (57%) KILLED 41 SURVIVED 17 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 15 NO_COVERAGE 25

---

org.pitest.mutationtest.engine.gregor.mutators.RemoveConditionalMutator_EQUAL_IF

Generated 25 Killed 4 (16%) KILLED 4 SURVIVED 3 TIMED_OUT 0 NON_VIABLE 0 MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 NO_COVERAGE 18

---

================================================================================

• Timings ================================================================================

pre-scan for mutations : < 1 second scan classpath : < 1 second coverage and dependency analysis : 1 minutes and 35 seconds build mutation tests : < 1 second run mutation analysis : 4 minutes and 39 seconds

---

Total : 6 minutes and 15 seconds

---

Since the tests are failing randomly, our confidence of using pitest is decreasing. Can you please take a look at this?