fladle: Configuration option to ignore failures

Standard test tasks support ignoreFailures, which Flank doesn’t appear to. Ignoring failures is useful for CI, where it’s more desirable for Gradle invocations to complete successfully, and test results to be analyzed afterward.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 21 (9 by maintainers)

Most upvoted comments

Gradle 6.1 is released with the ability to check the exit code for JavaExec tasks! https://docs.gradle.org/6.1/release-notes.html

https://github.com/gradle/gradle/pull/10763

I reopened https://github.com/Flank/flank/issues/602

Current idea is if ignoreExitValue is set then we’ll always return a 0 exit code.

@runningcode Should ignoreExitValue be a part of flank extension?

In case anyone stumbles into this Github issue in the future. I would not recommend ignoring flanks exit value. If a matrix fails for any reason other than the test itself failing (bad matrix, device failure, infrastructure overload) than the failure will not appear in the junit report. This is a dangerous false positive situation in the test reporting system in flank. See this issue for more details: https://github.com/TestArmada/flank/issues/457

Kotlin needs a bit of help to figure out the type of the execFlank task. Please try the following kts compatible™ snippet and let me know how it goes. 😃

afterEvaluate {
  tasks.named("execFlank", JavaExec::class.java).configure {
    isIgnoreExitValue = true
  }
}