mockk: Tests failing after coroutines update

We’re also affected by this. Running a single test works, but running multiple tests (say multiple test classes) throws UncaughtExceptionsBeforeTest.

We’re seeing the same thing: multiple failures with previously-working tests after updating to Kotlin 1.8.22, which updates coroutines from 1.6.4 to 1.7.1. The failures are kotlinx.coroutines.test.UncaughtExceptionsBeforeTest followed by Suppressed: io.mockk.MockKException: no answer found for: ..., as noted above.

Is there any other way? fixing all issues is not possible 😦(

For now, the workaround we use is to use “runBlocking” instead of “runTest”.

What if the exceptions are thrown by a third party library, cannot be fixed and are not blocking ? How are we supposed to fix this please ?

Seeing the same issue. Running single test class works, but running a suite or running all of it in a module on CI breaks it.

EDIT: So it looks like runTest is checking for previous exceptions before executing the provided block and this could explain why a test won’t fail when run individually, but it can indeed fail when multiple tests are run and at least one of them threw a suppressed exception. Furthermore this suggest that the same TestScope is used for a single test run, regardless of the number of tests involved.

Great discovery! Mystery solved.

I don’t know why that happens, but I suspect if you scroll to the bottom of the console output you will see issues whether the tests are run singly or in a group. And once you fix the issues, the tests will pass when run in a group and when run singly.

Thank you for sharing this! So if this is the case I think adding this to the build file might help finding the culprit:

tasks.withType<Test> {
    testLogging {
        exceptionFormat = TestExceptionFormat.FULL
        showStackTraces = true
    }
}

Great idea! Looks like showStandardStreams might help, too. It logs stdOut and stdErr as well. See https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.logging.TestLogging.html for more options.

By the way it is more convenient to observe stack traces in the generated HTML report than in the terminal. It seems like they can get truncated in the terminal.

EDIT: So it looks like runTest is checking for previous exceptions before executing the provided block and this could explain why a test won’t fail when run individually, but it can indeed fail when multiple tests are run and at least one of them threw a suppressed exception. Furthermore this suggest that the same TestScope is used for a single test run, regardless of the number of tests involved.

I don’t know why that happens, but I suspect if you scroll to the bottom of the console output you will see issues whether the tests are run singly or in a group. And once you fix the issues, the tests will pass when run in a group and when run singly.

Thank you for sharing this! So if this is the case I think adding this to the build file might help finding the culprit:

tasks.withType<Test> {
    testLogging {
        exceptionFormat = TestExceptionFormat.FULL
        showStackTraces = true
    }
}

In our case, the solution was to fix all the issues with our unit tests. These errors existed before updating coroutines, but they didn’t cause the unit tests to fail. Rather, the errors were output to the console and were not visible unless you scrolled to the bottom of a lot of log lines. Because the tests passed we didn’t know there were any issues.

After updating coroutines the tests failed, at least when run in groups, so we were forced to address all the issues. This is a good thing. Most of the issues were the No answer for... types described above, though we did find other issues, and we discovered that some unit tests were not correct once we provided all mocks so the tests could complete without throwing Exceptions in coroutines.

Though it was a pain to fix the issues, our code is in much better shape now and we have a lot more confidence in our tests.

Interesting! How does this explain the fact that the tests are successful when run individually?