mockk: Android UI Test crashing on Android Emulator (but fine on physical devices) running Android P+ due to various SIGSEGV at random time
Prerequisites
- I am running the latest version
- I checked the documentation and found no answer
- I checked to make sure that this issue has not already been filed
Expected Behavior
Using MockK on Android UI Tests should work on any emulator running Android 9+ as it does on physical devices.
Current Behavior
On the emulator, most of the time, a test (as part of a test class file) using mockK feature will fail randomly with the following message:
Test failed to run to completion. Reason: ‘Instrumentation run failed due to ‘Process crashed.’’. Check device logcat for details Test running failed: Instrumentation run failed due to ‘Process crashed.’
This so-called crash can come as early as right before the 1st test could event start or happen right in the middle of the tests sequence. It’s quite random regardless of the test order.
Also, in very rare cases, the crash might not occur and all tests pass. But again, way too random.
Failure Information (for bugs)
In most cases, the crash reason was printed in the logs. Most of them are SIGENV but the cause of it can vary, without any clear reasons. Looking deeper in the logs there seem to be an issue related to the InputDispatcher:
InputDispatcher: channel ‘9fa7335 my.company.com.android.debug/my.company.com.ui.login.LoginActivity(server)’ ~ Channel is unrecoverably broken and will be disposed!
Looking on SO about that message, it seams it could be related to some threading issue when manipulating Activities that are not “active”.
Steps to Reproduce
Unfortunately I cannot provide the code to reproduce the issue. However, my usage of mockK features is as follow, in a MVP architecture:
@Before fun setUp()will callmockkObjectbefore each test to mock various Repositories and Services Kotlin Objects@After fun teardDown()will callunmockkAllto clean mocks after each test.coEveryandeveryto mock the relevent methods call in order to return custom values for the SUTverifywhen I need to make sure a specific method was used
These tests also include Espresso View assertions to validate the UI being displayed.
Context
Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.
- MockK version: 1.10.0
- OS: Windows 10
- Kotlin version: 1.3.72’
- JDK version: 1.8.0_242-release
- JUnit version: 4.12
- Type of test: android instrumented test
Failure Logs
Here are some of the SIGSEGV I could catch in the logs:
A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 10425 (e.android.debug), pid 10425 (e.android.debug)
A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc in tid 10968 (HeapTaskDaemon), pid 10957 (e.android.debug)
A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 15050 (firebase-instal), pid 14972 (e.android.debug)
A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xd in tid 8902 (Measurement Wor), pid 8858 (e.android.debug)
A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 22004 (Binder:21832_5), pid 21832 (e.android.debug)
In two cases, I had very different reasons:
E/OpenGLRenderer: GL error: 0x506 A/OpenGLRenderer: glViewport error! UNKNOWN (0x506) A/libc: Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 11968 (RenderThread), pid 11862 (e.android.debug)
E/e.android.debu: Thread[ThreadState[0],Thread*=0xe9481c10,peer=0x0,“null”] suspend count already zero.
Minimal reproducible code (the gist of this issue)
// -----------------------[ GRADLE DEFINITIONS ] -----------------------
dependencies {
// JUnit
testImplementation "junit:junit:$junit_version"
androidTestImplementation "androidx.test.ext:junit-ktx:$androidx_test_junit_ktx_version"
// Unit Testing
testImplementation "io.mockk:mockk:$mockk_version"
testImplementation "org.jetbrains.kotlinx:kotlinx-coroutines-test:$kotlin_coroutines_test_version"
testImplementation "org.robolectric:robolectric:$robolectric_version"
// Android X Testing
androidTestImplementation "androidx.test:core:$androidx_test_version"
androidTestImplementation "androidx.test:rules:$androidx_test_version"
androidTestImplementation "androidx.test:runner:$androidx_test_version"
// Fragment Testing
debugImplementation "androidx.fragment:fragment-testing:$fragment_test_version"
debugImplementation "androidx.fragment:fragment-ktx:$fragment_test_version"
// Espresso Testing
androidTestImplementation "androidx.test.espresso:espresso-contrib:$espresso_version"
androidTestImplementation "androidx.test.espresso:espresso-core:$espresso_version"
// Mock Testing
androidTestImplementation "io.mockk:mockk-android:$mockk_version"
// variables for versions
androidx_test_version = "1.2.0"
androidx_test_junit_ktx_version = "1.1.1"
espresso_version = "3.2.0"
fragment_test_version = "1.2.5"
junit_version = "4.12"
kotlin_coroutines_test_version = "1.3.7"
robolectric_version = "4.3.1"
mockk_version = "1.10.0"
}
Finally, before deciding to poste this issue, I investigated more and I posted my issue on SO: https://stackoverflow.com/questions/63059986/android-ui-tests-with-espresso-mockk-crash-with-sigsegv
And after further investigation, some other people posted it on the android-test Github repo more than a year ago: https://github.com/android/android-test/issues/352.
About this issue
- Original URL
- State: open
- Created 4 years ago
- Reactions: 24
- Comments: 34 (2 by maintainers)
This should not be closed
This was in our stacktrace:
Switching to a 64 bit emulator solved the problem.
Switching to x86_64 emulator in Android Studio solved this for me, too! I am not sure if you can specify the type of emulator architecture in Firebase Test Lab, unfortunately. mockK 1.10.0 and mockK Android 1.9.3 with objenesis forced at 2.6. I hope the Androidx test team can fix this bug. In Firebase Test Lab, you can’t specify the ABI of the emulators used, so it appears basically impossible to run your UI tests in Firebase Test Lab on emulators if you use mockK or mockito
https://github.com/android/android-test/issues/352
Hi there 👋 I’ve been able to create a minimal reproducible example that will almost always segfault on virtual FTL devices, but always succeed on FTL physical devices AND local emulators.
https://github.com/SimonMarquis/FTL-segfault
Here is an example output of a failed test run: https://github.com/SimonMarquis/FTL-segfault/actions/runs/2873978593
All failed tests are due to segmentation faults. This is very similar to what’s reported in https://github.com/android/android-test/issues/352.
I sincerely hope this will help pinpoint the issue that is crippling our CI builds for too long.
Looking ahead: with the deeper understanding of mocking and APIs the mocking libraries use, would the mockk team be able to report issue(s) to Android so that they can fix and improve in this regard? Hopefully they would prioritize things if it was coming from the direct source on behalf of all the Android developers using mockk. This way in the mid-far future we would have Android versions that have a fully working JVM and compatible with advanced mocking.
Hi everyone and much thanks for all of your feedback for the past year.
I would like so summarize / conclude.
First of all, it now seems evident that this issue is no related to MockK nor to Mockito inasmuch both of these libraries rely to the same internal Android instrumentation APIs that do not work well with Kotlin Objects. All in all, as long as you don’t try to mock Kotlin Objects in UI instrumented tests, you should be fine.
Some of you have reported that using 64 bits simulators yielded much better results and this is good news indeed but still an issue / impediment anyway, in my opinion.
For the project I work on, we recently resumed UI tests but with a new approach where we decided that we should not mock Kotlin Objects but instead, the layer that stand between the UI and the services / business logics. These Kotlin Objects will only be mocked for unit testings which work very well.
I sincerely hope this issue can be solved but I really do believe neither Mockk and Mockito are responsible.
@NitroG42 bad news, the crashs are back. So I now believe that MockK is not culprit.
More info here: https://github.com/android/android-test/issues/352#issuecomment-669049551
Any updates on this !! Anybody
I’ve commented on the #test-lab slack channel of Firebase Community, and I’m in contact with Firebase Support by email.
Does anyone have any information on if/when Firebase Test Lab will provide an emulator that uses x86_64? I created this StackOverflow question to ask about it. Somebody mentioned “x86_64 support is expected to come later this year when Android S devices are released.” But, I can’t find official documentation of that claim anywhere.
I had exactly the same issue. In my case it was
com.linkedin.dexmaker:dexmaker-mockito-inlinecausing the random crashes. For details see my answer on stackoverflow:I just run into this issue when our tests would fail on emulator, but run on physical device. I used
git bisectand identified the commit that caused the issue and it was the one that introducedmockk-android. We have some 300+ instrumented tests and virtually all of them are using mocks, mostly with Mockito. Only 3 of them are using mock-android and if I remove the usage from them, the whole test suite runs fine. So in my experience, even if mockk might not be the root cause of the issue, it is definitely the thing that triggers it.I tried running the tests on emulator with different versions:
After a little experimenting, what I found was that if we use
@Mock + MockKAnnotations.init(this), the crash is pretty much guaranteed immediately. When I changed the mock to callingmockk(), I usually get at least 1 test to run, sometimes more before crashing the process.