android-test: Parameterized Android tests on Robolectric could not be run

Description

I had the need to test a specific feature of a paging component that used Room, and I needed to use a real in memory version of Room, so I needed the Context. Since the tests cases needed to cover different inputs, but the logic was the same, I intended to use Parameterized tests from JUnit 4

Steps to Reproduce

Open the SimpleParameterizedTest.kt class and try to execute it with Robolectric.

Actual Results

If we run the suite with @RunWith(AndroidJUnit4::class), we get the following error:

java.lang.RuntimeException: Delegate runner 'org.robolectric.RobolectricTestRunner' for AndroidJUnit4 could not be loaded.

	at androidx.test.ext.junit.runners.AndroidJUnit4.throwInitializationError(AndroidJUnit4.java:92)
	at androidx.test.ext.junit.runners.AndroidJUnit4.loadRunner(AndroidJUnit4.java:82)
	at androidx.test.ext.junit.runners.AndroidJUnit4.loadRunner(AndroidJUnit4.java:51)
	at androidx.test.ext.junit.runners.AndroidJUnit4.<init>(AndroidJUnit4.java:46)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:104)
	at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:49)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at androidx.test.ext.junit.runners.AndroidJUnit4.loadRunner(AndroidJUnit4.java:72)
	... 16 more
Caused by: org.junit.runners.model.InitializationError
	at org.junit.runners.ParentRunner.validate(ParentRunner.java:418)
	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:84)
	at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:65)
	at org.robolectric.internal.SandboxTestRunner.<init>(SandboxTestRunner.java:56)
	at org.robolectric.RobolectricTestRunner.<init>(RobolectricTestRunner.java:92)
	... 21 more

If we run the suite with @RunWith(Parameterized::class), we instead get the following error:

java.lang.IllegalStateException: No instrumentation registered! Must run under a registering instrumentation.

	at androidx.test.platform.app.InstrumentationRegistry.getInstrumentation(InstrumentationRegistry.java:45)
	at androidx.test.core.app.ApplicationProvider.getApplicationContext(ApplicationProvider.java:41)
	at net.orgiu.tests.SimpleParameterizedTest.<init>(SimpleParameterizedTest.kt:15)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.junit.runners.parameterized.BlockJUnit4ClassRunnerWithParameters.createTestUsingConstructorInjection(BlockJUnit4ClassRunnerWithParameters.java:43)
	at org.junit.runners.parameterized.BlockJUnit4ClassRunnerWithParameters.createTest(BlockJUnit4ClassRunnerWithParameters.java:38)
	at org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

AndroidX Test and Android OS Versions

I am trying with the following dependencies

testImplementation 'androidx.test:runner:1.1.1'
testImplementation 'androidx.test.espresso:espresso-core:3.1.1'
testImplementation 'androidx.test.ext:junit:1.1.0'
testImplementation 'org.robolectric:robolectric:4.1'

Link to a public git repo demonstrating the problem:

GitHub Repository

About this issue

Original URL
State: open
Created 5 years ago
Reactions: 7
Comments: 15 (2 by maintainers)

Most upvoted comments

Hey @tiwiz So, this is kind of tough right now…

Robolectric needs its own test runner in order to create its own sandbox environment to execute Android code, so to write parameterized tests in Robolectric you need to use @RunWith(RobolectricParameterizedTestRunner.class)
Instrumentation tests don’t have that restriction so you can legitimately use @RunWith(Parameterized.class)
The unified test runner is AndroidJUnit4.class which will detect the environment in which you run and delegate to either Robolectric or Instrumentation under the covers. However, it just supports regular JUnit4 style tests, not parameterized.

Long term Nitrogen will solve this as we won’t need any custom JUnit4 test runners to bootstrap Robolectric or Instrumentation tests (which will be done via N2 itself.

So… what can we do in the meantime?

I chatted with the team and we think creating a ParameterizedAndroidJUnit4.class which will would replace RobolectricParameterizedTestRunner.class and also delegate to the regular Parameterized.class to work on-device too.

This helps with our goal of consolidating Robolectric test runners into AndroidX Test and it will also remain compatible (although not necessary) once Nitrogen + Sheep device are shipped.

What do you think about having a go at implementing this ParameterizedAndroidJUnit4 test runner? We’re not going to get to it right very soon, but we’d provide full support in guidance and code reviews and getting it into a release. You’d get all the glory 😃

jongerrish on Mar 29, 2019

And here we are one year later 😢

mrctrifork on Jun 3, 2020

@sergio-sastre @alixwar Maybe you can check https://android-review.googlesource.com/c/platform/platform_testing/+/2623172, the CL for ParameterizedAndroidJUnit4. @brettchabot Will you have a plan to public it or does it exist in the AXT repository already?

utzcoz on Jun 14, 2023

Here’s how you do it as per 4/06/2020

@RunWith(ParameterizedRobolectricTestRunner::class)
@Config(sdk = [Build.VERSION_CODES.O_MR1] )
class CypherUnitTest(
    val param1: Double,
    val param2: Double,
    val param3: Double,
    val param4: String,
    val param5 String
) {

      companion object {
             private fun mold(
                  val param1: Double,
                  val param2: Double,
                  val param3: Double,
                  val param4: String,
                  val param5 String
            ) = arrayOf(param1, param2, param3, param4, param5)
           
            @JvmStatic
            @ParameterizedRobolectricTestRunner.Parameters
             fun testCases() = listOf(
                   mold(…),
                   mold(…),
                   mold(…)
             )
      }

     @Test
      fun yourTest(){ … }

}

mrctrifork on Jun 4, 2020

@jongerrish Appreciate the thoughtful response here. Is there any update on the release of N2? Anxiously waiting to get my hands on it. Would love to help contribute if any of it is open source.

mrk-han on Mar 29, 2019