testng: Parameterized test class crashes when data provider returns empty array

Let’s say I have a test class with a @Factory-annotated constructor:

import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;

import static org.testng.Assert.assertEquals;

public class TestTest {
    @DataProvider(name = "values")
    public static Object[][] values() {
        return new Object[][]{{1}, {2}, {3}};
    }

    private final int value;

    @Factory(dataProvider = "values")
    public TestTest(int value) {
        this.value = value;
    }

    @Test
    public void test() {
        assertEquals(value, 2);
    }
}

This works perfectly (apart from the assertion errors this silly test throws).

However, if I change the data provider to return an empty array instead (new Object[][]{}), TestNG crashes:

org.testng.TestNGException: 
Can't invoke public void TestTest.test(): either make it static or add a no-args constructor to your class 

I expected the test to be ignored, just like it is when the data provider is on method-level instead of class-level. Is this the desired behaviour? If so, why? I ran into this issue because I have data providers that are supposed to be NOPs in certain test configurations.

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 21 (8 by maintainers)

Commits related to this issue

Most upvoted comments

@priyanshus getInstance() is null because FactoryMethod#invoke returns an empty array when the data provider is empty.

This then prevents the loop in TestNGClassFinder from adding any instances.

Since this is the only place an instance would be found in this case, we have an instance test method with no instance that can be used to call it which causes Utils#checkInstanceOrStatic to throw the error.

I have code that fixes it, but it would also hide real user errors by preventing the error thrown by Utils#checkInstanceOrStatic. So I don’t think it is a good solution yet.

@juherr do you know where I could go from here?

@cbeust Without using @Factory for dataProvider test result look like this

Total : 0 Passed: 0 Failed : 0 Skipped : 0

Either we should fix both the cases or let it be same as without @Factory i.e. 0 result.

Currently a test with an empty data provider returns 0 tests results, which seems intuitive to me, instead of marking the test as passed. I don’t believe it has a log line though, which I agree should be added.

This issue only happens when combining an empty data provider with a Factory. Should it also just show 0 tests ran, or as you mentioned, make both situations show 1 test success?