testng: DataProvider not running tests In Parallel when using Factory

TestNG Version

7.0.0-beta1

Expected behavior

I have a class that contains a DataProvider with 5 data sets including browser name, version, and platform as you can see here:

public class DataProviderManager {
    @DataProvider(parallel = true)
    public static Object[][] defaultWebDataProvider() {
        return new Object[][] {
                new Object[]{"chrome", "70", "Windows 10"},
                new Object[]{"internet explorer", "11", "Windows 10"},
                new Object[]{"safari", "12.0", "Mac 10.13"},
                new Object[]{"chrome", "70", "Mac 10.13"},
                new Object[]{"firefox", "63", "Mac 10.13"}
        };
    }
}

In my test class I am using Factory to pass data as such:

public class MSGWebExampleTest extends BaseTest {
    // Declare page objects
    private MSGHome msgHome;
    // Declare test data

    @Factory(dataProvider = StaticProps.DEFAULT_WEB_DATA_PROVIDER, dataProviderClass = DataProviderManager.class)
    public MSGWebExampleTest(String browser, String browserVersion, String platform) {
        super.setRunParams(browser, browserVersion, platform);
    }

    @BeforeMethod(alwaysRun = true)
    public void setupTest() {
        msgHome = new MSGHome();
    }

    @Features({GroupProps.WEB})
    @Test(groups = {GroupProps.DEBUG})
    @Parameters({ParamProps.WEB, ParamProps.MOBILE_WEB})
    public void openSiteTest() {
        new WebInteract(null,null).pause(1000).openUrl(URLBuilder.buildUrl());
    }
}

testing.xml:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Test Engineering Core Suite" parallel="methods" thread-count="2" data-provider-thread-count="4">

If I set thread-count =2 and data-provider-thread-count =4 and parallel=true my test should run 8 times in parallel.

Actual behavior

Tests run sequentially regardless of how many threads I try to set as if parallel=true is ignored. I tried numerous attempts to get this to run in parallel using the testing.xml above, by passing -Dthreadcount=2 -Ddataproviderthreacount=4 from commandline and even setting these values in the config tag for the surefire plugin and nothing is working.

Is the issue reproducible on runner?

  • Shell
  • Maven
  • Gradle
  • Ant
  • Eclipse
  • IntelliJ
  • NetBeans

Test case sample

Please, share the test case (as small as possible) which shows the issue

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 15 (8 by maintainers)

Most upvoted comments

@borbamartin - To the best of my knowledge, there’s nothing in TestNG that will automatically do this for you. You would need to do it yourself.

@VinceBarresi - I have added a detailed answer to your question on stack overflow.

To wrap up in a nutshell, when you use a @Factory with a @DataProvider to produce test class instances, TestNG instantiates test class instances in parallel, but runs each of the @Test methods in each of the test classes in sequence.