testng: Priority of test methods is ignored

TestNG Version

org.testng:testng:7.0.0

(This worked just fine with version 6.9.6)

Expected behavior

Because of the set priorities I expect following execution order: test05 test04 test02 test03 test01

This works when running from Eclipse IDE.

Actual behavior

Unfortunately, when running from IntelliJ or Gradle (tried with 5.4.1 and 5.6.2) it results in following execution order: test01 test02 test03 test04 test05

Is the issue reproductible on runner?

  • Shell
  • Maven
  • Gradle
  • Ant
  • Eclipse
  • IntelliJ
  • NetBeans

Test case sample

I have following test class:

public class TryOut {

	@Test(priority = 4)
	public void test01() {
		System.out.println("test01");
	}

	@Test(priority = 3)
	public void test02() {
		System.out.println("test02");
	}

	@Test(priority = 3)
	public void test03() {
		System.out.println("test03");
	}

	@Test(priority = 2)
	public void test04() {
		System.out.println("test04");
	}

	@Test(priority = 1)
	public void test05() {
		System.out.println("test05");
	}
}

build.gradle:

plugins {
    id 'java'
}

repositories {
    mavenCentral()
}

dependencies {
    implementation 'org.testng:testng:7.0.0'
}

// Test Logging
test {
	useTestNG()
    testLogging {
        showStandardStreams = true
    }
}

The Gradle command: ./gradlew test --tests mypackage.TryOut

Here the complete Gradle project: TestNgExample.zip

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 33 (19 by maintainers)

Most upvoted comments

Problem with 7.4.0 is still exist. I am using Gradle project.

I’m able to reproduce the issue in a fresh IntelliJ project but not if I add the test in the TestNG project itself. I confirm the issue happens only with 7.x versions and not previous ones.

And did you go through the comments? It was confirmed by one of the users that they aren’t experiencing the issue any more.

Not sure what is expected here after a confirmation from someone that the problem is no longer seen.

I can understand that you are frustrated with this issue but that doesnt mean that you are entitled to vent it out without any consideration for the time that we spend as committers on an open source project.

@armadco - can we please keep the comments a bit more civil and considerate. Demanding that the issue be fixed is not going to help in getting it fixed. You have shared a sample. Thanks. I will try and use it to figure out what is going wrong. Please remember that i have a day job and i do this out of passion.

If this is a very urgent issue, maybe you could help propose a PR that fixes the problem? We will be more than glad to help get it reviewed and merged.

@ispringer - Thanks for the confirmation. I am going ahead and closing this issue and marking this fixed in 7.5 and higher (7.7.0 as of today)