testng: throwing SkipException sets iTestResult status to Failure instead of Skip

TestNG Version 6.13.1

Expected behavior

previously I had in beforeInvocation:

throw new SkipException("Skipped due to previous failure of: " + failedTestName);

So I have TestResult.SKIP in afterInvocation.

Discussion started here: https://github.com/cbeust/testng/pull/1611

Actual behavior

After upgrading to 6.13.1 it stopped working, because iTestResult.getStatus() is now TestResult.FAILURE instead of TestResult.SKIP in afterInvocation.

to workaround this I’m setting status explicitly before throwing SkipException:

iTestResult.setStatus(TestResult.SKIP);
throw new SkipException("Skipping the test case");

Is the issue reproductible on runner?

Shell
Maven
Gradle
Ant
Eclipse
IntelliJ
NetBeans

Test case sample

// listener
public class TestNGExecutionListener implements IInvokedMethodListener {
    @Override
    public void beforeInvocation(IInvokedMethod iInvokedMethod, ITestResult iTestResult) {
        throw new SkipException("skip");
    }

    @Override
    public void afterInvocation(IInvokedMethod iInvokedMethod, ITestResult iTestResult) {
        iTestResult.getStatus(); // actual Failure (2), expected Skip (3)
    }
}

// add listener
testng.addListener((ITestNGListener) new TestNGExecutionListener());

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 8
Comments: 22 (11 by maintainers)

Commits related to this issue

Streamline skipped test results in listeners Closes #1632 — committed to krmahadevan/testng by krmahadevan 5 years ago
Streamline skipped test results in listeners Closes #1632 — committed to krmahadevan/testng by krmahadevan 5 years ago
Streamline skipped test results in listeners Closes #1632 — committed to testng-team/testng by krmahadevan 5 years ago
Streamline TestResult management from listeners Fixes #1632 — committed to juherr/testng by krmahadevan 3 years ago

Most upvoted comments

@juherr - Lol… NO that was not what I meant. I was meaning to say that since at work, I am trying to transition into a backend developer, I am having to invest a lot of time trying to figure out things in Spring etc., and hence I haven’t been able to find a lot of time to get to take a look at this issue. btw thanks for adding that sample. I will take it up from there hopefully over this weekend.

krmahadevan on Jan 16, 2021

@juherr - As a first step, i went ahead and added 7.4.0 as a milestone to all defects that we have fixed so far after we released 7.3.0. Going forward when we close an issue, we should ensure we update the milestone to whatever is the current upcoming version. That way this mapping would be there all the time.

krmahadevan on Jan 22, 2021

@sbrannen - Thank you for bringing up this. I will start adding this milestone for all the issues that would be part of 7.4.0 going forward.

krmahadevan on Jan 21, 2021

@sbrannen - 7.3.0 is already released. This should be part of 7.4.0 (which would be the next upcoming release for TestNG)

@juherr - I think we should start leveraging the milestone capability in github so that our users exactly know as to when would a fix be available. Any thoughts on how to go about doing this ?

krmahadevan on Jan 21, 2021

I believe this is effectively a duplicate of https://github.com/spring-projects/spring-framework/issues/26387.

See https://github.com/spring-projects/spring-framework/issues/26387#issuecomment-760326643 for further details.

sbrannen on Jan 14, 2021