protractor-cucumber-framework: If all scenarios pass on retry, protractor still exits with status code 1

I was never able to get it to work and ended up writing a script to handle the situation. It is annoying having to do it as adds a lot of overhead but works. It creates a txt file with a test/fail result logged for each test. At end of the run it will check if any txt file includes a fail status. If not, it will override exit code to ensure pipeline passes

It just was merged, @AlexandrosD do we need a version bump / release?

the issue seemed to be back in latest version 7 & 8. but the code structure seemed to be changed a lot since 6.2.1.

@AlexandrosD would you maybe suggest for a solution? I’m willing to provide fix if possible.

I’m still getting the same error even after upgrading to v6.2.1

Same here, did you get the fix? anyone have a look？

I’m still getting the same error even after upgrading to v6.2.1

I am trying to debug an issue where I am using the new CucumberJS retry option yet the protractor process still returns an exit code of 1. My guess is that protractor-cucumber-framework reports the results to protractor which protractor then checks the failedCount and exits appropriately.

I don’t know the best spot to fix this - in protractor? In protractor-cucumber-framework?

One idea I had was that protractor-cucumber-framework would not report a failure to protractor if the retry worked, only if strict mode was false. Is this a valid option?

Right now, this is killing my ability to include this in my pipelines, as they evaluate the exit code. I would like my build to pass if everything works on retry.

@AlexandrosD Many thanks for the update. @achuinard Has 6.2.1 fixed your issue? I am still seeing same behaviour but am now wondering if it’s caused by BrowserStack exe we use to run tests:

[16:20:38] I/launcher - 0 instance(s) of WebDriver still running
[16:20:38] I/launcher - Edge #31 passed
[16:20:38] I/launcher - Firefox #41 passed
[16:20:38] I/launcher - Chrome #61 passed
[16:20:38] I/launcher - Chrome #51 passed
[16:20:38] I/launcher - android #81 passed
[16:20:38] I/launcher - Safari #71 passed
[16:20:38] I/launcher - IE #11 passed
[16:20:38] I/launcher - iPhone #01 passed
[16:20:38] I/launcher - IE #21 failed 2 test(s)
[16:20:38] I/launcher - overall: 2 failed spec(s)t
Stopping BrowserStack Local...
[16:20:44] E/launcher - Process exited with error code 1

In the above output with 6.2.1 the IE instance failed twice but both scenarios retried and passed. However, the exit code was still 1.

@achuinard how will your solution behave when restartBrowserBetweenTests is true in Protractor config? As far as I can see you’re not sending test fail/pass to Protractor so the retry will not have clean browser state in such case if I’m not mistaken.

Got a solution here - https://github.com/protractor-cucumber-framework/protractor-cucumber-framework/pull/178 - works well for me using C6.