webdriverio: [🐛 Bug]: if session initialisation failed, the result of the test is not be part of the reporter

Have you read the Contributing Guidelines on issues?

I have read the Contributing Guidelines on issues.

WebdriverIO Version

7.16.11

Node.js Version

Mode

WDIO Testrunner

Which capabilities are you using?

No response

What happened?

We are running more than 100 specs in an execution, with 68 instances, via saucelabs. For an unknown reason, some tests at the beginning fail with this error:

[0-25] 2022-05-05T12:28:36.873Z ERROR webdriver: MaxRedirectsError: Redirected 10 times. Aborting.
[0-25]     at Request._onResponseBase (/usr/src/app/node_modules/got/dist/source/core/index.js:859:35)
[0-25]     at Request._onResponse (/usr/src/app/node_modules/got/dist/source/core/index.js:940:24)
[0-25]     at ClientRequest.<anonymous> (/usr/src/app/node_modules/got/dist/source/core/index.js:954:23)
[0-25]     at Object.onceWrapper (events.js:520:26)
[0-25]     at ClientRequest.emit (events.js:412:35)
[0-25]     at ClientRequest.emit (domain.js:475:12)
[0-25]     at ClientRequest.origin.emit (/usr/src/app/node_modules/@szmarczak/http-timer/dist/source/index.js:43:20)
[0-25]     at HTTPParser.parserOnIncomingClient (_http_client.js:647:27)
[0-25]     at HTTPParser.parserOnHeadersComplete (_http_common.js:127:17)
[0-25]     at TLSSocket.socketOnData (_http_client.js:515:22)
[0-25] 2022-05-05T12:28:36.893Z ERROR @wdio/runner: Error: Failed to create session.
[0-25] Redirected 10 times. Aborting.
[0-25]     at startWebDriverSession (/usr/src/app/node_modules/webdriver/build/utils.js:72:15)
[0-25]     at processTicksAndRejections (internal/process/task_queues.js:95:5)
[0-25]     at Function.newSession (/usr/src/app/node_modules/webdriver/build/index.js:58:45)
[0-25]     at remote (/usr/src/app/node_modules/webdriverio/build/index.js:73:22)
[0-25]     at Runner._startSession (/usr/src/app/node_modules/@wdio/runner/build/index.js:223:56)
[0-25]     at Runner._initSession (/usr/src/app/node_modules/@wdio/runner/build/index.js:176:25)
[0-25]     at Runner.run (/usr/src/app/node_modules/@wdio/runner/build/index.js:88:19)
[0-25] FAILED in chrome - /e2e/specs/STS/STS_DataRestrictions/TC_DataRestrictionsList.spec.js

So, the tests which are getting this error are “failed”. But our specific reporter (the same with all reporters like spec, allure) doesn’t show the failed tests.

If I understand correctly, it’s because the runnerStat is undefined : the runner:end is called in the _shutdown method here and the runner:start where we define the runnerStat has never been called at this step.

To sum up, if session initialisation failed, the result of the test will not be part of the reporter.

What is your expected behavior?

if session initialisation failed, the result of the test should be part of the reporter and should be failed.

How to reproduce the bug.

You can do: In ‘node_modules\webdriver\build\utils.js’, replace (in the try/catch) the line 67 by:

throw new Error('Failed to create session.\n');

Relevant log output

not relevant

Code of Conduct

I agree to follow this project’s Code of Conduct

Is there an existing issue for this?

I have searched the existing issues

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 16 (16 by maintainers)

Most upvoted comments

@NIBOSAP thanks for confirming. I was mistaking then.

christian-bromann on Sep 5, 2022

The was merged and released. Thanks for the contribution!

christian-bromann on Aug 26, 2022

@christian-bromann In fact, nothing changes for the spec-reporter, with or without the fix the log will be:

The only difference is: in this case of failure, the onRunnerEnd will be called in the reporter.

That means it depends on what each reporter does in the onRunnerEnd. So, this fix can cause breaking change.

spec-reporter: nothing change
allure-reporter: nothing change as the onRunnerEnd is not implemented (cf @wyhmichael )
dot-reporter: nothing change as the onRunnerEnd is not implemented
concise-reporter: Before the fix, the concise reporter was not called, so nothing showed up. Now, with the fix:

junit-reporter: Before the fix, the junit reporter was not called, so nothing showed up. Now, with the fix:

json-reporter : Before the fix, the json reporter was not called, so nothing showed up. Now with the fix, an error will happen (cf @wyhmichael )

A better way could be to add another property to the runnerStat like: initialisationFailed. It doesn’t solve the potential breaking change, but it helps to handle it properly in the reporters side.

NIBOSAP on May 18, 2022

Hi @erwinheitzman , Many thanks to have a look ! The correct total amount cames from the cli: https://github.com/webdriverio/webdriverio/blob/main/packages/wdio-cli/src/interface.ts#L298 Not from the spec reporter. I think that above this line in the console you don’t see the test in failure.

NIBOSAP on May 6, 2022