webdriverio: Error message ' Error in "" ' is logging in case of using jasmineNodeOpts setting failFast

Environment (please complete the following information):

  • WebdriverIO version: [5.22.4]
  • Mode: [Standalone mode]
  • If WDIO Testrunner, running sync/async: [sync]
  • Node.js version: [10.19.0]
  • NPM version: [6.14.4]
  • Browser name and version: [Chrome 80.0.3987.163]
  • Platform name and version: [Mac OS X]
  • Additional wdio packages used (if applicable):
 "devDependencies": {
    "@wdio/allure-reporter": "^5.22.4",
    "@wdio/browserstack-service": "^5.22.1",
    "@wdio/cli": "^5.22.4",
    "@wdio/jasmine-framework": "^5.18.6",
    "@wdio/local-runner": "^5.22.4",
    "@wdio/selenium-standalone-service": "^5.16.10",
    "@wdio/spec-reporter": "^5.22.4",
    "@wdio/sync": "^5.18.7",
    "webdriverio": "5.22.4",
},

Config of WebdriverIO

exports.config = {
  bail: 0,
  framework: 'jasmine',
  jasmineNodeOpts: {
    defaultTimeoutInterval: 100000,
    failFast: true,
    stopSpecOnExpectationFailure: true,
    grep: null,
    invertGrep: null,
  },
};

Describe the bug Extra message is logging in case jasmineNodeOpts contains the setting failFast: true:
Error in "<unknown test>"

To Reproduce

describe('My test: describe', () => {
  beforeAll(() => {
    console.log('beforeAll');
    throw new Error('before all error');
  });
  beforeEach(() => {
    console.log('beforeEach');
  });
  it('My test: it #1', () => {
    console.log('My test: it #1');
  });
  it('My test: it #2', () => {
    console.log('My test: it #2');
  });
  afterEach(() => {
    console.log('afterEach');
  });
  afterAll(() => {
    console.log('afterAll');
  });
});

Expected behavior As the setting failFast: true interrupts executing of all hooks in the spec, any additional messages should not be logged after failure appears Exception: afterAll() hook after jasmine issue 1533 will be implemented

Log

Execution of 1 spec files started at 2020-04-09T12:32:37.778Z

[0-0] RUNNING in chrome - /my-test.js
[0-0] beforeAll
[0-0] Error in ""before all" hook"
before all error
[0-0] Error in "<unknown test>"
Error: before all error
[0-0] FAILED in chrome -/my-test.js

 "spec" Reporter:
------------------------------------------------------------------
[chrome 80.0.3987.163 Mac OS X #0-0] Spec: /my-test.js
[chrome 80.0.3987.163 Mac OS X #0-0] Running: chrome (v80.0.3987.163) on Mac OS X
[chrome 80.0.3987.163 Mac OS X #0-0] Session ID: e3b43cc63cfe305a340a5fa7047fdbe5
[chrome 80.0.3987.163 Mac OS X #0-0]
[chrome 80.0.3987.163 Mac OS X #0-0] My test: describe
[chrome 80.0.3987.163 Mac OS X #0-0]    ✖ "before all" hook
[chrome 80.0.3987.163 Mac OS X #0-0]    ✖ <unknown test>
[chrome 80.0.3987.163 Mac OS X #0-0]
[chrome 80.0.3987.163 Mac OS X #0-0] 2 failing (4.4s)
[chrome 80.0.3987.163 Mac OS X #0-0]
[chrome 80.0.3987.163 Mac OS X #0-0] 1) My test: describe "before all" hook
[chrome 80.0.3987.163 Mac OS X #0-0] before all error
[chrome 80.0.3987.163 Mac OS X #0-0]
[chrome 80.0.3987.163 Mac OS X #0-0] 2) My test: describe <unknown test>
[chrome 80.0.3987.163 Mac OS X #0-0] Error: before all error
[chrome 80.0.3987.163 Mac OS X #0-0] Error: before all error
[chrome 80.0.3987.163 Mac OS X #0-0]     at UserContext.<anonymous> (/my-test.js:6:11)

Spec Files:      0 passed, 1 failed, 1 total (100% completed) in 00:00:09 

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 19 (12 by maintainers)

Most upvoted comments

@sushantsoni5392 I took a look into this and it seems this is what WebdriverIO gets from Jasmine. The Jasmine reporter returns:

{
  id: 'spec0.8036648635101069',
  description: '<unknown test>',
  fullName: '<unknown test>',
  failedExpectations: [
    {
      matcherName: '',
      message: 'Error: My error',
      stack: '    at UserContext.<anonymous> (file:///private/tmp/jasmine-boilerplate/test/specs/checkbox.spec.ts:6:15)\n' +
        '    at processTicksAndRejections (node:internal/process/task_queues:95:5)',
      passed: false,
      expected: '',
      actual: ''
    }
  ],
  deprecationWarnings: [],
  passedExpectations: [],
  status: 'failed',
  duration: null,
  properties: {},
  pendingReason: '',
  debugLogs: null
}

which is why WebdriverIO propagates this to the reporter. I don’t think there is anything we can do. While this experience is definitely not ideal I will go ahead and close this as this should be fixed within Jasmine. You might want to give Mocha a try as it could provide better reporting and is similar to Jasmine.

Hi! We have rewritten the reporter to the new allure-js version. Please try version 8.5.0 of wdio-allure-reporter and provide more details if the issue still persists. Feel free to create a new issue with actual details if it reproduces.