nightwatch: Failure on browser.assert not ending test runner on async/await tests

Failure on browser.assert not ending test runner on async/await tests

I’m doing nightwatch tests using async/await method. Having a browser.assert.elementPresent fail doesn’t seem to skip the rest of the tests properly and instead continues on the next step. end_session_on_fail nor skip_testcases_on_fail settings don’t seem to help even when set manually to true.

Sample test

module.exports = {
    "Assert error not ending test runner": async function (browser) {
        const modalCloseSelector = `.consent-form > .btn.primary`;
        await browser.url("https://yahoo.com");
        browser.assert.elementPresent(".this-doesnt-exist");
        await browser.waitForElementVisible(modalCloseSelector);
        await browser.click(modalCloseSelector);
    }
};

Run with command

$ ./node_modules/.bin/nightwatch

Verbose output

debug.log

❯ ./node_modules/.bin/nightwatch --verbose
 Starting ChromeDriver on port 9515...
 ChromeDriver up and running on port 9515 with pid: 11074 (130ms).

[Example] Test Suite
====================
⠋ Connecting to localhost on port 9515...
   Request POST  /session
   {
     desiredCapabilities: {
       browserName: 'chrome',
       chromeOptions: { w3c: false, args: [] },
       name: 'Example'
     }
⠹ Connecting to localhost on port 9515...
   Response 200 POST /session (1064ms)
   {
     sessionId: '6fc36777aa98163776bfb191f766ee20',
     status: 0,
     value: {
       acceptInsecureCerts: false,
       acceptSslCerts: false,
       applicationCacheEnabled: false,
       browserConnectionEnabled: false,
       browserName: 'chrome',
       chrome: {
         chromedriverVersion: '86.0.4240.22 (398b0743353ff36fb1b82468f63a3a93b4e2e89e-refs/branch-heads/4240@{#378})',
         userDataDir: '/var/folders/c2/138xwh354g5_qh1822c5bwlw0000gn/T/.com.google.Chrome.skjeSS'
       },
       cssSelectorsEnabled: true,
       databaseEnabled: false,
       'goog:chromeOptions': { debuggerAddress: 'localhost:60941' },
       handlesAlerts: true,
       hasTouchScreen: false,
       javascriptEnabled: true,
       locationContextEnabled: true,
       mobileEmulationEnabled: false,
       nativeEvents: true,
       networkConnectionEnabled: false,
       pageLoadStrategy: 'normal',
       platform: 'Mac OS X',
       proxy: {},
       rotatable: false,
       setWindowRect: true,
       strictFileInteractability: false,
       takesHeapSnapshot: true,
       takesScreenshot: true,
       timeouts: { implicit: 0, pageLoad: 300000, script: 30000 },
       unexpectedAlertBehaviour: 'ignore',
       version: '86.0.4240.111',
       webStorageEnabled: true,
       'webauthn:virtualAuthenticators': true
     }
ℹ Connected to localhost on port 9515 (1109ms).
  Using: chrome (86.0.4240.111) on Mac OS X platform.

 Received session with ID: 6fc36777aa98163776bfb191f766ee20

 → Running [before]:
 → Completed [before].
Running:  Assert error not ending test runner

 → Running [beforeEach]:
 → Completed [beforeEach].

 → Running command: url ('https://yahoo.com')
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/url
   { url: 'https://yahoo.com' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/url (1259ms)
   {
     sessionId: '6fc36777aa98163776bfb191f766ee20',
     status: 0,
     value: null
}
  → Completed command: url ('https://yahoo.com') (1261ms)

 → Running command: assert.elementPresent ('.this-doesnt-exist')

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (11ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (13ms)

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (7ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (8ms)

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (5ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (6ms)

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (7ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (8ms)

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (6ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (7ms)

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (7ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (8ms)

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (8ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (8ms)

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (7ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (8ms)

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (8ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (8ms)

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (5ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (5ms)

 → Running command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function])
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.this-doesnt-exist' }
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (7ms)
   { sessionId: '6fc36777aa98163776bfb191f766ee20', status: 0, value: [] }
✖ Testing if element <.this-doesnt-exist> is present in 5000ms - expected "is present" but got: "not present" (5138ms)
    at Object.Assert error not ending test runner (/Users/xxxx/tests/nightwatch-build/example.js:5:24)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

  → Completed command: elements ('css selector', {name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement}, [Function]) (10ms)

 → Running command: waitForElementVisible ('.consent-form > .btn.primary')
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.consent-form > .btn.primary' }
  → Completed command: assert.elementPresent ('.this-doesnt-exist') (5141ms)
 → Running [afterEach]:
 → Completed [afterEach].

FAILED: 1 assertions failed (6.408s)
 → Running [after]:
 → Completed [after].

 → Running command: end ()
 Failures in "Assert error not ending test runner". Taking screenshot...

 → Running command: saveScreenshot ('/Users/xxxx/e2e-screenshots/example/Assert-error-not-ending-test-runner_FAILED_Nov-03-2020-165419-GMT+0200-(Eastern-European-Standard.png', [Function])

 → Running command: screenshot (false, [Function])
   Request GET  /session/6fc36777aa98163776bfb191f766ee20/screenshot
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (17ms)
   {
     sessionId: '6fc36777aa98163776bfb191f766ee20',
     status: 0,
     value: [ { ELEMENT: '0.4633783548434449-1' } ]
}
   Request GET  /session/6fc36777aa98163776bfb191f766ee20/element/0.4633783548434449-1/displayed
   Response 200 GET /session/6fc36777aa98163776bfb191f766ee20/screenshot (304ms)
   {
     sessionId: '6fc36777aa98163776bfb191f766ee20',
     status: 0,
     value: '',
     suppressBase64Data: true
}
  → Completed command: screenshot (false, [Function]) (330ms)
   Response 200 GET /session/6fc36777aa98163776bfb191f766ee20/element/0.4633783548434449-1/displayed (319ms)
   {
     sessionId: '6fc36777aa98163776bfb191f766ee20',
     status: 0,
     value: true
}
✔ Element <.consent-form > .btn.primary> was visible after 340 milliseconds.
  → Completed command: waitForElementVisible ('.consent-form > .btn.primary') (341ms)

 → Running command: click ('.consent-form > .btn.primary')
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/elements
   { using: 'css selector', value: '.consent-form > .btn.primary' }
  → Completed command: saveScreenshot ('/Users/xxxx/e2e-screenshots/example/Assert-error-not-ending-test-runner_FAILED_Nov-03-2020-165419-GMT+0200-(Eastern-European-Standard.png', [Function]) (339ms)
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/elements (6ms)
   {
     sessionId: '6fc36777aa98163776bfb191f766ee20',
     status: 0,
     value: [ { ELEMENT: '0.4633783548434449-1' } ]
}
   Request POST  /session/6fc36777aa98163776bfb191f766ee20/element/0.4633783548434449-1/click
{}
   Response 200 POST /session/6fc36777aa98163776bfb191f766ee20/element/0.4633783548434449-1/click (1134ms)
   {
     sessionId: '6fc36777aa98163776bfb191f766ee20',
     status: 0,
     value: null
}
  → Completed command: click ('.consent-form > .btn.primary') (1141ms)

 → Running command: session ('delete', [Function])
   Request DELETE  /session/6fc36777aa98163776bfb191f766ee20
   Response 200 DELETE /session/6fc36777aa98163776bfb191f766ee20 (52ms)
   {
     sessionId: '6fc36777aa98163776bfb191f766ee20',
     status: 0,
     value: null
}
  → Completed command: end () (1531ms)
  → Completed command: session ('delete', [Function]) (53ms)
_________________________________________________

TEST FAILURE:  1 assertions failed, 1 passed (9.296s)

 ✖ example
 – Assert error not ending test runner (6.408s)
   Testing if element <.this-doesnt-exist> is present in 5000ms - expected "is present" but got: "not present" (5138ms)
       at Object.Assert error not ending test runner (/Users/xxxx/tests/nightwatch-build/example.js:5:24)
       at processTicksAndRejections (internal/process/task_queues.js:97:5)

 Wrote report file to: tests_output/CHROME_86.0.4240.111_Mac_OS_X_example.xml.
 Wrote log file to: /Users/xxxx/chromedriver.log.
 ChromeDriver process closed.

Configuration

nightwatch.json

    src_folders:  ["tests/nightwatch-build"],
    webdriver: {
        start_process: true,
        server_path: "node_modules/.bin/chromedriver",
        port: 9515,
    },
    test_settings: {
        default: {
            end_session_on_fail: true,
            skip_testcases_on_fail: true,
            globals: {
                waitForConditionTimeout: 10000,
            },
            screenshots : {
                enabled : true,
                on_failure : true,
                on_error : true,
                path : "e2e-screenshots/"
            },
            desiredCapabilities: {
                browserName: "chrome",
                chromeOptions: {
                    w3c: false,
                    args: chromeArgs
                }
            },
        }
    },

Your Environment

Executable Version
nightwatch --version 1.5.1
npm --version 6.14.4
node --version v12.18.0
Browser driver Version
chromedriver 86.0.0
OS Version
macOS Catalina 10.15.6

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 20 (4 by maintainers)

Commits related to this issue

Most upvoted comments

We’ll have a new version out very soon which we’ll address this issue. Thanks for your patience 😃

+5
beatfactor on May 28, 2021

I started to avoid async/awaits. Too many problems in NW with it. I changed some methods to chain api commands. Some are wrapped by browser.perform()

+1
Pieras2 on May 27, 2021

@samnit There was no workaround for this, besides not using async/await

+1
alechirsch on May 27, 2021
Read more comments on GitHub
← nightwatch: Errors when working with Firefox against selenium hub (only)
nightwatch: isVisible is calling getElementProperty on Safari, which is not supported for iOS →