webdriverio: afterTest hook does not execute for tests that fail with timeOut errors

Environment (please complete the following information):

  • WebdriverIO version: 6.1.20
  • Mode: WDIO Testrunner
  • If WDIO Testrunner, running sync/async: sync
  • Node.js version: 12.18.1
  • NPM version: 6.14.5
  • Browser name and version: Chrome 83
  • Platform name and version: Ubuntu 18.04.4
  • Additional wdio packages used (if applicable): @wdio/spec reporter, @wdio/devtools-service, @wdio/mocha-framework,

Config of WebdriverIO

const debug = process.env.DEBUG
const defaultTimeout = 60000
const defaultCapabilities = [{ maxInstances: 5, browserName: 'chrome' }]
var slack = require('wdio-slack-service')
const video = require('wdio-video-reporter');
let tests = 0;

exports.config = {
    runner: 'local',
    specs: [
        './test/specs/**/*.js'
    ],
    maxInstances: debug ? 1 : 10,
    capabilities: debug ? [{ browserName: 'chrome' }] : defaultCapabilities,
    execArgv: debug ? ['--inspect'] : [],
    logLevel: 'trace',
    logLevels: {
        webdriver: 'info',
        '@wdio/applitools-service': 'silent',
        '@wdio/devtools-service': 'silent'
    },
    bail: 5,
    baseUrl: process.env.baseUrl,
    waitforTimeout: 60000,
    connectionRetryTimeout: 120000,
    connectionRetryCount: 3,
    specFileRetriesDeferred: false,
    services: ['chromedriver', 'devtools'],
    framework: 'mocha',
    reporters: [
        'spec', 
        ['allure', {
            outputDir: 'allure-results',
            disableWebdriverStepsReporting: true,
            disableWebdriverScreenshotsReporting: false,
        }]
    ],
    mochaOpts: {
        ui: 'bdd',
        timeout: debug ? (24 * 60 * 60 * 1000) : defaultTimeout
    },
    beforeTest: function (test, context) {
        browser.maximizeWindow()
    },
    afterTest: function(test, context, { error, result, duration, passed, retries }) {
        if (!passed) {
            browser.takeScreenshot();
        }
        tests++;
        console.log('----------------------------------------------')
        console.log(`No of tests: ${tests}`)
        console.log(`error is:\n${error}`)
    }
}

Describe the bug When any test fails with a timeOut error, code inside afterTest does not execute after that test. The hook does execute for tests that pass or fail with assertion errors.

To Reproduce

  1. Implement some code inside afterTest hook
tests++;
console.log('----------------------------------------------')
console.log(`No of tests: ${tests}`)
  1. Fail a test with timeOut error

Expected behavior The code inside afterTest hook should be executed after the test fails with timeOut error

Log https://gist.github.com/Harshad3009/fb0e465211ea5f48de92a483a157b315#file-logs-text

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 9
  • Comments: 20 (9 by maintainers)

Commits related to this issue

Most upvoted comments

Just curious if there are any updates on this? We are seeing the same issue in our test framework (‘afterTest’ hook gets skipped whenever a test times out). This is causing issues for us because we set a lot of important metadata in our afterTest hook, such as the video recording, etc.

@christian-bromann can you please let us know by when can we expect fix for this issue. Or an alternate solution to handle it for time being. Need to take screenshot on failure with latest V7 Webdriverio Appium framework. Ant help here will be appreciated

Aren’t the hooks async by default?

Sure, but if you do an async operation in there like saveScreenshot you have to define it as async function.

Ran into this but only if the timeout occurred close to the end of the test. The webdriver session would close before the hook finished execution (in my case it is just a save screenshot function). Oddly enough I got it to behave by declaring the whole hook async

async afterTest(test, context, { error, result, duration, passed, retries }) {
    if (!passed) {
      await browser.saveScreenshot(
        `${this.screenshotPath}/${test.title.replace(/ /g, '_').substring(0, 70)}` +
          `_` +
          `${(i += 1)}.png`,
      );
    }
  }

After a bit of reading up I’m not sure why this even does anything. Aren’t the hooks async by default?

can you please let us know by when can we expect fix for this issue.

@Sumit-Soman I think I made it very clear that help on this issue is very much appreciated. Don’t wait until this gets fixed by the core team because this will likely not happen anytime soon. Get involved in the project and give back by contributing a fix for this.