webdriverio: afterTest hook does not execute for tests that fail with timeOut errors
Environment (please complete the following information):
- WebdriverIO version: 6.1.20
- Mode: WDIO Testrunner
- If WDIO Testrunner, running sync/async: sync
- Node.js version: 12.18.1
- NPM version: 6.14.5
- Browser name and version: Chrome 83
- Platform name and version: Ubuntu 18.04.4
- Additional wdio packages used (if applicable): @wdio/spec reporter, @wdio/devtools-service, @wdio/mocha-framework,
Config of WebdriverIO
const debug = process.env.DEBUG
const defaultTimeout = 60000
const defaultCapabilities = [{ maxInstances: 5, browserName: 'chrome' }]
var slack = require('wdio-slack-service')
const video = require('wdio-video-reporter');
let tests = 0;
exports.config = {
runner: 'local',
specs: [
'./test/specs/**/*.js'
],
maxInstances: debug ? 1 : 10,
capabilities: debug ? [{ browserName: 'chrome' }] : defaultCapabilities,
execArgv: debug ? ['--inspect'] : [],
logLevel: 'trace',
logLevels: {
webdriver: 'info',
'@wdio/applitools-service': 'silent',
'@wdio/devtools-service': 'silent'
},
bail: 5,
baseUrl: process.env.baseUrl,
waitforTimeout: 60000,
connectionRetryTimeout: 120000,
connectionRetryCount: 3,
specFileRetriesDeferred: false,
services: ['chromedriver', 'devtools'],
framework: 'mocha',
reporters: [
'spec',
['allure', {
outputDir: 'allure-results',
disableWebdriverStepsReporting: true,
disableWebdriverScreenshotsReporting: false,
}]
],
mochaOpts: {
ui: 'bdd',
timeout: debug ? (24 * 60 * 60 * 1000) : defaultTimeout
},
beforeTest: function (test, context) {
browser.maximizeWindow()
},
afterTest: function(test, context, { error, result, duration, passed, retries }) {
if (!passed) {
browser.takeScreenshot();
}
tests++;
console.log('----------------------------------------------')
console.log(`No of tests: ${tests}`)
console.log(`error is:\n${error}`)
}
}
Describe the bug When any test fails with a timeOut error, code inside afterTest does not execute after that test. The hook does execute for tests that pass or fail with assertion errors.
To Reproduce
- Implement some code inside afterTest hook
tests++;
console.log('----------------------------------------------')
console.log(`No of tests: ${tests}`)
- Fail a test with timeOut error
Expected behavior The code inside afterTest hook should be executed after the test fails with timeOut error
Log https://gist.github.com/Harshad3009/fb0e465211ea5f48de92a483a157b315#file-logs-text
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 9
- Comments: 20 (9 by maintainers)
Commits related to this issue
- (docs): saveScreenshot requires afterTest hook to be async When requesting a screenshot in `afterTest` I would sometime lack a screenshot file. I eventually found `browser.saveScreenshot` is async a... — committed to hashar/webdriverio by hashar 10 months ago
- wdio-mediawiki: await saveScreenshot On a test failure, I noticed it lacked the final screenshot as if the content of our `afterTest` hook did not run. `browser.saveScreenshot` is asynchronous and I ... — committed to wikimedia/mediawiki by hashar 10 months ago
- (docs): saveScreenshot requires afterTest hook to be async When requesting a screenshot in the `afterTest` hook, I would sometime not have any screenshot generated. I eventually found `browser.saveSc... — committed to hashar/webdriverio by hashar 10 months ago
- wdio-mediawiki: await saveScreenshot On a test failure, I noticed it lacked the final screenshot as if the content of our `afterTest` hook did not run. `browser.saveScreenshot` is asynchronous and I ... — committed to wikimedia/mediawiki by hashar 10 months ago
- wdio-mediawiki: await saveScreenshot On a test failure, I noticed it lacked the final screenshot as if the content of our `afterTest` hook did not run. `browser.saveScreenshot` is asynchronous and I ... — committed to wikimedia/mediawiki by hashar 10 months ago
- (docs): saveScreenshot requires afterTest hook to be async (#11005) When requesting a screenshot in the `afterTest` hook, I would sometime not have any screenshot generated. I eventually found `bro... — committed to webdriverio/webdriverio by hashar 10 months ago
Just curious if there are any updates on this? We are seeing the same issue in our test framework (‘afterTest’ hook gets skipped whenever a test times out). This is causing issues for us because we set a lot of important metadata in our afterTest hook, such as the video recording, etc.
@christian-bromann can you please let us know by when can we expect fix for this issue. Or an alternate solution to handle it for time being. Need to take screenshot on failure with latest V7 Webdriverio Appium framework. Ant help here will be appreciated
Sure, but if you do an async operation in there like
saveScreenshotyou have to define it asasyncfunction.Ran into this but only if the timeout occurred close to the end of the test. The webdriver session would close before the hook finished execution (in my case it is just a save screenshot function). Oddly enough I got it to behave by declaring the whole hook async
After a bit of reading up I’m not sure why this even does anything. Aren’t the hooks async by default?
@Sumit-Soman I think I made it very clear that help on this issue is very much appreciated. Don’t wait until this gets fixed by the core team because this will likely not happen anytime soon. Get involved in the project and give back by contributing a fix for this.
The correct link is https://github.com/webdriverio/webdriverio/blob/master/packages/wdio-utils/src/test-framework/testFnWrapper.ts