CodeceptJS: Headless Chrome slows tests by 10x

What are you trying to achieve?

I am trying to run my tests in headless Chrome using the --headless argument. I expect the tests to run in approximately the same amount of time as the usual runs, as running a clean webdriverio script runs faster headless than in a normal browser window.

What do you get instead?

The tests run about ten times as slow, using 44 seconds instead of 4 seconds.

$ time $(npm bin)/codeceptjs run --verbose > headless-output.txt

real	0m44.973s
user	0m0.992s
sys	0m0.068s

$ nvim codecept.json # removing --headless option

developer at developer-XPS-13-9360 in ~/dev/dx-nuflo/tests/acceptance (acceptance-testing)
$ time $(npm bin)/codeceptjs run --verbose > output.txt

real	0m4.879s
user	0m0.768s
sys	0m0.088s

verbose output from headless run


CodeceptJS v0.6.3
Using test root "/home/developer/dev/dx-nuflo/tests/acceptance"

Top level-routes --
   [1] Starting recording promises
   Emitted | suite.before ([object Object])
   [1] Queued | hook WebDriverIO._beforeSuite()
 a page to add a new encounter should be rendered
 > [Session] Starting singleton browser session
   [2] Starting recording promises
   Emitted | test.before
   [2] Queued | hook WebDriverIO._before()
   Emitted | test.start ([object Object])
   Emitted | step.before (I am on page "/encounter/new")
   [2] Queued | amOnPage: "/encounter/new"
   Emitted | step.after (I am on page "/encounter/new")
   [2] Queued | return step result
   Emitted | step.before (I see "LOGG INN")
   [2] Queued | see: "LOGG INN"
   Emitted | step.after (I see "LOGG INN")
   [2] Queued | return step result
   [2] Queued | fire test.passed
   [2] Queued | finish test
   Emitted | step.start (I am on page "/encounter/new")
 • I am on page "/encounter/new"
   Emitted | step.start (I see "LOGG INN")
 • I see "LOGG INN"
   Emitted | test.passed ([object Object])
 ✓ OK in 22317ms

   Emitted | test.after
   [2] Queued | hook WebDriverIO._after()
 > [Session] cleaning cookies and localStorage
 fooScen
   [3] Starting recording promises
   Emitted | test.before
   [3] Queued | hook WebDriverIO._before()
   Emitted | test.start ([object Object])
   Emitted | step.before (I am on page "/encounter/new")
   [3] Queued | amOnPage: "/encounter/new"
   Emitted | step.after (I am on page "/encounter/new")
   [3] Queued | return step result
   Emitted | step.before (I see "FOOLOGG INN")
   [3] Queued | see: "FOOLOGG INN"
   Emitted | step.after (I see "FOOLOGG INN")
   [3] Queued | return step result
   [3] Queued | fire test.passed
   [3] Queued | finish test
   Emitted | step.start (I am on page "/encounter/new")
 • I am on page "/encounter/new"
   Emitted | step.start (I see "FOOLOGG INN")
 • I see "FOOLOGG INN"
   [3] Error | Error
   [3] Starting <teardown> session
   Emitted | test.failed ([object Object])
   [3] <teardown> Queued | hook WebDriverIO._failed()
   [3] <teardown> Queued | () => done(err)
   [3] <teardown> Stopping recording promises
 > Screenshot has been saved to /home/developer/dev/dx-nuflo/tests/acceptance/output/fooScen.failed.png
 ✖ FAILED in 11599ms

   Emitted | test.after
   [3] <teardown> Queued | hook WebDriverIO._after()
 > [Session] cleaning cookies and localStorage
   [4] Starting recording promises
   Emitted | suite.after ([object Object])
   [4] Queued | hook WebDriverIO._afterSuite()

-- FAILURES:

  1) Top level-routes: fooScen:

      expected web page to include "FOOLOGG INN"
      + expected - actual

      -Innlogging
      -Brukernavn
      -Passord
      -LOGG INN
      +FOOLOGG INN
      
  
  Scenario Steps:
  
  - I.see("FOOLOGG INN") at Test.Scenario (top-level-routes_test.js:18:5)
  - I.amOnPage("/encounter/new") at Test.Scenario (top-level-routes_test.js:14:5)
  
  Error
      at Object.<anonymous> (/home/developer/dev/dx-nuflo/node_modules/codeceptjs/lib/helper/WebDriverIO.js:1129:51)
      at Object.exec (/home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/helpers/safeExecute.js:28:24)
      at Object.resolve (/home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/webdriverio.js:189:29)
      at /home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/webdriverio.js:538:32
      at _fulfilled (/home/developer/dev/dx-nuflo/node_modules/q/q.js:854:54)
      at self.promiseDispatch.done (/home/developer/dev/dx-nuflo/node_modules/q/q.js:883:30)


  FAIL  | 1 passed, 1 failed   // 34s
   Emitted | global.result ([object Object])


real	0m44.973s
user	0m0.992s
sys	0m0.068s

verbose output from normal (non-headless) run

CodeceptJS v0.6.3
Using test root "/home/developer/dev/dx-nuflo/tests/acceptance"

Top level-routes --
   [1] Starting recording promises
   Emitted | suite.before ([object Object])
   [1] Queued | hook WebDriverIO._beforeSuite()
 a page to add a new encounter should be rendered
 > [Session] Starting singleton browser session
   [2] Starting recording promises
   Emitted | test.before
   [2] Queued | hook WebDriverIO._before()
   Emitted | test.start ([object Object])
   Emitted | step.before (I am on page "/encounter/new")
   [2] Queued | amOnPage: "/encounter/new"
   Emitted | step.after (I am on page "/encounter/new")
   [2] Queued | return step result
   Emitted | step.before (I see "LOGG INN")
   [2] Queued | see: "LOGG INN"
   Emitted | step.after (I see "LOGG INN")
   [2] Queued | return step result
   [2] Queued | fire test.passed
   [2] Queued | finish test
   Emitted | step.start (I am on page "/encounter/new")
 • I am on page "/encounter/new"
   Emitted | step.start (I see "LOGG INN")
 • I see "LOGG INN"
   Emitted | test.passed ([object Object])
 ✓ OK in 2748ms

   Emitted | test.after
   [2] Queued | hook WebDriverIO._after()
 > [Session] cleaning cookies and localStorage
 fooScen
   [3] Starting recording promises
   Emitted | test.before
   [3] Queued | hook WebDriverIO._before()
   Emitted | test.start ([object Object])
   Emitted | step.before (I am on page "/encounter/new")
   [3] Queued | amOnPage: "/encounter/new"
   Emitted | step.after (I am on page "/encounter/new")
   [3] Queued | return step result
   Emitted | step.before (I see "FOOLOGG INN")
   [3] Queued | see: "FOOLOGG INN"
   Emitted | step.after (I see "FOOLOGG INN")
   [3] Queued | return step result
   [3] Queued | fire test.passed
   [3] Queued | finish test
   Emitted | step.start (I am on page "/encounter/new")
 • I am on page "/encounter/new"
   Emitted | step.start (I see "FOOLOGG INN")
 • I see "FOOLOGG INN"
   [3] Error | Error
   [3] Starting <teardown> session
   Emitted | test.failed ([object Object])
   [3] <teardown> Queued | hook WebDriverIO._failed()
   [3] <teardown> Queued | () => done(err)
   [3] <teardown> Stopping recording promises
 > Screenshot has been saved to /home/developer/dev/dx-nuflo/tests/acceptance/output/fooScen.failed.png
 ✖ FAILED in 1299ms

   Emitted | test.after
   [3] <teardown> Queued | hook WebDriverIO._after()
 > [Session] cleaning cookies and localStorage
   [4] Starting recording promises
   Emitted | suite.after ([object Object])
   [4] Queued | hook WebDriverIO._afterSuite()

-- FAILURES:

  1) Top level-routes: fooScen:

      expected web page to include "FOOLOGG INN"
      + expected - actual

      -Innlogging
      -Brukernavn
      -Passord
      -LOGG INN
      +FOOLOGG INN
      
  
  Scenario Steps:
  
  - I.see("FOOLOGG INN") at Test.Scenario (top-level-routes_test.js:18:5)
  - I.amOnPage("/encounter/new") at Test.Scenario (top-level-routes_test.js:14:5)
  
  Error
      at Object.<anonymous> (/home/developer/dev/dx-nuflo/node_modules/codeceptjs/lib/helper/WebDriverIO.js:1129:51)
      at Object.exec (/home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/helpers/safeExecute.js:28:24)
      at Object.resolve (/home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/webdriverio.js:189:29)
      at /home/developer/dev/dx-nuflo/node_modules/webdriverio/build/lib/webdriverio.js:538:32
      at _fulfilled (/home/developer/dev/dx-nuflo/node_modules/q/q.js:854:54)
      at self.promiseDispatch.done (/home/developer/dev/dx-nuflo/node_modules/q/q.js:883:30)


  FAIL  | 1 passed, 1 failed   // 4s
   Emitted | global.result ([object Object])


real	0m4.879s
user	0m0.768s
sys	0m0.088s

Test source code


Feature('Top level-routes');

Scenario('a page to add a new encounter should be rendered', (I) => {
  I.amOnPage('/encounter/new');
  I.see('LOGG INN');
});

// just to see how a failing test looks
Scenario('fooScen', (I) => {
  I.amOnPage('/encounter/new');
  I.see('FOOLOGG INN');
});

Details

CodeceptJS version: 0.6.3
NodeJS Version: 8.1.2
Operating System: Ubuntu 16.04 LTS
WebDriverIO: 4.8.0
Configuration file:

This is my configuration:

{
  "tests": "./*_test.js",
  "timeout": 5000,
  "output": "./output",
  "helpers": {
    "WebDriverIO" : {
      "browser": "chrome",
      "desiredCapabilities": {
          "browserName": "chrome",
          "chromeOptions": {
              "args": [
                  "headless",
                  "disable-gpu"
              ]
          }
      },
      "windowSize": "320x568",
      "restart": false,
      "timeouts": {
        "script": 10000,
        "page load": 3000,
        "implicit" : 5000
      },
      "url": "http://localhost:23450"
    }
  },
  "include": {
    "I": "./steps_file.js"
  },
  "bootstrap": false,
  "mocha": {},
  "name": "acceptance tests",
  "windowSize": "320x568"
}

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 8
Comments: 28 (3 by maintainers)

Commits related to this issue

Improve headless Chrome performance And decrease single test timeout, optimistically. See https://github.com/Codeception/CodeceptJS/issues/561#issuecomment-373666779 — committed to Opetushallitus/omatsivut by timorantalaiho 6 years ago
Improve headless Chrome performance And decrease single test timeout, optimistically. See https://github.com/Codeception/CodeceptJS/issues/561#issuecomment-373666779 — committed to Opetushallitus/omatsivut by timorantalaiho 6 years ago
Add possible fix to chrome stucked Find this fix at https://github.com/codeceptjs/CodeceptJS/issues/561 — committed to il12/livekit-recorder by il12 2 years ago

Most upvoted comments

Try these two ChromeOptions:

options.addArguments("--proxy-server='direct://'");
options.addArguments("--proxy-bypass-list=*");

I dunno in wich forum it was but this did the trick for me

+81

oligee80 on Mar 16, 2018

Only thing that worked for me is adding UserAgent to the options: options.add_argument("user-agent=Chrome/110.0.5481.77")

mys1erious on Feb 13, 2023

I am having a similar issue on this one still. I tried with the following arguments
options.add_argument("--proxy-server='direct://'")
options.add_argument("--proxy-bypass-list=*")
options.add_argument("--proxy-server=")
I am still not seeing any speed up. 40 seconds to run in headless and 4 seconds to run in windowed mode. I am running ChromeDriver 75.0.3770.140.

Please let me know if anyone solves this one! Could really help me out. All I am doing is calling driver.get("https://www.google.com")

Ok so after looking into it a bit more it seems that avoiding using headless chrome is the solution for me. Instead I use the pyvirtualdisplay library and set its visibilty to false. This sped up my performance 10x. Here is the code:

from selenium import webdriver
from pyvirtualdisplay import Display
import time

start = time.time()
display = Display(visible=0, size=(800, 600))
display.start()
driver = webdriver.Chrome(
        "/home/george/Desktop/chromedriver")
driver.get("https://www.google.com")
driver.quit()
display.stop()
end = time.time()
print(end-start)

with output:

5.253071069717407

instead of:

42.1179986000061

GeorgeDubuque on Jul 21, 2019

in my case, add the following argument solve my problem chrome_options.add_argument('blink-settings=imagesEnabled=false')

zjhgx on Nov 24, 2020

I saw some people get this to work and others do not have any success with it:

chromeOptions.addArguments("--proxy-server='direct://'");
chromeOptions.addArguments("--proxy-bypass-list=*");

I did not have success at first ether, then I looked at my code and realized that I have called setChromeOptions() more than one time and wondered if options are getting over-written. I moved things around to make only one call to setChromeOptions() and it worked.

var $driver = require("selenium-webdriver");
let chrome = require("selenium-webdriver/chrome");

const chromeOptions = new chrome.Options();//.headless();
chromeOptions.addArguments("--headless"); 
chromeOptions.addArguments("--proxy-server='direct://'");
chromeOptions.addArguments("--proxy-bypass-list=*");
chromeOptions.addArguments("window-size=1024,1324");
chromeOptions.addArguments("--blink-settings=imagesEnabled=false"); 

var $browser = new $driver.Builder()
	.forBrowser("chrome")
	.setChromeOptions(chromeOptions) 
	.withCapabilities(capabilities)
	//.forBrowser('firefox')
	.build();

You can ignore the lines dealing with hiding images, windows size etc. if you choose.

wilsonweb on Nov 6, 2019

I’m seeing a similar issue using Puppeteer and Chromium (no WebDriver), the proxy arguments do not seem to make a difference in my case. This may be an issue related to Chromium.

macOS: 10.13.4 Node: 8.10.0 headless: true results in 7747.209ms headless: false results in 4775.771ms

Robula on May 17, 2018

@krrahulmanikanta It workes fine for me . But I am in Windows7 System.setProperty(“path to driver); ChromeOptions chromeOptions = new ChromeOptions(); chromeOptions.addArguments(”–headless"); chromeOptions.addArguments(“window-sized1200,600”); chromeOptions.addArguments(“–proxy-server=‘direct://’”); chromeOptions.addArguments(“–proxy-bypass-list=*”); ChromeDriver driver = new ChromeDriver(chromeOptions);

Resh5 on Apr 25, 2018