protractor: protractor hangs indefinitely on "[launcher] Running 1 instances of WebDriver" - when running with xvfb

At first glance this issue seems to be related with #1764 but I don’t see how a network issue can cause the hanging here (directConnect is set to true). So here is the setup:

protractor-config.js

'use strict';

/* globals jasmine, browser */
exports.config = {
    allScriptsTimeout: 300000, // 5 min

    // Spec patterns are relative to this directory.
    specs: [
        'protractor/*.js'
    ],

    capabilities: {
        'browserName': 'chrome',
        version: '',
        platform: 'ANY'
    },

    directConnect: true,

    baseUrl: 'http://localhost:3000/', // gets overwritten

    troubleshoot: true,

    framework: 'jasmine',
    onPrepare: function beforeProtractorRuns() {
        browser.driver.manage().window().setSize(1280, 1024);

        // Disable animations so e2e tests run more quickly
        var disableNgAnimate = function () {
            angular.module('disableNgAnimate', []).run([
                '$animate',
                function ($animate) {
                    $animate.enabled(false);
                }
            ]);
        };
        browser.addMockModule('disableNgAnimate', disableNgAnimate);

        // see: https://github.com/angular/protractor/issues/60
        require('jasmine-reporters');
        var capsPromise = browser.getCapabilities();
        capsPromise.then(function setOutputDir(caps) {
            var browserName = caps.caps_.browserName.toUpperCase();
            var browserVersion = caps.caps_.version;
            var filePrefix = browserName + '-' + browserVersion + '-';
            jasmine.getEnv().addReporter(
                new jasmine.JUnitXmlReporter(
                    'build/reports/testresults',
                    true, //consolidate: save nested describes within the same file as their parent
                    true, //useDotNotation: separate suite names with dots rather than spaces (ie "Class.init" not "Class init")
                    filePrefix
                )
            );
        });
    },

    jasmineNodeOpts: {
        isVerbose: true,
        showColors: true,
        includeStackTrace: true,
        defaultTimeoutInterval: 300000, // 5 min
    }
};

Here is an extract of the essential parts of the Jenkins Job console log:

⋮
13:28:22 DEBUG - Running with --troubleshoot
13:28:22 DEBUG - Protractor version: 2.1.0
13:28:22 DEBUG - Your base url for tests is http://localhost:57505/
13:28:22 Using ChromeDriver directly...
13:28:22 [launcher] Running 1 instances of WebDriver
13:28:22 DEBUG - WebDriver session successfully started with capabilities { caps_: 
13:28:22    { acceptSslCerts: true,
13:28:22      applicationCacheEnabled: false,
13:28:22      browserConnectionEnabled: false,
13:28:22      browserName: 'chrome',
13:28:22      chrome: { userDataDir: '/tmp/.com.google.Chrome.AuA2qr' },
13:28:22      cssSelectorsEnabled: true,
13:28:22      databaseEnabled: false,
13:28:22      handlesAlerts: true,
13:28:22      javascriptEnabled: true,
13:28:22      locationContextEnabled: true,
13:28:22      mobileEmulationEnabled: false,
13:28:22      nativeEvents: true,
13:28:22      platform: 'Linux',
13:28:22      rotatable: false,
13:28:22      takesHeapSnapshot: true,
13:28:22      takesScreenshot: true,
13:28:22      version: '44.0.2403.125',
13:28:22      webStorageEnabled: true } }
13:28:30 Testing foobar
13:28:30   Setup
13:28:30     should display foobar - pass
⋮                                                   <--- more describe-s and it-s
13:28:35 Finished in 12.616 seconds
13:28:35 2 tests, 9 assertions, 0 failures
13:28:35 
13:28:35 [launcher] 0 instance(s) of WebDriver still running
13:28:35 [launcher] chrome #1 passed
⋮                                                   <--- other tests scenarios also run successfully
13:28:47 DEBUG - Running with --troubleshoot
13:28:47 DEBUG - Protractor version: 2.1.0
13:28:47 DEBUG - Your base url for tests is http://localhost:34976/
13:28:47 Using ChromeDriver directly...
13:28:47 [launcher] Running 1 instances of WebDriver <--- hanging from here,
                                                          can't figure out why,
                                                          wish I had logs : [
15:05:23 Xvfb stopping                               <--- canceled build manually,
                                                          see time difference, no timeouts triggered,
                                                          neither from Protractor nor from WebDriver : [
15:05:23 Build has been canceled

Note that it is not always the same tests scenario which leads to the hanging. Same problems with node 0.10.33 as with 0.12.7. Using Ubuntu 12.04.5 LTS (GNU/Linux 3.13.0-37-generic x86_64).

$ file /opt/google/chrome/chrome
/opt/google/chrome/chrome: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.26, BuildID[sha1]=0xe4fc0257b42110fc269782274f1e774718135349, stripped

About this issue

  • Original URL
  • State: open
  • Created 9 years ago
  • Comments: 72 (12 by maintainers)

Commits related to this issue

Most upvoted comments

It looks like setting DBUS_SESSION_BUS_ADDRESS to /dev/null alone is sufficient to prevent chrome from deadlocking. I ran a whole bunch of iterations of my test suite, and not a single hang.

Wow! Thanks!

I set the environment variable with export DBUS_SESSION_BUS_ADDRESS=/dev/null and no longer have any hanging chromedriver processes!

I ran one file that had an empty it statement with 12 concurrent browser sessions, and looped it 6000 times. After almost 2 hours of continuously running, there was not a single hanging process.

[12:20:33] Protractor ran 6000 files (6000 specs) – 6000 passed [12:20:33] [12:20:33] Finished ‘protractor’ after 1.92 h

Great news for everyone in the thread: from version 59 Chrome will be supporting headless mode, so xvfb will not be needed anymore. Details are here: https://developers.google.com/web/updates/2017/04/headless-chrome

The solution of @jrharshath worked for me. Thank you very much!

I am getting similar issue where 1 instance of webdriver remains forever until I abort the test run in jenkins. Protractor hangs at
I/launcher - 1 instance(s) of WebDriver still running

Setup (protractor | headless chrome | docker | jenkins) protractor 5.3.2 chromedriverVersion: ‘2.40.565383’ chrome ‘version’ => ‘68.0.3440.75’ platform=Linux 3.10.0-862.3.2.el7.x86_64 x86_64)

Docker image : FROM node:9-stretch RUN apt-get update &&
DEBIAN_FRONTEND=“noninteractive”
apt-get install -y --no-install-recommends
chromium
dbus-x11
dumb-init
libgconf-2-4
openjdk-8-jre-headless
&& rm -rf /var/lib/apt/lists/* USER node ENV CHROME_BIN /usr/bin/chromium ENV DBUS_SESSION_BUS_ADDRESS /dev/null

protractor.conf.js directConnect: true, capabilities: { shardTestFiles: true, maxInstances: 6,

    'browserName': 'chrome',
    'chromeOptions': {
        args: ['no-sandbox','headless','disable-gpu','window-size=1366,768','enable-logging' ,'v=1','disable-dev-shm-usage'],
    }
},

Tried using : ENV DBUS_SESSION_BUS_ADDRESS /dev/null as suggested in docker file but error still exist.

We have the protractor hanging after

14:50:50 [14:50:48] I/launcher - Running 1 instances of WebDriver
14:50:50 [14:50:48] I/hosted - Using the selenium server at http://localhost:7840/wd/hub

even when using the DBUS_SESSION_BUS_ADDRESS workaround. It happens about 20-30% of the cases in our Jenkins builds (even on the very same Jenkins slave).

node.js 6.9.5 protractor 5.1.1 chromedriver 2.28 selenium 3.1.1 webdriver-manager 12.0.2 chrome 56.0.2924.87 (we have fixed these versions from constantly changing and potentially breaking things) Jenkins slaves are running on CentOS 7 The build does not use docker (yet).

protractor is started by a npm script (in package.json): "ci-e2e": "bash -c \"set -o pipefail ; export DBUS_SESSION_BUS_ADDRESS=/dev/null ; protractor protractor-conf.js --seleniumAddress http://localhost:${npm_package_config_seleniumPort}/wd/hub 2>&1 | tee .ci/protractor.log\"", (of course there are pre and post scripts as well to start/stop things)

chromedriver logs are enabled by the following shell script line (run by the pre npm script): nohup webdriver-manager start --versions.standalone $SELENIUM_STANDALONE_EXPECTED_VERSION --versions.chrome $SELENIUM_CHROMEDRIVER_EXPECTED_VERSION --chrome_logs ${THIS_SCRIPT_DIR}/../.ci/chromedriver.log --seleniumPort $SELENIUM_PORT >.ci/webdriver-manager_start.log 2>&1 & (there is a 5 seconds sleep after that)

Xvfb is started before webdriver in a similar style (nohup + background)

I just checked the chromedriver log and it shows:

[0.096][INFO]: COMMAND InitSession {
   "alwaysMatch": {
      "count": 1,
      "platform": "ANY",
      "version": "ANY"
   },
   "capabilities": {
      "desiredCapabilities": {
         "browserName": "chrome",
         "chromeOptions": {
            "args": [ "--no-sandbox", "--test-type=browser" ],
            "prefs": {
               "download": {
                  "default_directory": "/tmp/",
                  "prompt_for_download": false
               }
            }
         },
         "count": 1,
         "platform": "ANY",
         "version": "ANY"
      },
      "requiredCapabilities": {

      }
   },
   "desiredCapabilities": {
      "browserName": "chrome",
      "chromeOptions": {
         "args": [ "--no-sandbox", "--test-type=browser" ],
         "prefs": {
            "download": {
               "default_directory": "/tmp/",
               "prompt_for_download": false
            }
         }
      },
      "count": 1,
      "platform": "ANY",
      "version": "ANY"
   },
   "firstMatch": [ {
      "browserName": "chrome",
      "chromeOptions": {
         "args": [ "--no-sandbox", "--test-type=browser" ],
         "prefs": {
            "download": {
               "default_directory": "/tmp/",
               "prompt_for_download": false
            }
         }
      }
   } ],
   "requiredCapabilities": {

   }
}
[0.099][INFO]: Populating Preferences file: {
   "alternate_error_pages": {
      "enabled": false
   },
   "autofill": {
      "enabled": false
   },
   "browser": {
      "check_default_browser": false
   },
   "distribution": {
      "import_bookmarks": false,
      "import_history": false,
      "import_search_engine": false,
      "make_chrome_default_for_user": false,
      "show_welcome_page": false,
      "skip_first_run_ui": true
   },
   "dns_prefetching": {
      "enabled": false
   },
   "download": {
      "default_directory": "/tmp/",
      "prompt_for_download": false
   },
   "profile": {
      "content_settings": {
         "pattern_pairs": {
            "https://*,*": {
               "media-stream": {
                  "audio": "Default",
                  "video": "Default"
               }
            }
         }
      },
      "default_content_setting_values": {
         "geolocation": 1
      },
      "default_content_settings": {
         "geolocation": 1,
         "mouselock": 1,
         "notifications": 1,
         "popups": 1,
         "ppapi-broker": 1
      },
      "password_manager_enabled": false
   },
   "safebrowsing": {
      "enabled": false
   },
   "search": {
      "suggest_enabled": false
   },
   "translate": {
      "enabled": false
   }
}
[0.100][INFO]: Populating Local State file: {
   "background_mode": {
      "enabled": false
   },
   "ssl": {
      "rev_checking": {
         "enabled": false
      }
   }
}
[0.107][INFO]: Launching chrome: /opt/google/chrome/google-chrome --disable-background-networking --disable-client-side-phishing-detection --disable-default-apps --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --disable-web-resources --enable-automation --enable-logging --ignore-certificate-errors --load-extension=/tmp/.org.chromium.Chromium.ekVmsH/internal --log-level=0 --metrics-recording-only --no-first-run --no-sandbox --password-store=basic --remote-debugging-port=12521 --safebrowsing-disable-auto-update --test-type=browser --use-mock-keychain --user-data-dir=/tmp/.org.chromium.Chromium.6KPDEf data:,

Does anyone know how to track this down or how can I provide more debugging information (when it happens)? Is there some more specific bug report for this somewhere? Could updating any of the above SW components (e.g. chrome, chrome driver) help?

Guys & girls, seems like I have just found a workaroung, not very nice but at least working: webdriver-manager clean webdriver-manager update and you are ready to go!

Same issue when running parallel tests. Commenting onComplete solved it for us.

Any progress with this issue? I am using protractor 5.4.0 with “protractor-cucumber-framework”: “6.0.0”, and if some test fails in parallel mode, instance of webdriver is not shut down and hangs indefinitely.

If you’re running into problems with chrome occasionally hanging at startup, its probably a Chrome bug. See https://code.google.com/p/chromium/issues/detail?id=309093 where Google ran into this problem in their chrome testing setup, and worked around it in their test infrastructure.

Basically, chrome is mulit-threaded, and relies on some standard gconf libraries, one of those libraries does a fork+exec to start up “dbus” if it is not already running. Doing a fork+exec in a multi-threaded application is bad because you will occasionally fork while a different thread has the malloc lock (or other critical lock) and the child will then deadlock when it tries to acquire that lock, and everything grinds to a halt. Generally on a desktop, dbus is already running, but in many stripped-down test environments dbus does not get started.

Our work-around is to make sure dbus is running by having our scripts launch that XVFB run its child processes via dbus-launch --exit-with-session. (We also have a script that polls for X to be ready before proceeding, that seems to have helped, but we’re less confident its strictly necessary).

Here’s a lightly modified version of the xvfb wrapper script we use that starts xvfb, dbus, and waits for X to be ready: https://gist.github.com/tullmann/2d8d38444c5e81a41b6d

And here’s the waitForX script that depends on: https://gist.github.com/tullmann/476cc71169295d5c3fe6