selenium: [šŸ› Bug]: "--headlessmode=new" with Chromedriver 128 in container results in SessionNotCreatedException

What happened?

Recently we upgraded our Jenkins to the latest version and most of our (Java based) Selenium tests started failing in their pipelines, SessionNotCreatedException caused by a timeout in org.openqa.selenium.remote.http.AddSeleniumUserAgent.

Upon further investigation we found that the following combination of circumstances causes consistent failure:

  • Use chromium 128 / chromedriver 128
  • Use the new headless mode as --headless=new
  • Run from a docker container

I’ve browsed the issues here to check if it’s been reported before and it looks similar to this issue, might be the same cause.

For now our workaround is to downgrade the chromium / chromedriver version our Jenkins runs with. We could also switch our tests to --headless=old but I see that as a fix of last resort. I’d much rather Selenium and new chromedriver versions work together out of the box, even in new headless mode.

How can we reproduce the issue?

See this repository for a minimal reproducing example. For instructions on how to reproduce the issue please see the readme.

Relevant log output

> Task :test

SeleniumTest > headlessNew() FAILED
    org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
    Host info: host: '83b1fb817f96', ip: '172.17.0.1'
        at app//org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:545)
        at app//org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:234)
        at app//org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:163)
        at app//org.openqa.selenium.chromium.ChromiumDriver.<init>(ChromiumDriver.java:114)
        at app//org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:88)
        at app//org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:83)
        at app//org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:72)
        at app//SeleniumTest.createChromeDriver(SeleniumTest.java:26)
        at app//SeleniumTest.headlessNew(SeleniumTest.java:9)

        Caused by:
        org.openqa.selenium.TimeoutException: java.util.concurrent.TimeoutException
        Build info: version: '4.23.0', revision: '4df0a231af'
        System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.0-18-amd64', java.version: '21.0.4'
        Driver info: driver.version: ChromeDriver
            at app//org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute0(JdkHttpClient.java:399)
            at app//org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
            at app//org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:55)
            at app//org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:355)
            at app//org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:89)
            at app//org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:75)
            at app//org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:61)
            at app//org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:162)
            at app//org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:216)
            at app//org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:174)
            at app//org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:527)
            ... 8 more

            Caused by:
            java.util.concurrent.TimeoutException
                at java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1960)
                at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2095)
                at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute0(JdkHttpClient.java:382)
                ... 18 more

SeleniumTest > headlessOld() STANDARD_ERROR
    Aug 29, 2024 12:25:03 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
    WARNING: Unable to find an exact match for CDP version 128, returning the closest version; found: 127; Please update to a Selenium version that supports CDP version 128

Gradle Test Executor 1 finished executing tests.

Operating System

Alpine 3.20, Debian 12

Selenium version

4.19.1, 4.23

What are the browser(s) and version(s) where you see this issue?

Chrome 128

What are the browser driver(s) and version(s) where you see this issue?

Chromedriver 128

Are you using Selenium Grid?

No

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 4
  • Comments: 29 (11 by maintainers)

Most upvoted comments

I’ve made some changes and now I am on a setup that seems to work consistently so far. I’ve changed:

  • Away from Chromium to Google Chrome (downloaded as debian package from here)
  • Away from Chromium-Chromedriver to Google’s Chromedriver (as fetched from the chromedriver for testing website).
  • Away from an alpine based image to an image based on ubuntu (was basically required for the first two changes)

With those changes made the selenium tests in the demo project now work again, seemingly consistently. I suspect the issue to be with chromium and / or chromium-chromedriver but that’s all I’ve got.

In case anyone wants to take a look at what the working project looks like I’ve created a separate branch. It’ll eventually go out of sync but I’m too tired to fix it today Edit: Maybe not a good solution but should stay in sync now.

@qaquest I can confirm that the issue exists with google chrome when using the selenium/node-chrome image. I’ve added a branch to my reproducer repository that consistently reproduces the issue for me.

Okay I did some more testing. The issue seems to occur only with selenium/node-chrome:4.23.0 or higher. On selenium/node-chrome:4.22.0 I don’t get the issue. I’ve checked the containers and it seems that 4.23.0 comes with Chrome 127 and 4.22.0 comes with Chrome 126. I’ve first seen these issues appear with Chromium 127, on Chromium 126 and lower everything works.

So it’s my assumption is that Chromium made some more breaking changes in their 127 version that don’t play well with Selenium, and the issue you’re seeing is probably not because of the latest amazonlinux version itself but rather that you end up running Selenium with some 127+ version of Chrome and those versions are just cursed.

PS: If someone finds a site that publishes the changelogs for chrome / chromium version I’d be thankful. All I’ve found was a blog post that basically said ā€œwe’re making everything better but btw we’re no loger posting changelogs, too much workā€. 🤬

Hi guys,

I faced this issue and after some tests with strace, it looks like chromium now tries to create a directory $HOME/.config/chromium/Crash Reports/ whereas the previous versions didn’t (at least it didn’t make it crash).

My test Dockerfile:

FROM debian:12-slim

RUN apt-get update -y && apt-get install -yq chromium-driver python3-pip strace
RUN pip3 install selenium==4.23.1 --break-system-packages

COPY test.py .

ENTRYPOINT ["python3", "test.py"]

test.py:

import re
import os
from selenium import webdriver

from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service

chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--headless')
chrome_options.add_argument('user-agent=test')

CHROMIUMDRIVER_PATH = os.environ.get('CHROMIUMDRIVER_PATH', "/usr/bin/chromedriver")

webdriver_service = Service(executable_path=CHROMIUMDRIVER_PATH)
driver = webdriver.Chrome(service=webdriver_service, options=chrome_options)

Test on read-only FS:

$ docker run -it --tmpfs "/tmp" --read-only plop bash
Traceback (most recent call last):
  File "//test.py", line 16, in <module>
    driver = webdriver.Chrome(service=webdriver_service, options=chrome_options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/chromium/webdriver.py", line 66, in __init__
    super().__init__(command_executor=executor, options=options)
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/remote/webdriver.py", line 212, in __init__
    self.start_session(capabilities)
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/remote/webdriver.py", line 299, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/remote/webdriver.py", line 354, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited normally.
  (session not created: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x56082bf3eb9e <unknown>
#1 0x56082bc74020 <unknown>
#2 0x56082bcac561 <unknown>
#3 0x56082bca71cd <unknown>
#4 0x56082bcf6202 <unknown>
#5 0x56082bcf5823 <unknown>
#6 0x56082bce8cf3 <unknown>
#7 0x56082bcb3b08 <unknown>
#8 0x56082bcb4abe <unknown>
#9 0x56082bf10986 <unknown>
#10 0x56082bf13cb7 <unknown>
#11 0x56082bf13778 <unknown>
#12 0x56082bf14145 <unknown>
#13 0x56082bf0125e <unknown>
#14 0x56082bf1448f <unknown>
#15 0x56082beeb2d5 <unknown>
#16 0x56082bf2e205 <unknown>
#17 0x56082bf2e3e6 <unknown>
#18 0x56082bf3d9ba <unknown>
#19 0x7f2942a70144 <unknown>

With .config mounted:

$ docker run -it --tmpfs "/tmp" --tmpfs "/root/.config" --read-only plop bash

Just mounting a tmp FS on $HOME/.config does the trick. I hope it can help to move forward on this subject.

I’ve finally found that one of my colleagues can run the tests and they consistently work for them as well. I’ll be digging more into that next week, hopefully I’ll be able to narrow down the exact causes of the error. I’ll let you know once I find out more