selenium: [๐Ÿ› Bug]: Chrome 111 is not compatible with default HTTP Client

What happened?

After the Chrome 111 update, you can no longer kick off a chromedriver instance unless you add an additional chrome option:

โ€œโ€“remote-allow-origins=*โ€

I submitted a bug with Google and they responded, and indicated this has to do with Selenium setting the origin header:

https://bugs.chromium.org/p/chromium/issues/detail?id=1422444

How can we reproduce the issue?

Just update to chrome 111, and try and run a UI selenium test.

I am using Java and Junit.  See the link to the Chrome ticket for more details on how I was personally starting the driver.

Relevant log output

Mar 07, 2023 8:18:50 PM org.openqa.selenium.remote.http.WebSocket$Listener onError
WARNING: Invalid Status code=403 text=Forbidden
java.io.IOException: Invalid Status code=403 text=Forbidden
	at org.asynchttpclient.netty.handler.WebSocketHandler.abort(WebSocketHandler.java:92)
	at org.asynchttpclient.netty.handler.WebSocketHandler.handleRead(WebSocketHandler.java:118)
	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)


org.openqa.selenium.remote.http.ConnectionFailedException: Unable to establish websocket connection to http://localhost:64820/devtools/browser/27cc1ab2-7320-47bf-b503-1099b7052fcc
Build info: version: '4.4.0', revision: 'e5c75ed026a'
System info: host: 'sdg-mac334.local', ip: '2601:681:8004:57a0:0:0:0:5bfc%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '12.1', java.version: '17.0.4.1'
Driver info: driver.version: Driver

Operating System

Mac OS Monterey

Selenium version

4.4.0

What are the browser(s) and version(s) where you see this issue?

Chrome 111

What are the browser driver(s) and version(s) where you see this issue?

Iโ€™m using webdriver manager, so whichever the latest one its grabbing.

Are you using Selenium Grid?

No response

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 18
  • Comments: 60 (8 by maintainers)

Commits related to this issue

Most upvoted comments

Adding

   <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-http-jdk-client</artifactId>
        </dependency>

and

        System.setProperty("webdriver.http.factory", "jdk-http-client");

makes tests work again

For people searching for the fix; Here is the โ€œOfficialโ€ answer:

  1. If you use Java 11+ please update to the latest HTTP Client. The default client is required for backwards compatibility with Java 8, but has not been maintained and is not in as good of shape; the new client is better and will be what we use when we (eventually) remove support for Java 8.
<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-http-jdk-client</artifactId>
 </dependency>

and

System.setProperty("webdriver.http.factory", "jdk-http-client");
  1. If you do not use Java 11+, you temporarily need to add an argument to your ChromeOptions instance:
options.addArguments("--remote-allow-origins=*");
  1. If you are using Headless, note that Headless is Going Away. Please update your code to use:
options.addArguments("--headless=new");
  1. This issue will be closed when Selenium adds a flag to the Service arguments so that this flag is turned on by default. The next Selenium release should have the fix and make the other solutions unnecessary.

This will be released in a couple of days. In the meantime, the workaround described above is still valid.

How can i add this new arg for the chrome options??

Is it something like this?

ChromeOptions options = new ChromeOptions(); options.addArguments(โ€œโ€“remote-allow-origins=*โ€);

??

Thanks guys

The issue is in Java Selenium when using the Java 8 HTTP client. We are working on a fix as we speak and should be out with version 4.9.0 next week the latest. Titus comment above is on point and has all the information needed to go around this issue while the new version comes out.

Folks, please avoid spamming the issue, otherwise we might need to lock comments on it.

WebDriver driver = new ChromeDriver(options); // update this mentioned line

Ok, we could also add this to the Service class the same way we are doing for Firefox. That probably is the right thing to do if we continue to support Java < 11

@kmosakowska-clgx it worked for me with serenity <serenity.version>3.2.5</serenity.version>.

I think the issue might be related to https://github.com/netty/netty/issues/9673 which seems to be fixed in netty 5 (I believe Selenium is using netty 4 still)

reminder: Itโ€™s fixed in the latest version of Selenium

rodriguesxd7 remove Webdriver from LoginPage super constructor. That should be easy to spot using an IDE (intellij, vscode, etc).

public LoginPage() {
       super(driver);
   }

Does anyone know a solution for Selenide?

looks like itโ€™s already added in 6.12.2 with PR-2194

The code below worked for me with Maven Project, but I could not use this same code in Java Project. So if you want to try this please go with this.

	public void launchBrowser()
	{
		WebDriverManager.chromedriver().setup();
		
		ChromeOptions options = new ChromeOptions();
		options.addArguments("--remote-allow-origins=*");
		DesiredCapabilities cp=new DesiredCapabilities();

		cp.setCapability(ChromeOptions.CAPABILITY, options);

		options.merge(cp);

		WebDriver driver = new ChromeDriver(options);
		
	}

Configuration:

<!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
		<dependency>
			<groupId>org.seleniumhq.selenium</groupId>
			<artifactId>selenium-java</artifactId>
			<version>4.8.1</version>
		</dependency>

<!-- https://mvnrepository.com/artifact/org.testng/testng -->
		<dependency>
			<groupId>org.testng</groupId>
			<artifactId>testng</artifactId>
			<version>7.7.1</version>
			<scope>test</scope>
		</dependency>

<!-- https://mvnrepository.com/artifact/io.github.bonigarcia/webdrivermanager -->
		<dependency>
			<groupId>io.github.bonigarcia</groupId>
			<artifactId>webdrivermanager</artifactId>
			<version>5.3.1</version>
		</dependency>

@derRichter Its still not working for me i am adding dependencies here which i am using.

 <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <aspectj.version>1.9.8</aspectj.version>
        <allure-testng-version>2.19.0</allure-testng-version>
    </properties>

    <dependencies>
        <!-- https://mvnrepository.com/artifact/junit/junit -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13.2</version>
            <scope>test</scope>
        </dependency>


        <!-- https://mvnrepository.com/artifact/io.appium/java-client -->
        <dependency>
            <groupId>io.appium</groupId>
            <artifactId>java-client</artifactId>
            <version>8.3.0</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>4.8.1</version>
        </dependency>

        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.19.0</version>
        </dependency>

        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.19.0</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.testng/testng -->
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.3.0</version>
            <scope>test</scope>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-http-jdk-client -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-http-jdk-client</artifactId>
            <version>4.8.1</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/com.aventstack/extentreports -->
        <dependency>
            <groupId>com.aventstack</groupId>
            <artifactId>extentreports</artifactId>
            <version>5.0.9</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/io.github.bonigarcia/webdrivermanager -->
        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>5.3.2</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-chrome-driver -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-chrome-driver</artifactId>
            <version>4.8.1</version>
        </dependency>

        <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-testng</artifactId>
            <version>${allure-testng-version}</version>
            <scope>test</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.github.stephenc.monte/monte-screen-recorder -->
        <dependency>
            <groupId>com.github.stephenc.monte</groupId>
            <artifactId>monte-screen-recorder</artifactId>
            <version>0.7.7.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.codeborne/phantomjsdriver -->
        <dependency>
            <groupId>com.codeborne</groupId>
            <artifactId>phantomjsdriver</artifactId>
            <version>1.4.1</version>
        </dependency>

    </dependencies>
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>17</source>
                    <target>17</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M9</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>Executor.xml</suiteXmlFile>
                    </suiteXmlFiles>
                    <argLine>
                        -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                    </argLine>
                    <systemPropertyVariables>
                        <allure.results.directory>RER_AllureReporting/allure-results</allure.results.directory>
                    </systemPropertyVariables>
                </configuration>
                <dependencies>
                    <!-- https://mvnrepository.com/artifact/org.aspectj/aspectjweaver -->
                    <dependency>
                        <groupId>org.aspectj</groupId>
                        <artifactId>aspectjweaver</artifactId>
                        <version>${aspectj.version}</version>
                        <scope>runtime</scope>
                    </dependency>
                </dependencies>

After updating the dependencies to 4.8.1 i am still facing the issue in headless.

SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
I am in onStart method Regression Cases For Announcement & First Registration
Starting ChromeDriver 111.0.5563.64 (c710e93d5b63b7095afe8c2c17df34408078439d-refs/branch-heads/5563@{#995}) on port 43961
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Mar 13, 2023 12:42:55 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 111, so returning the closest version found: 110

java.lang.AssertionError: Issue in launching browser - no such window: target window already closed
from unknown error: web view not found
  (Session info: chrome=111.0.5563.64)
Build info: version: '4.8.1', revision: '8ebccac989'
System info: os.name: 'Mac OS X', os.arch: 'aarch64', os.version: '13.2', java.version: '17.0.6'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [f04ce54ee7e63c97726ab4891feaa12a, get {url=https://test-caseworker-portal.platform.rer.dev/}]
Capabilities {acceptInsecureCerts: true, browserName: chrome, browserVersion: 111.0.5563.64, chrome: {chromedriverVersion: 111.0.5563.64 (c710e93d5b63..., userDataDir: /var/folders/v8/mrd778lx16n...}, goog:chromeOptions: {debuggerAddress: localhost:51928}, networkConnectionEnabled: false, pageLoadStrategy: none, platformName: MAC, proxy: Proxy(), se:cdp: ws://localhost:51928/devtoo..., se:cdpVersion: 111.0.5563.64, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
Session ID: f04ce54ee7e63c97726ab4891feaa12a