selenium: [🐛 Bug]: Selenium Bidi with Chromedriver CDP Throws Exception

What happened?

Been trying to implement the examples for Bidi functionality in Selenium, rough code as shown in the how to reproduce the issue section.

Again, roughly, there’s some navigation occurring before I actually use the select there but it’s probably excessively verbose and specific to my org. And the methods are abstracted a bit but basically TestStep contains variables and methods to parse the arguments and capabilities for when the browser get created in setDriverChrome(path) and is used to get the driver object and set the page it navigates to. Underneath the driver is being assigned as a new RemoteWebDriver with the ChromeOptions passed in.

This is put together from two sources: https://www.selenium.dev/documentation/webdriver/bidirectional/chrome_devtools/bidi_api/ https://www.selenium.dev/documentation/webdriver/bidirectional/chrome_devtools/cdp_api/

The first is on the Mutation Observation for the Bidi API and the second goes over what’s needed to setup an event listener for CDP, since the two seemed related I combined them in troubleshooting but for reference the error below occurs without the second link code, so without the DevTools creation or session portion.

When I run this code the browser opens, navigates to the selected page, sets the Select value by id to a specified value attribute and then it should wait, print mutations counted over the sleep period, and end the session.

However, immediately after selecting the value for the select, many errors are logged in the console as shown below in relevant log output.

Note: This does NOT end the run, the test continues and sleeps, logs the output, and ends the session. I’m trying to figure out how I avoid the tens of thousands of lines of error text this generates in the console log every time it starts logging mutations after this point.

What is causing the the Connection Unable to process error at the start of the message, like I see that it’s eventually down to a ClosedChannelException, am I missing something here? Or should this be caught and maybe not logged excessively if it’s expected to occur when a event is being tracked/logged this way? Or is there something else underneath this I haven’t set yet that’s required for the Bidi functionality to work correctly? I’ve looked for similar errors but they’ve largely been unhelpful or unrelated entirely.

How can we reproduce the issue?

getTestStep().withBinaryPath("C:\\bin\\chrome\\chrome.exe")
.withBrowserArguments("--no-proxy-server", "--remote-debugging-port=9222")
.withBrowserCapability("pref:download.default_directory", System.getProperty(Common.LOCAL_PATH),					"webSocketUrl", true)
.setDriverChrome(getData(TestRun.OPTION_DRIVER_PATH));
DevTools dt = ((HasDevTools)getTestStep().getDriver()).getDevTools();
dt.createSession();
dt.send(Runtime.enable());
	getTestStep().setPage(getData(TestRun.OPTION_START_PAGE));

List<DomMutationEvent> mutations = Collections.synchronizedList(new ArrayList<>());
((HasLogEvents)getTestStep().getDriver()).onLogEvent(domMutation(mutation -> {
			mutations.add(mutation);
		}));
        
Select test = new Select(getTestStep().getDriver().findElement(By.id("price_fact.rate_type_code::0")));
test.selectByValue("FIXED");

Thread.sleep(1000);
System.out.println("Mutations: " + mutations.size());

getTestStep().getDriver().quit();

Relevant log output

[2024-01-24 00:07:44.143] - 25572 WARNING [CDP Connection] --- org.openqa.selenium.devtools.Connection: Unable to process: {"method":"Runtime.bindingCalled","params":{"name":"__webdriver_attribute","payload":"{\"target\":\"ii98x8xfi5elrrg0qif\",\"name\":\"disabled\",\"value\":\"disabled\",\"oldValue\":null}","executionContextId":8},"sessionId":"BEBBE32FED58DAF57378FFBD8EC15435"}
org.openqa.selenium.remote.UnreachableBrowserException: Error communicating with the remote browser. It may have died.
Build info: version: '4.16.1', revision: '9b4c83354e'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '17.0.2'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [4712cf89cde5020185336bd362837a82, findElements [using, value]]
Capabilities {acceptInsecureCerts: true, browserName: chrome, browserVersion: 119.0.6045.105, chrome: {chromedriverVersion: 119.0.6045.105 (38c72552c5e..., userDataDir: C:\Users\user\AppData\Lo...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:9222}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: windows, proxy: Proxy(), se:cdp: ws://localhost:9222/devtool..., se:cdpVersion: 119.0.6045.105, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 500, pageLoad: 60000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webSocketUrl: ws://localhost:9302/session..., webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
Session ID: 4712cf89cde5020185336bd362837a82
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:550)
	at org.openqa.selenium.remote.ElementLocation$ElementFinder$2.findElements(ElementLocation.java:182)
	at org.openqa.selenium.remote.ElementLocation.findElements(ElementLocation.java:103)
	at org.openqa.selenium.remote.RemoteWebDriver.findElements(RemoteWebDriver.java:373)
	at org.openqa.selenium.remote.RemoteWebDriver.findElements(RemoteWebDriver.java:367)
	at org.openqa.selenium.devtools.events.CdpEventTypes$2.lambda$initializeListener$0(CdpEventTypes.java:109)
	at org.openqa.selenium.devtools.idealized.Javascript.lambda$addBindingCalledListener$2(Javascript.java:95)
	at org.openqa.selenium.devtools.Connection.lambda$handle$5(Connection.java:345)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.stream.ReferencePipeline$15$1.accept(ReferencePipeline.java:541)
	at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1850)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.openqa.selenium.devtools.Connection.handle(Connection.java:308)
	at org.openqa.selenium.devtools.Connection$Listener.lambda$onText$0(Connection.java:236)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.io.UncheckedIOException: java.net.ConnectException
	at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute0(JdkHttpClient.java:442)
	at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
	at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:55)
	at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:358)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:188)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:200)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:175)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:523)
	... 22 more
Caused by: java.net.ConnectException
	at java.net.http/jdk.internal.net.http.common.Utils.toConnectException(Utils.java:1047)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:198)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.checkRetryConnect(PlainHttpConnection.java:230)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$1(PlainHttpConnection.java:206)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1773)
	... 3 more
Caused by: java.nio.channels.ClosedChannelException
	at java.base/sun.nio.ch.SocketChannelImpl.ensureOpen(SocketChannelImpl.java:195)
	at java.base/sun.nio.ch.SocketChannelImpl.beginConnect(SocketChannelImpl.java:760)
	at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:848)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$0(PlainHttpConnection.java:183)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:185)
	... 9 more

Operating System

Windows 10

Selenium version

4.16.1 and using Java 17

What are the browser(s) and version(s) where you see this issue?

Chrome for Test/Chrome v119.0.6045.105

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver v119.0.6045.105

Are you using Selenium Grid?

Not for this issue

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 23 (14 by maintainers)

Commits related to this issue

Most upvoted comments

Thank you, Titus! The issues shared are facing the same error. However, the fix I have will only fix things for this particular use case (domMutation listener). I am sensing there could be an underlying Java 11 HttpClient bug that needs further investigation.

@RhamaDaragaaz I appreciate the solution recommended. The solution I have is simple and uses a synchronized block. I have spent sufficient time eliminating other root causes that I thought were causing this error for this use case. Our long-term goal is to support the same functionality but using the W3C BiDi protocol and investing time in that space. So goal is to ensure this error does not occur but does not necessarily improve performance.

We have a couple other open issues referencing parser received no bytes; not sure if related, but wanted to link them since this seems like a recent thing.

https://github.com/SeleniumHQ/selenium/issues/13481 https://github.com/SeleniumHQ/selenium/issues/13536

You don’t want to mix methods from those pages.

We are working to switch the current implementation (Chrome DevTools Protocol) to the new WebDriver BiDi specification. The “BiDi API” is designed to be implementation agnostic, so whether it happens with CDP or BiDi spec it will just work. As such your this:

List<DomMutationEvent> mutations = Collections.synchronizedList(new ArrayList<>());
((HasLogEvents)getTestStep().getDriver()).onLogEvent(domMutation(mutation -> {
			mutations.add(mutation);
		}));

this is already starting the session and enabling the domains. Start by removing this piece:

DevTools dt = ((HasDevTools)getTestStep().getDriver()).getDevTools();
dt.createSession();
dt.send(Runtime.enable());
	getTestStep().setPage(getData(TestRun.OPTION_START_PAGE));

Second step would be to turn on logging to see what is happening when the error occurs: https://www.selenium.dev/documentation/webdriver/troubleshooting/logging/