selenium: [🐛 Bug]: Could not start a new session. Unable to parse remote response:

What happened?

I have no idea why is my bean is not there. I just upgraded from java 1.8 to java 17 and upgraded my selenium from 4.6.x to 4.16.x, and got this issue.

How can we reproduce the issue?

I have no idea why is my bean is not there. I just upgraded from java 1.8 to java 17 and upgraded my selenium from 4.6.x to 4.16.x, and got this issue.

    @WebdriverScopeBean
    @ConditionalOnMissingBean
    @ConditionalOnProperty(name = "browser", havingValue = "chrome")
    @Primary
    public WebDriver remoteChromeDriver() throws MalformedURLException {
        DesiredCapabilities caps = new DesiredCapabilities();

        caps.setCapability("browserName", "chrome");
        caps.setCapability("version", "");
        caps.setCapability("caseName", "testName");
        caps.setCapability("group", "lazada-enttech-private");
        caps.setCapability("f2etest.video", true);
        caps.setCapability("f2etest.userid", "natasha01467692");  // Put your userID here
        caps.setCapability("f2etest.apiKey", "a03a046e966a1d24dee3d29f819dfa89"); // Put your apiKey here
        caps.setCapability("f2etest.hosts", "f2etest.alibaba-inc.com");
//
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--incognito");
        options.setAcceptInsecureCerts(true);
        options.setPageLoadStrategy(PageLoadStrategy.EAGER);

        caps.setCapability(ChromeOptions.CAPABILITY, options);
        options.merge(caps);

        String hubAddress = String.format("http://f2etest.alibaba-inc.com/wd/hub");
        URL url = new URL(hubAddress);


        driver =  new RemoteWebDriver(url, options, false);

        System.out.println(driver + " >>> hello");


        return driver;
    }

BaseTest.class:

    @BeforeEach
    public void beforeEach(TestInfo testInfo) {
        System.out.println(this.ctx.containsBean(String.valueOf(WebDriver.class)) + "before each statement");
        webDriver = (WebDriver) this.ctx.getBean("remoteChromeDriver");
        System.out.println(webDriver + ">>> alibaba");
        this.testInfo = testInfo;
    }

Relevant log output

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'remoteChromeDriver' defined in class path resource [com/xxx/portal/configuration/RemoteWebDriverConfig.class]: Failed to instantiate [org.openqa.selenium.WebDriver]: Factory method 'remoteChromeDriver' threw exception with message: Could not start a new session. Unable to parse remote response: <h2>Internal Server Error, real status: 500</h2>

Operating System

macOS

Selenium version

Java 17

What are the browser(s) and version(s) where you see this issue?

Chrome 120

What are the browser driver(s) and version(s) where you see this issue?

Chrome 120

Are you using Selenium Grid?

4.16.1