selenium: [🐛 Bug]: Java: Unable to modify request method or uri when using Filter

What happened?

I’m unable to change the request method or the URI. As far I know from the protocol, it should be possible: https://chromedevtools.github.io/devtools-protocol/tot/Fetch/#method-continueRequest But I’m not sure if Filter uses continueRequest under the hood.

Unfortunately creating a new request with other RequestMethod / URI does not work, as it seems the reference to the request being sent to you is used. I think next.execute(newReq) should work though.

How can we reproduce the issue?

package com.test

import org.openqa.selenium.remote.http.HttpResponse as SeleniumHttpResponse
import java.io.InputStream
import java.time.Duration
import java.util.function.Supplier
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.withTimeout
import org.openqa.selenium.chrome.ChromeDriver
import org.openqa.selenium.chrome.ChromeOptions
import org.openqa.selenium.devtools.NetworkInterceptor
import org.openqa.selenium.remote.http.Filter
import org.openqa.selenium.remote.http.HttpHandler

class Test() {
    fun run() {
        val options = ChromeOptions()
        val driver = ChromeDriver(options)
        driver.get("http://google.com/generate_204")

        val responseChannel = Channel<SeleniumHttpResponse>(1)
        val supplier = Supplier<InputStream>{ "hello world".byteInputStream() }
        val filter = Filter { next ->
            HttpHandler { req ->
                if (req.uri.contains("http://google.com/generate_204")) {
                    // can't do req.setMethod(HttpMethod.POST)
                    // can't do req.setUri("http://myotherurl")
                    // val newReq = HttpRequest(HttpMethod.POST, "http://myotherurl") // does not work - it seems like next.execute holds to the reference of the HttpHandler request and unless that reference is modified, it won't work
                    req.setHeader("Content-Type", "application/json")
                    req.setContent(supplier)
                    val response = next.execute(req)
                    runBlocking {
                        responseChannel.send(response)
                    }
                    response
                } else {
                    val response = next.execute(req)
                    response
                }
            }
        }
        val interceptor = NetworkInterceptor(driver, filter)

        driver.get("http://google.com/generate_204")

        val response = runBlocking {
            withTimeout(60000) {
                responseChannel.receive()
            }
        }
        println(response)
        interceptor.close()
        driver.close()
    }
}

Thanks a lot!



### Relevant log output

```shell
N/A

Operating System

Ubuntu

Selenium version

Java 4.13.0

What are the browser(s) and version(s) where you see this issue?

Chrome 117

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver 117

Are you using Selenium Grid?

No response

About this issue

Original URL
State: closed
Created 9 months ago
Comments: 19 (16 by maintainers)

Commits related to this issue

[java][cdp] Ensure request can be modified while intercepting Fixes #12930 — committed to aguspe/selenium by pujagani 8 months ago

Most upvoted comments

Turns out bazel did not sync changes made to v118 package locally and hence was not picking up the new changes 😓 I have a fix ready.

pujagani on Oct 17, 2023

I mean, what I have working in .NET is not working in Java, so I think it’s a bug. 😃

@p0deje didn’t we see this in Ruby back when? I don’t remember what ~we~ you had to do…

titusfortner on Oct 16, 2023

Thanks @pujagani. I ended up using the lower level API for my use case, here’s the code if anyone else needs it before it gets fixed or something:

import java.util.*
import org.openqa.selenium.devtools.v117.fetch.Fetch
import org.openqa.selenium.devtools.v117.fetch.model.HeaderEntry
import org.openqa.selenium.devtools.v117.fetch.model.RequestPattern
import org.openqa.selenium.chrome.ChromeDriver
import org.openqa.selenium.chrome.ChromeOptions

class Test {

    fun run() {
        val options = ChromeOptions()
        val driver = ChromeDriver(options)
        val devTools = driver.devTools
        devTools.createSession()

        devTools.send(
            Fetch.enable(
                Optional.of(listOf(RequestPattern(Optional.of("*generate_204*"), Optional.empty(), Optional.empty()))),
                Optional.of(false),
            ),
        )
        devTools.addListener(Fetch.requestPaused()) { handler ->
            if (handler.responseStatusCode.isEmpty) {
                if (handler.request.url == "http://google.com/generate_204") {
                    val headers = listOf(HeaderEntry("Content-Type", "application/json"), HeaderEntry("Access-Control-Allow-Origin", "*"))
                    devTools.send(
                        Fetch.continueRequest(
                            handler.requestId,
                            Optional.of("http://httpbin.org/headers"),
                            Optional.of("POST"),
                            Optional.of(Base64.getEncoder().encodeToString("""{"hello":"world!"}""".toByteArray())),
                            Optional.of(headers),
                            Optional.of(true),
                        ),
                    )
                } else {
                    devTools.send(Fetch.continueRequest(handler.requestId, Optional.of(handler.request.url), Optional.of(handler.request.method), handler.request.postData, Optional.of(handler.request.headers.map { HeaderEntry(it.key, it.value.toString()) }.toMutableList()), Optional.of(false)))
                }
            } else {
                val responseBody = devTools.send(Fetch.getResponseBody(handler.requestId))
                devTools.send(
                    Fetch.fulfillRequest(
                        handler.requestId,
                        handler.responseStatusCode.get(),
                        handler.responseHeaders,
                        Optional.empty(),
                        Optional.of(responseBody.body),
                        Optional.empty(),
                    ),
                )

                val decodedBody = Base64.getDecoder().decode(responseBody.body)
                // use body & response
            }
        }
        driver.get("http://google.com/generate_204")

        Thread.sleep(60000)

        driver.close()
    }
}

brenol on Oct 13, 2023