go: net/http: DefaultServeMux incorrectly redirect (301) to path if path includes `%2F%2F`

This is clearly a bug.

The encoding method can encode arbitrary data in a Uniform Resource Identifier (URI), which is the point of the encoding.

“/” is the reserved character slash. “%2f” is an arbitrary string. “/” and “%2f” are two different things.

Is “%2f” is a slash? No, of course it is not a slash. and “%25” is not a percent sign Otherwise “https://www.google.com/” will be equivalent to “https:%252f%252525252fwww.google.com%2f”.

@tombergan @bradfitz

As a countermeasure against this similar problem, there is nginx merge_slashes syntax. Even with nginx, the above options are used for cases where you do not want to integrate encoded duplicate slashes into one.

And then, I have take a look at RFC-3986 for confirming the encoded double slashes validity. According to it, the encoded string pchar is considered valid.

   path          = path-abempty    ; begins with "/" or is empty
                 / path-absolute   ; begins with "/" but not "//"
                 / path-noscheme   ; begins with a non-colon segment
                 / path-rootless   ; begins with a segment
                 / path-empty      ; zero characters

   path-abempty  = *( "/" segment )
   path-absolute = "/" [ segment-nz *( "/" segment ) ]
   path-noscheme = segment-nz-nc *( "/" segment )
   path-rootless = segment-nz *( "/" segment )
   path-empty    = 0<pchar>

   segment       = *pchar
   segment-nz    = 1*pchar
   segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" )
                 ; non-zero-length segment without any colon ":"

   pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"

   query         = *( pchar / "/" / "?" )

   fragment      = *( pchar / "/" / "?" )

   pct-encoded   = "%" HEXDIG HEXDIG

Therefore, I think, if we follow RFC-3986, the current behavior is buggy.

Just faced this issue in the latest go 1.18.3

curl http://localhost/%2Fhello is decoded internally to //hello (r.URL.Path) and the decision for a 301 redirect based on r.URL.Path instead of r.URL.RawPath.

A dirty hack that fixed useless 301 redirect issue for me:

--- go/src/net/http/server.go   2022-06-01 18:44:51.000000000 +0200
+++ go_/src/net/http/server.go  2022-06-15 23:44:02.602151436 +0200
@@ -2412,7 +2412,7 @@
        // All other requests have any port stripped and path cleaned
        // before passing to mux.handler.
        host := stripHostPort(r.Host)
-       path := cleanPath(r.URL.Path)
+       path := cleanPath(r.URL.RawPath)
 
        // If the given path is /tree and its handler is not registered,
        // redirect for /tree/.
@@ -2420,7 +2420,7 @@
                return RedirectHandler(u.String(), StatusMovedPermanently), u.Path
        }
 
-       if path != r.URL.Path {
+       if path != r.URL.RawPath {
                _, pattern = mux.handler(host, path)
                u := &url.URL{Path: path, RawQuery: r.URL.RawQuery}
                return RedirectHandler(u.String(), StatusMovedPermanently), pattern

To expand on @tombergan’s comment (https://github.com/golang/go/issues/21955#issuecomment-379057478), the ServeMux documentation states:

ServeMux also takes care of sanitizing the URL request path and the Host header, stripping the port number and redirecting any request containing . or … elements or repeated slashes to an equivalent, cleaner URL.

The documentation is not explicit on whether %2f is considered a slash, but the implementation consistently treats it as such. ServeMux.Handle("/foo/", f) matches the request /foo%2fbar.

I believe the current behavior is consistent with the documentation and probably can’t be changed without violating compatibility. Perhaps the documentation should explicitly mention the treatment of escaped path components.

Redirecting http://localhost:8878// to http://localhost:8878/ is correct. But redirecting http://localhost:8878/%2F to http://localhost:8878/ is not correct. I think this is a bug to be fixed.