go: net/http: *Request.URL.Scheme returns an empty string. No alternative way present to get request url scheme.

What version of Go are you using (go version)?

$ go version
go version go1.11 linux/amd64

Does this issue reproduce with the latest release?

Yes!

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN="/home/anix/go/bin"
GOCACHE="/home/anix/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/tmp/blackhole:/home/anix/gopath"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build796132705=/tmp/go-build -gno-record-gcc-switches"

What did you do?

package main

import (
        "fmt"
        "net/http"
)

func main() {
        fmt.Println("started!")
        http.HandleFunc("/", redr)
        http.ListenAndServe(":8080", nil)
}

func redr(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintln(w, "SCHEME:", r.URL.Scheme, "HOST:", r.Host, "PATH", r.URL.Path, )
}

What did you expect to see?

$ curl -i http://localhost:8080/
HTTP/1.1 200 OK
Date: Sun, 25 Nov 2018 07:05:42 GMT
Content-Length: 36
Content-Type: text/plain; charset=utf-8

SCHEME: http  HOST: localhost:8080 PATH /

What did you see instead?

The SCHEME: is empty!

$ curl -i http://localhost:8080/
HTTP/1.1 200 OK
Date: Sun, 25 Nov 2018 07:05:42 GMT
Content-Length: 36
Content-Type: text/plain; charset=utf-8

SCHEME:  HOST: localhost:8080 PATH /

Yes!

I read this part of godoc!

$ go doc http.Request.URL

type Request struct {
    // URL specifies either the URI being requested (for server
    // requests) or the URL to access (for client requests).
    //
    // For server requests the URL is parsed from the URI
    // supplied on the Request-Line as stored in RequestURI. For
    // most requests, fields other than Path and RawQuery will be
    // empty. (See RFC 7230, Section 5.3)
    //
    // For client requests, the URL's Host specifies the server to
    // connect to, while the Request's Host field optionally
    // specifies the Host header value to send in the HTTP
    // request.
    URL *url.URL

    // ... other fields elided ...
}

The most focused part is:

// For server requests the URL is parsed from the URI // supplied on the Request-Line as stored in RequestURI. For // most requests, fields other than Path and RawQuery will be // empty. (See RFC 7230, Section 5.3)

But there should be an alternative way to not get an empty string. As r.URL.Host has an alternative r.Host. But what about r.URL.Scheme?

No way!

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 5
  • Comments: 16 (7 by maintainers)

Most upvoted comments

Checking TLS is all well and good if the only schemes in existence were http and https, but the real world is far broader than that. I arrived here looking for the best way to check for a ws or wss scheme for proxying purposes. I’m sure others have use cases for other schemes as well.

I think checking this field for nil on http.Request is the closest you will get:

        // TLS allows HTTP servers and other software to record
        // information about the TLS connection on which the request
        // was received. This field is not filled in by ReadRequest.
        // The HTTP server in this package sets the field for
        // TLS-enabled connections before invoking a handler;
        // otherwise it leaves the field nil.
        // This field is ignored by the HTTP client.
        TLS *tls.ConnectionState

@ChrisSalisbury the scheme simply doesn’t exist as part of the HTTP protocol. It’s not in the request at all, there’s no way to populate the field. The scheme is used when a client interprets a URL for the client to decide which language to use when talking to the server, there’s no point where the client is mandate to tell the server what “language” is being used to talk to to it (there’s an assumption that if we are communicating at all, then we must already know what language we are talking).

The suggestions here focus on disambiguating http:// vs https://. How can we determine if an incoming request has e.g. a ws:// scheme?

Thanks.

Checking for a nil TLS field on an http.Request is the correct way to determine if a request is http. We likely will not provide an alternative method since it’s redundant to do so given the current solution.

The fundamental point here is that we simply do not know what the scheme is. Scheme is part of the URI spec. As unpalatable as it is, the http requests never gets the URL the client requested, it only gets the path (and, most of the time, the hostname), Consider the case of a simple TCP based SSL proxy. The connection comes in as https, ssl is stripped, and the connection is forwarded on to the non-ssl server. Which scheme do you want? The one the client plugged into their browser (we can never get that), “https”, because the original server saw a TLS connection, or “http”.