go: cmd/go: support GOPROXY fallback on unexpected errors
In #26334, particularly https://github.com/golang/go/issues/26334#issuecomment-472049820, we decided that the go command should only try the the next proxy in GOPROXY if it received a deliberate not found (404/410) response. This prevents unwanted leakage of private module paths in case a private proxy has an outage – if the go command tried the next proxy on 500s, it would leak the request to proxy.golang.org or any other public proxy in the chain.
For public proxies like proxy.golang.org, this argument doesn’t apply. Falling back from a public proxy, particularly to direct, should be much less risky. The only information leaked is that a particular IP address wants a public module, which must by definition have been public to be served by the public proxy in the first place. And that’s not a new risk – public proxies are free to serve a 404 whenever they want.
Therefore, allowing fallback on all errors would improve reliability of the ecosystem with only minimal costs. As a strawman, we could support | delimiters, which would be used like:
GOPROXY=goproxy.corp,proxy.golang.org|direct
meaning to require an affirmative response from goproxy.corp, then try proxy.golang.org and fall back to direct on any failure, expected or otherwise. Precisely, | after an entry means “if the prior entry fails in any way, continue to the next entry”.
The default value of GOPROXY would presumably then change to GOPROXY=proxy.golang.org|direct.
@FiloSottile just in case there are security implications, but I’m pretty sure sum.golang.org covers this same as it does anything else.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 3
- Comments: 17 (15 by maintainers)
Discussing with @FiloSottile, we realized that while step (2) would improve cmd/go’s reliability posture it would significantly weaken the security posture, because now cmd/go could be easily forced to invoke git and other VCS binaries, and the never-ending stream of git/other-VCS remote code execution vulnerabilities would be back in scope. Today the proxy completely removes that security problem from users in the default configuration. I need to think some more about this, but this might be a good enough reason not to take step (2) by default.
Of course, special casing
proxy.golang.orgwould have the same security cost as adding|and using it by default, without giving an option to opt-out, so it’s even worse in terms of vulnerability exposure.@marwan-at-work As I understand, the use case is more about CI systems that use the
gocommand in production. ChangingGOPROXYif there’s an outage onproxy.golang.orgmay be difficult or risky to do at scale. While adding a new fallback method here does increase complexity, I expect most people won’t have to know or think about it.Also, re fallback strategy, it is important to note that there are two steps here:
There are not changes in security posture from (1), just new possibilities (with security implications, but again, only if you use it).
The possible change in security possible is from (2). The argument is that reliability is more important than security here, because go.sum is providing security. (If it’s not, we should fix that!)