go: cmd/go: build cache does not check #included headers for changes

It (may) be nice to have a mode that invalidates all cgo but not proper go.

I think this is basically working as expected. If you change the underlying C code, or you change the compiler, then you have to rebuild with -a.

While true, there’s a hidden security subtlety here. Lets suppose there’s a crypto library called go-crypto, which internally wraps the c-crypto project (random names). The devs of c-crypto find a fatal flaw, fix it and notify go-crypto, who update their vendored C code and issue a new release too.

I - as a user of the go-crypto library - see this and do a go get -u to fetch the new code, sleeping easy that I’m all protected. Except Go didn’t bother to actually recompile anything because only the C code changes, so my binary is still vulnerable, even though I built it with the new code.

This same issue will happen arbitrarily high a dependency chain, where anyone forgetting to rebuild with -a could potentially be vulnerable.

Btw, I’m not saying I know how to fix this or whether it’s even fixable. I just wanted to add a bit of weight behind this issue.

I think so too, a couple of safeguards could save a lot of people a lot of time and confusion. If you change the underlying C code, or you change the compiler, then you have to rebuild with -a. This is going to cause confusing issues in practice.

After many people update the package code, they don’t even know that the cache of go1.10 caused the bug to not be fixed.

Actually, go test -a does not seem to be enough to get the cache up to date. Subsequent go test runs without -a still use some older cached version of the C code.

The !!! enter message comes from C. No code was changed between the two test runs. First result is up to date, the second result is some older version of the C code:

$ go test -run Test_get_process_stats -a
!!! enter333
[...]

$ go test -run Test_get_process_stats
!!! enterYYY
[...]

This seems to fix it, so probably better to use this instead of (in addition to?) the -a flag:

go clean -cache -testcache .

Ah, actually I see that there is already material included about GOCACHE and cgo interaction in the go tool docs

As I have trouble finding what you meant there, I’m gonna copy-paste it from https://golang.org/cmd/go/#hdr-Build_and_test_caching for future readers of this ticket:

However, the build cache does not detect changes to C libraries imported with cgo. If you have made changes to the C libraries on your system, you will need to clean the cache explicitly or else use the -a build flag (see ‘go help build’) to force rebuilding of packages that depend on the updated C libraries.

In other words, if you use Cgo, you MUST use go build -a and go test -a, otherwise you’ll never know what ended up in your binary, or what C code you were actually testing.

Ah, actually I see that there is already material included about GOCACHE and cgo interaction in the go tool docs; I’ll hold off until the 1.14 cycle is open to poke at fixing this.