go: cmd/go: go mod tidy error when importing std package in later go version

for now you can run go mod tidy -e using Go 1.16.

The snag here being that go mod tidy -e is required for Go 1.15X 😃

Edit: the context for me saying that is the following example:

exec go mod tidy
-- go.mod --
module mod.com
go 1.15
require golang.org/x/tools v0.1.1-0.20210219012152-f3748ed8ec89
-- main.go --
package main
import (
	_ "golang.org/x/tools/go/analysis/passes/buildtag"
)

Works fine with go1.16, but fails with go1.15.8:

> exec go mod tidy
[stderr]
mod.com imports
        golang.org/x/tools/go/analysis/passes/buildtag imports
        go/build/constraint: package go/build/constraint is not in GOROOT (/home/myitcv/gos/src/go/build/constraint)

This now affects golang.org/x/tools/godoc/vfs, as CL 291669 added io/fs usage guarded by a build tag to an existing package.

Given:

package main

import (
	"golang.org/x/tools/godoc/vfs"
)

func main() {
	var _ vfs.FileSystem
}

And go.mod:

module some.tld/vfsthing

go 1.15

require golang.org/x/tools v0.1.1-0.20210217234408-19ff21fbe961

go mod tidy with Go 1.15 says:

some.tld/vfsthing imports
	golang.org/x/tools/godoc/vfs imports
	io/fs: package io/fs is not in GOROOT (/usr/lib/go/src/io/fs)

This doesn’t affect me personally, but I would sort of expect people to see this in the wild.

While this is caused by this issue, I can open a new issue just for x/tools/godoc/vfs if desired, if a new package needs to be created instead, like how mapfs and httpfs exist.

cc @rsc (since it was his CL)

This is mostly working as intended, but the error seems very frustrating.

go mod tidy ignores build constraints when deciding what dependencies are needed. If it didn’t, it would remove platform-specific dependencies (for example, modules providing packages only imported from files with // +build windows). That would be frustrating for teams working in mixed environments. (As an exception, files with the ignore tag are still excluded, as are files in testdata directories or directories starting with . or _).

In this case though, tidy in 1.13 treats hash/maphash as a regular import and reports an error. Since there’s no dot in the first path element, and there’s no required module that could provide that package, perhaps tidy should assume it’s a standard package in a future version of Go and ignore it.

cc @bcmills @matloob

And now x/tools/go/analysis/passes/buildtag depends on the new go/build/constraint package via CL 293834, so the tool I wrote to manage my $GOBIN via modules fails because it cannot tidy when pulling in gopls anymore:

tmpmod imports
	golang.org/x/tools/gopls imports
	golang.org/x/tools/gopls/internal/hooks imports
	golang.org/x/tools/internal/lsp/source imports
	golang.org/x/tools/go/analysis/passes/buildtag imports
	go/build/constraint: package go/build/constraint is not in GOROOT (/usr/lib/go/src/go/build/constraint)

(And I’m still waiting for my distribution to get Go 1.16, which probably won’t happen until 1.16.1 to fix some of the ICE/miscompilations ☹️)

@avivataqua, for now you can run go mod tidy -e using Go 1.16.

(Longer term, I think we should probably change go mod tidy to automatically suppress errors for missing imports in build-constrained files.)

I was thinking about this the other day; would it make sense to add a mechanism to ignore these packages’ absence during commands that don’t know about build tags (e.g., tidy), then once newer versions of Go are released, backport a list of the new packages to the supported releases so that they know that the new packages are “okay”?

Given only the previous two releases are supported and the number of new packages added per release are small, I can see this working reasonably well. I recall a few releases ago (1.9 and 1.10?) where some changes were backported to help with the introduction of modules in 1.11, and I find that similar in intention.