go: cmd/go: 'go get' does not add missing hash to go.sum when ziphash file missing from cache

Hello up there. I’ve hit a problem with Go1.16 where its module subsystem misbehaves.

The steps that show the problem are in https://github.com/golang/go/issues/44129#issuecomment-788611690 . However @bcmills later says he cannot reproduce the problem and asks for more details about how my Go toolchain is bult (https://github.com/golang/go/issues/44129#issuecomment-789135792, https://github.com/bcmills). My Go is just a regular Go source checkout built from go1.16-12-g2b7243a62f with $GOROOT_BOOTSTRAP pointing to go1.4.3-16-g4d5426a570. I’ve reverified building it from scratch in a new user environment and the problem is indeed gone.

HOWEVER

I’ve tried to rebuild my Go the same way in my original user and the problem persisted. Then I’ve removed ~/.cache/go-build and the problem peristed. Then I’ve removed ~/go/pkg and the problem disappeared.

I’ve preserved original content of my ~/go/pkg and tried to trim it down to something minimal. I attach it here for the reference - if I set ~/go/pkg to that the problem of not updating go.sum on go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230 shows. With ~go reset to empty the problem disappears.

Offhand it looks to me to be a “stale cache not handled correctly” kind of problem.

Thanks beforehand, Kirill

/cc @rsc, @jayconrod

What version of Go are you using (go version)?

$ go version
go version go1.16 linux/amd64

Does this issue reproduce with the latest release?

No, I had not seen it with Go1.15

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/kirr/.cache/go-build"
GOENV="/home/kirr/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/kirr/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/kirr/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/kirr/src/tools/go/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/kirr/src/tools/go/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16"
GCCGO="/usr/bin/gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/kirr/tmp/trashme/wendelin.core/wcfs/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3989139164=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version go1.16 linux/amd64
GOROOT/bin/go tool compile -V: compile version go1.16
uname -sr: Linux 5.10.0-3-amd64
Distributor ID:	Debian
Description:	Debian GNU/Linux bullseye/sid
Release:	testing
Codename:	bullseye
/lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Debian GLIBC 2.31-9) stable release version 2.31.
gdb --version: GNU gdb (Debian 10.1-1.7) 10.1.90.20210103-git

What did you do?

(copied from https://github.com/golang/go/issues/44129#issuecomment-788611690)

kirr@deco:~/tmp/trashme$ git clone https://lab.nexedi.com/kirr/wendelin.core
Клонирование в «wendelin.core»…
warning: переадресация на https://lab.nexedi.com/kirr/wendelin.core.git/
remote: Enumerating objects: 15831, done.
remote: Counting objects: 100% (15831/15831), done.
remote: Compressing objects: 100% (4249/4249), done.
remote: Total 15831 (delta 11537), reused 15632 (delta 11373)
Получение объектов: 100% (15831/15831), 7.04 MiB | 2.96 MiB/s, готово.
Определение изменений: 100% (11537/11537), готово.

kirr@deco:~/tmp/trashme$ cd wendelin.core/

kirr@deco:~/tmp/trashme/wendelin.core$ git checkout 481ffb89e78d310f9bdc79f14771494215d0d55a
Note: switching to '481ffb89e78d310f9bdc79f14771494215d0d55a'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD сейчас на 481ffb8 X wcfs: v↑ * (checkpoint)

kirr@deco:~/tmp/trashme/wendelin.core$ cd wcfs/

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.16 build
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20210302025843-863c4602a230/xnet/lonet/registry_sqlite.go:28:2: missing go.sum entry for module providing package crawshaw.io/sqlite (imported by lab.nexedi.com/kirr/go123/xnet/lonet); to add:
        go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20210302025843-863c4602a230/xnet/lonet/registry_sqlite.go:29:2: missing go.sum entry for module providing package crawshaw.io/sqlite/sqlitex (imported by lab.nexedi.com/kirr/go123/xnet/lonet); to add:
        go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/neo/go@v0.0.0-20210302031331-91085e39ea86/neo/neonet/misc.go:27:2: missing go.sum entry for module providing package github.com/philhofer/fwd (imported by lab.nexedi.com/kirr/neo/go/neo/neonet); to add:
        go get lab.nexedi.com/kirr/neo/go/neo/neonet@v0.0.0-20210302031331-91085e39ea86

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.16 get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.16 get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ git st
HEAD отделён на 481ffb8
нечего коммитить, нет изменений в рабочем каталоге

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.16 get -v lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.16 build
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20210302025843-863c4602a230/xnet/lonet/registry_sqlite.go:28:2: missing go.sum entry for module providing package crawshaw.io/sqlite (imported by lab.nexedi.com/kirr/go123/xnet/lonet); to add:
        go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20210302025843-863c4602a230/xnet/lonet/registry_sqlite.go:29:2: missing go.sum entry for module providing package crawshaw.io/sqlite/sqlitex (imported by lab.nexedi.com/kirr/go123/xnet/lonet); to add:
        go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/neo/go@v0.0.0-20210302031331-91085e39ea86/neo/neonet/misc.go:27:2: missing go.sum entry for module providing package github.com/philhofer/fwd (imported by lab.nexedi.com/kirr/neo/go/neo/neonet); to add:
        go get lab.nexedi.com/kirr/neo/go/neo/neonet@v0.0.0-20210302031331-91085e39ea86

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.15 build

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.15 get -v lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
go: found lab.nexedi.com/kirr/go123/xnet/lonet in lab.nexedi.com/kirr/go123 v0.0.0-20210302025843-863c4602a230

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ git st
HEAD отделён на 481ffb8
нечего коммитить, нет изменений в рабочем каталоге

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.16 build
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20210302025843-863c4602a230/xnet/lonet/registry_sqlite.go:28:2: missing go.sum entry for module providing package crawshaw.io/sqlite (imported by lab.nexedi.com/kirr/go123/xnet/lonet); to add:
        go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20210302025843-863c4602a230/xnet/lonet/registry_sqlite.go:29:2: missing go.sum entry for module providing package crawshaw.io/sqlite/sqlitex (imported by lab.nexedi.com/kirr/go123/xnet/lonet); to add:
        go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/neo/go@v0.0.0-20210302031331-91085e39ea86/neo/neonet/misc.go:27:2: missing go.sum entry for module providing package github.com/philhofer/fwd (imported by lab.nexedi.com/kirr/neo/go/neo/neonet); to add:
        go get lab.nexedi.com/kirr/neo/go/neo/neonet@v0.0.0-20210302031331-91085e39ea86

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ GO111MODULE=on go1.15 build

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.16 build
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20210302025843-863c4602a230/xnet/lonet/registry_sqlite.go:28:2: missing go.sum entry for module providing package crawshaw.io/sqlite (imported by lab.nexedi.com/kirr/go123/xnet/lonet); to add:
        go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20210302025843-863c4602a230/xnet/lonet/registry_sqlite.go:29:2: missing go.sum entry for module providing package crawshaw.io/sqlite/sqlitex (imported by lab.nexedi.com/kirr/go123/xnet/lonet); to add:
        go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/neo/go@v0.0.0-20210302031331-91085e39ea86/neo/neonet/misc.go:27:2: missing go.sum entry for module providing package github.com/philhofer/fwd (imported by lab.nexedi.com/kirr/neo/go/neo/neonet); to add:
        go get lab.nexedi.com/kirr/neo/go/neo/neonet@v0.0.0-20210302031331-91085e39ea86

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ echo $?
1

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.16 get -t .

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ git st
HEAD отделён на 481ffb8
нечего коммитить, нет изменений в рабочем каталоге

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ git diff

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.16 build
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20210302025843-863c4602a230/xnet/lonet/registry_sqlite.go:28:2: missing go.sum entry for module providing package crawshaw.io/sqlite (imported by lab.nexedi.com/kirr/go123/xnet/lonet); to add:
        go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/go123@v0.0.0-20210302025843-863c4602a230/xnet/lonet/registry_sqlite.go:29:2: missing go.sum entry for module providing package crawshaw.io/sqlite/sqlitex (imported by lab.nexedi.com/kirr/go123/xnet/lonet); to add:
        go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230
/home/kirr/go/pkg/mod/lab.nexedi.com/kirr/neo/go@v0.0.0-20210302031331-91085e39ea86/neo/neonet/misc.go:27:2: missing go.sum entry for module providing package github.com/philhofer/fwd (imported by lab.nexedi.com/kirr/neo/go/neo/neonet); to add:
        go get lab.nexedi.com/kirr/neo/go/neo/neonet@v0.0.0-20210302031331-91085e39ea86

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.15 version
go version go1.15.8 linux/amd64   # built from go1.15.8-6-gf75ab2d5a6

kirr@deco:~/tmp/trashme/wendelin.core/wcfs$ go1.16 version
go version go1.16 linux/amd64     # built from go1.16-12-g2b7243a62f

What did you expect to see?

go.sum updated and second attempt to build succeeds

What did you see instead?

go get lab.nexedi.com/kirr/go123/xnet/lonet@v0.0.0-20210302025843-863c4602a230 - that go build suggests to update go.sum, changes nothing.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 2
  • Comments: 17 (15 by maintainers)

Commits related to this issue

Most upvoted comments

Attaching content of ~/go with which the problem reproduces: homego.zip

+2
navytux on Mar 3, 2021

Backport issue #44872 opened. (gopherbot won’t open it due to #25574).

+1
jayconrod on Mar 8, 2021

Makes sense, keeping the releases consistent is definitely a good thing. I’ll open a backport issue and CL.

To be clear, in Go 1.16, go build is correctly reporting a problem. The bug is that go get doesn’t fix it. go get doesn’t fix that problem in 1.15 either, but the problem is harder to notice, since go build doesn’t complain.

+1
jayconrod on Mar 8, 2021

If the fix applies cleanly to 1.15 and is safe, it makes sense to consider for backport to both. It would keep the two releases more in sync, which generally helps people be able to move from Go 1.15 to 1.16 (or the opposite direction if needed) smoothly. Otherwise, given the go build doesn’t report an error in 1.15, fixing just that problem in 1.16 only is also reasonable.

+1
dmitshur on Mar 8, 2021
Read more comments on GitHub
← go: cmd/go: go get can block for extremely long periods of time without any indication of functionality
go: cmd/go: go get failing with vague error 'unknown revision v0.0.0' with kubernetes →