go: cmd/go: unclear error when SumDB tree note is corrupted

What version of Go are you using (go version)?

$ go version
go version go1.14.2 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Nicolas\AppData\Local\go-build
set GOENV=C:\Users\Nicolas\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS= -mod=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=D:\Mis Documentos\Programacion\Go
set GOPRIVATE=
set GOPROXY=direct
set GOROOT=C:\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\Nicolas\AppData\Local\Temp\go-build045229398=/tmp/go-build -gno-record-gcc-switches

What did you do?

I’m trying to use go mod tidy command. Since my computer turned off because of a blackout it never worked again

What did you expect to see?

Succesful go.mod file

What did you see instead?

For every project and import I’m seeing this error:

github.com/sirupsen/logrus: github.com/sirupsen/logrus@v1.5.0: verifying module: github.com/sirupsen/logrus@v1.5.0: initializing sumdb.Client: reading tree note: malformed note
note:
 github.com/dgrijalva/jwt-go: github.com/dgrijalva/jwt-go@v1.0.2: verifying module: github.com/dgrijalva/jwt-go@v1.0.2: initializing sumdb.Client: reading tree note: malformed note
note:

I know I just copied 2 libraries, but It’s happening with every one

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 1
  • Comments: 17 (7 by maintainers)

Most upvoted comments

Hi,

Not sure, if it helps, but was facing the same issue, the following steps actually resolved the issue for me (Windows 10 - go version go1.14.4 windows/amd64)

The file %GOPATH%\pkg\sumdb\sum.golang.org\latest did not exist when I was got this error

Steps:

  1. touch %GOPATH%\pkg\sumdb\sum.golang.org\latest
  2. Ran the build got the following output

go: finding module for package github.com/gomodule/redigo/redis                                                                                                                                                          
go: finding module for package github.com/go-sql-driver/mysql                                                                                                                                                            
go: finding module for package github.com/jinzhu/gorm                                                                                                                                                                    
go: finding module for package github.com/google/uuid                                                                                                                                                                    
go: finding module for package github.com/gin-gonic/gin                                                                                                                                                                  
go: finding module for package github.com/dgrijalva/jwt-go                                                                                                                                                               
go: finding module for package github.com/nats-io/nats.go                                                                                                                                                                
go: finding module for package github.com/gin-contrib/cors                                                                                                                                                               
go: downloading github.com/jinzhu/gorm v1.9.15                                                                                                                                                                           
go: downloading github.com/go-sql-driver/mysql v1.5.0                                                                                                                                                                    
go: downloading github.com/google/uuid v1.1.1                                                                                                                                                                            
go: downloading github.com/gin-contrib/cors v1.3.1                                                                                                                                                                       
go: downloading github.com/gomodule/redigo/redis v0.0.0-do-not-use                                                                                                                                                       
go: downloading github.com/gin-gonic/gin v1.6.3                                                                                                                                                                          
go: downloading github.com/dgrijalva/jwt-go v1.0.2                                                                                                                                                                       
go: downloading github.com/gomodule/redigo v1.8.2                                                                                                                                                                        
go: downloading github.com/nats-io/nats.go v1.10.0                                                                                                                                                                       
internal\middlewares\auth.go:12:2: github.com/dgrijalva/jwt-go@v1.0.2: verifying module: github.com/dgrijalva/jwt-go@v1.0.2: initializing sumdb.Client: reading tree note: malformed note                                
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
internal\app\app.go:10:2: github.com/gin-contrib/cors@v1.3.1: verifying module: github.com/gin-contrib/cors@v1.3.1: initializing sumdb.Client: reading tree note: malformed note                                         
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
internal\app\app.go:11:2: github.com/gin-gonic/gin@v1.6.3: verifying module: github.com/gin-gonic/gin@v1.6.3: initializing sumdb.Client: reading tree note: malformed note                                               
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
internal\conf\configuration.go:12:2: github.com/go-sql-driver/mysql@v1.5.0: verifying module: github.com/go-sql-driver/mysql@v1.5.0: initializing sumdb.Client: reading tree note: malformed note                        
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
cmd\merchant-backend\main.go:9:2: github.com/gomodule/redigo/redis@v0.0.0-do-not-use: verifying module: github.com/gomodule/redigo/redis@v0.0.0-do-not-use: initializing sumdb.Client: reading tree note: malformed note 
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
internal\helpers\utils.go:19:2: github.com/google/uuid@v1.1.1: verifying module: github.com/google/uuid@v1.1.1: initializing sumdb.Client: reading tree note: malformed note                                             
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
cmd\merchant-backend\main.go:10:2: github.com/jinzhu/gorm@v1.9.15: verifying module: github.com/jinzhu/gorm@v1.9.15: initializing sumdb.Client: reading tree note: malformed note                                        
note:                                                                                                                                                                                                                    
                                                                                                                                                                                                                         
internal\conf\configuration.go:10:2: github.com/nats-io/nats.go@v1.10.0: verifying module: github.com/nats-io/nats.go@v1.10.0: initializing sumdb.Client: reading tree note: malformed note                              
note:                                                                                                                                                                                                                    

  1. go clean -modcache
  2. Ran go build again, with the same error message as in Step 2.
  3. rm %GOPATH%\pkg\sumdb\sum.golang.org\latest
  4. Ran go build again this time successfully with the following output
go: finding module for package github.com/gomodule/redigo/redis                         
go: finding module for package github.com/jinzhu/gorm                                   
go: finding module for package github.com/go-sql-driver/mysql                           
go: finding module for package github.com/gin-gonic/gin                                 
go: finding module for package github.com/nats-io/nats.go                               
go: finding module for package github.com/dgrijalva/jwt-go                              
go: finding module for package github.com/gin-contrib/cors                              
go: finding module for package github.com/google/uuid                                   
go: downloading github.com/nats-io/nats.go v1.10.0                                      
go: downloading github.com/go-sql-driver/mysql v1.5.0                                   
go: downloading github.com/dgrijalva/jwt-go v1.0.2                                      
go: downloading github.com/gin-contrib/cors v1.3.1                                      
go: downloading github.com/gomodule/redigo v1.8.2                                       
go: downloading github.com/google/uuid v1.1.1                                           
go: downloading github.com/gin-gonic/gin v1.6.3                                         
go: downloading github.com/gomodule/redigo/redis v0.0.0-do-not-use                      
go: downloading github.com/jinzhu/gorm v1.9.15                                          
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible                         
go: found github.com/gomodule/redigo/redis in github.com/gomodule/redigo v1.8.2         
go: found github.com/jinzhu/gorm in github.com/jinzhu/gorm v1.9.15                      
go: found github.com/gin-contrib/cors in github.com/gin-contrib/cors v1.3.1             
go: found github.com/gin-gonic/gin in github.com/gin-gonic/gin v1.6.3                   
go: found github.com/go-sql-driver/mysql in github.com/go-sql-driver/mysql v1.5.0       
go: found github.com/nats-io/nats.go in github.com/nats-io/nats.go v1.10.0              
go: found github.com/dgrijalva/jwt-go in github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: found github.com/google/uuid in github.com/google/uuid v1.1.1                       
go: downloading github.com/mattn/go-isatty v0.0.12                                      
go: downloading github.com/gin-contrib/sse v0.1.0                                       
go: downloading github.com/jinzhu/inflection v1.0.0                                     
go: downloading github.com/ugorji/go v1.1.7                                             
go: downloading github.com/nats-io/nkeys v0.1.4                                         
go: downloading github.com/go-playground/validator/v10 v10.2.0                          
go: downloading gopkg.in/yaml.v2 v2.2.8                                                 
go: downloading github.com/golang/protobuf v1.3.3                                       
go: downloading github.com/ugorji/go/codec v1.1.7                                       
go: downloading github.com/nats-io/jwt v0.3.2                                           
go: downloading github.com/nats-io/nuid v1.0.1                                          
go: downloading golang.org/x/crypto v0.0.0-20200323165209-0ec3e9974c59                  
go: downloading github.com/go-playground/universal-translator v0.17.0                   
go: downloading github.com/leodido/go-urn v1.2.0                                        
go: downloading github.com/go-playground/locales v0.13.0         
```                       ` 

I can see the latest file had been created after this step. 

Ok. Putting on hold until / unless we get more reports.

For what it’s worth, I’ll add that I similarly had a power outage (as in, my PC at some point deadlocked on all processes, Go or otherwise, including in the middle of a go get, so I was forced to cut the power), and I too got the “malformed note” error, which went away after deleting the %GOPATH%\pkg\sumdb\sum.golang.org\latest file (which was just 187 null bytes at that point). However, unlike the comments above me, another error took its place:

go: downloading github.com/hajimehoshi/oto/v2 v2.0.0
SECURITY ERROR
go.sum database server misbehavior detected!

old database:
        go.sum database tree
        7494163
        oOF65d3/MJZ5c1hjrktwPzGAPxhg7myKELBEqBK4gm8=

        — sum.golang.org Az3grmCvlocGJUpU2KWCCTzTMGZ/B+qyaGxoesMGbruKmiHiVax9N/UqyJ9gkALuSDZyKI2J9hu2DMUnd37LcSbmkwg=

new database:
        go.sum database tree
        7797325
        3B75PrlHOu76bf9kJUniWb4/d2xLX7oJCkCm0URIB50=

        — sum.golang.org Az3grvS7Ch4euazi+BB2bRhhpVmffiefMwk797VmmVbpvFGI9SRLKruR635hVEE2YS4QvqIzmuOjmYP1tkhFEdzI8QE=

proof of misbehavior:
        evhMbFcgHraUjA7LoYBgALFsih/yafHjiJIfuHnoTV4=    internal error: generated inconsistent proof
SECURITY ERROR
go.sum database server misbehavior detected!

old database:
        go.sum database tree
        7493958
        FlqjTFRXLZ4NQKfa50iQygh2OsnzxOuRSDf9fzwapAw=

        — sum.golang.org Az3griuSFkEngPLvClbWM4YETXJ9SXj0eaEqhGHFFG2F4+OblSQV0s7r4k4ssQPhSTvU4f5+ofnip76W7CavZ4c8oAY=

new database:
        go.sum database tree
        7797325
        3B75PrlHOu76bf9kJUniWb4/d2xLX7oJCkCm0URIB50=

        — sum.golang.org Az3grvS7Ch4euazi+BB2bRhhpVmffiefMwk797VmmVbpvFGI9SRLKruR635hVEE2YS4QvqIzmuOjmYP1tkhFEdzI8QE=

proof of misbehavior:
        etMZn0ytt7DnXmFpE++/UuKSsP0vveP0pbSOZAn2g5U=    internal error: generated inconsistent proof

I’m also running on Windows, in case it’s relevant.

Apologies if this new error deserves a separate issue; this seemed relevant enough to me to warrant adding to this issue, but I’m also rather new to Go.

@nicolascamilo Can you share with us the contents of $(go env GOPATH)/pkg/sumdb/sum.golang.org/latest? Then you can delete that file and it should unblock you.

$ cat "$(go env GOPATH)/pkg/sumdb/sum.golang.org/latest"
go.sum database tree
1072016
5rxqXjwvPB0445OEz5nw+s/gdsfoR0EUt3FEqMzbtCc=

— sum.golang.org Az3grhjle9b69brGvNYM1ZPKtFcpzaeKwQHomx3L7S5WuvtNRoVIM/KgmpsbfQSMEwqNFmoVR/gizfLYnOhtS7sUJgk=

@bcmills If the corrupted file is GOPATH/pkg/sumdb/sum.golang.org/latest, it would be important to understand how it came to be corrupted, because if an attacker can cause that they can downgrade the security guarantees significantly. If it’s any other file, we can blow it away.