athens: NOSUMDB/ATHENS_GONOSUM_PATTERNS Being Ignored

Describe the bug

I’m setting ATHENS_GONOSUM_PATTERNS but I’m still seeing Athens attempt to reach out to https://sum.golang.org/lookup for the package that should be matched in the GONOSUM_PATTERNS

My Config:

ATHENS_GONOSUM_PATTERNS: github.mycorp.local/*

Error Message

time="2019-09-18T17:47:35Z" level=error msg="exit status 1: go: finding github.mycorp.local/chris-schmitt/mypkg v1.1.3\ngo: finding github.mycorp.local/chris-schmitt/mypkg v1.1.4\nverifying github.mycorp.local/chris-schmitt/mypkg@v1.1.3/go.mod: github.mycorp.local/chris-schmitt/mypkg@v1.1.3/go.mod: reading https://sum.golang.org/lookup/github.mycorp.local/chris-schmitt/mypkg@v1.1.3: 410 Gone\n" http-method=GET http-path=/github.mycorp.local/chris-schmitt/mypkg/@v/v1.1.3.info http-url=/github.mycorp.local/chris-schmitt/mypkg/@v/v1.1.3.info kind="Internal Server Error" module=github.mycorp.local/chris-schmitt/mypkg operation=download.InfoHandler ops="[download.InfoHandler pool.Info protocol.Info protocol.processDownload stash.Pool stasher.Stash stasher.fetchModule goGetFetcher.Fetch module.downloadModule]" version=v1.1.3

To Reproduce

Set ATHENS_GONOSUM_PATTERNS to a pattern matching an internal GitHub/GitLab or private repo
Attempt to fetch the package locally with env GOPROXY="https://myathens" and GONOSUMDB=github.mycorp.local
Get failure message on compilation `go: github.mycorp.local/chris-schmitt/mypkg@v1.1.3: unexpected status (https://goproxy.mycorp.local/github.mycorp.local/chris-schmitt/mypkg/@v/v1.1.3.info): 500 Internal Server Error

Expected behavior

Working builds where Athens will fetch and cache packages that are in our internal GitHub instance while not reaching out to https://sum.golang.org/lookup (thus skipping the checksum database)

Environment (please complete the following information):

OS: Kubernetes (using your helm chart)
Go version : 1.13
Proxy version : v0.6.0
Storage (fs/mongodb/s3 etc.) : fs

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 16 (2 by maintainers)

Most upvoted comments

@housejester the GOSUM still being checked is a different issue. It’s because we don’t pass GOPRIVATE or GONOSUMDB to our internal go mod download command. I’m pushing another PR for this really soon 😃

marwan-at-work on Sep 20, 2019

Ah, I didn’t realize it was using path.Match

Keep in mind though, that you would need another /* for each sub package deeper an import could be.

This is somewhat concerning since given the end users don’t have much control over it? I don’t want anything matching github.mycorp.local to ever hit the Public Checksum DB but at the same time I have no idea about my other users and their sub packages. It’d be nice if this used Regex (since ** isn’t supported) or a custom matcher that implemented a true wildcard for all sub-packages

tankbusta on Sep 18, 2019

I ran tests with two of our bigger projects. Now works as expected with private repos, no sums are checked. Thanks @marwan-at-work!

iwittkau on Sep 24, 2019

v0.6.1 is now released, please feel free to try it and re-open the issue if the error persists

marwan-at-work on Sep 23, 2019

I have the same problem.

Even setting a correct pattern does result in athens querying sum.golang.org (internal server error).

My configuration is similar to this:

Pattern: git.region.company.com/*/*
Module Name: git.region.company.com/group/project

which results in a positive (true) match for path.Match. But athens still tries to query sum.golang.org and returns an internal server error.

I think to avoid this, the fetcher needs to set GONOSUMDB in the environment for the underlying go mod download command?

iwittkau on Sep 19, 2019