go: Packages under rsc.io are no longer available

What version of Go are you using (go version)?

Go 1.12, Go 1.14

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

I don’t think this matters but here it goes. This failed both on my laptop with Go 1.14 and in GitHub Action that uses Go1.12

go env Output
$ go env

GO111MODULE=“”
GOARCH=“amd64”
GOBIN=“”
GOCACHE=“/Users/yenlinc/Library/Caches/go-build”
GOENV=“/Users/yenlinc/Library/Application Support/go/env”
GOEXE=“”
GOFLAGS=“”
GOHOSTARCH=“amd64”
GOHOSTOS=“darwin”
GOINSECURE=“”
GONOPROXY=“"
GONOSUMDB="”
GOOS=“darwin”
GOPATH=“/Users/yenlinc/go”
GOPRIVATE=“*”
GOPROXY=“direct”
GOROOT=“/usr/local/Cellar/go/1.14.3/libexec”
GOSUMDB=“off”
GOTMPDIR=“”
GOTOOLDIR=“/usr/local/Cellar/go/1.14.3/libexec/pkg/tool/darwin_amd64”
GCCGO=“gccgo”
AR=“ar”
CC=“clang”
CXX=“clang++”
CGO_ENABLED=“1”
GOMOD=“/Users/yenlinc/workplace/yenlinc/amazon-kinesis-streams-for-fluent-bit/go.mod”
CGO_CFLAGS=“-g -O2”
CGO_CPPFLAGS=“”
CGO_CXXFLAGS=“-g -O2”
CGO_FFLAGS=“-g -O2”
CGO_LDFLAGS=“-g -O2”
PKG_CONFIG=“pkg-config”
GOGCCFLAGS=“-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/v4/nbxvs8tj4ms68w5ghhnzn_f1ckngqg/T/go-build594642831=/tmp/go-build -gno-record-gcc-switches -fno-common”

What did you do?

Ran go build command

What did you expect to see?

Modules should be fetched successfully and compilation should succeed.

What did you see instead?

Since this morning, 2 packages under rsc.io have been failing:

go build -buildmode c-shared -o ./bin/kinesis.so ./
go: github.com/aws/amazon-kinesis-firehose-for-fluent-bit@v1.4.1 requires
	github.com/golang/mock@v1.4.3 requires
	rsc.io/quote/v3@v3.1.0: unrecognized import path "rsc.io/quote/v3": reading https://rsc.io/quote/v3?go-get=1: 503 Service Unavailable

$ go mod download

go: finding github.com/aws/amazon-kinesis-firehose-for-fluent-bit v1.2.2-0.20200717080229-e3ffb88e79bb
go: finding github.com/aws/aws-sdk-go v1.33.13
go: finding github.com/lestrrat-go/strftime v1.0.2-0.20200618102204-1b0bc59fa4ab
go: finding github.com/stretchr/objx v0.3.0
go: finding github.com/sirupsen/logrus v1.6.0
go: finding github.com/yuin/goldmark v1.1.33
go: rsc.io/sampler@v1.99.99: unrecognized import path "rsc.io/sampler" (parse https://rsc.io/sampler?go-get=1: no go-import meta tags ())
go: finding golang.org/x/crypto v0.0.0-20200709230013-948cd5f35899
go: finding golang.org/x/text v0.3.3
go: finding github.com/google/gofuzz v1.1.0
go: finding gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776
go: finding github.com/cenkalti/backoff v2.2.1+incompatible
go: finding golang.org/x/net v0.0.0-20200707034311-ab3426394381
go: finding github.com/fluent/fluent-bit-go v0.0.0-20200707230002-2a28684e2382
go: finding github.com/stretchr/testify v1.6.1
go: finding github.com/json-iterator/go v1.1.10
go: finding gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f
go: finding golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae
go: finding golang.org/x/tools v0.0.0-20200717024301-6ddee64345a6
go: finding github.com/golang/mock v1.4.3
go: error loading module requirements

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 2
  • Comments: 18 (9 by maintainers)

Most upvoted comments

@menghanl Fixing rsc.io/quote is being tracked in issue https://github.com/rsc/quote/issues/4.

+4
dmitshur on Jul 28, 2020

cc @rsc @dmitshur @toothrot

While this would probably not normally be considered a Go issue, it is breaking the -longtest builders.

e.g., https://storage.googleapis.com/go-build-log/3c347861/linux-amd64-longtest_bb29e807.log

+3
prattmic on Jul 28, 2020

@dmitshur

go version outputs:

go version go1.14.3 darwin/amd64

go env outputs:

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/yenlinc/Library/Caches/go-build"
GOENV="/Users/yenlinc/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY="*"
GONOSUMDB="*"
GOOS="darwin"
GOPATH="/Users/yenlinc/go"
GOPRIVATE="*"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.3/libexec"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/yenlinc/workplace/yenlinc/amazon-kinesis-firehose-for-fluent-bit/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/v4/nbxvs8tj4ms68w5ghhnzn_f1ckngqg/T/go-build709008692=/tmp/go-build -gno-record-gcc-switches -fno-common"

Content of go.mod:

module github.com/aws/amazon-kinesis-firehose-for-fluent-bit

go 1.12

require (
	github.com/aws/aws-sdk-go v1.33.13
	github.com/cenkalti/backoff v2.2.1+incompatible
	github.com/fluent/fluent-bit-go v0.0.0-20190614024040-c017a8579953
	github.com/golang/mock v1.4.3
	github.com/json-iterator/go v1.1.10
	github.com/lestrrat-go/strftime v1.0.2-0.20200618102204-1b0bc59fa4ab
	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
	github.com/modern-go/reflect2 v1.0.1 // indirect
	github.com/sirupsen/logrus v1.6.0
	github.com/stretchr/testify v1.6.1
)

Commands ran:

$ mkdir -p ./bin && go build -buildmode c-shared -o ./bin/firehose.so ./

go: github.com/golang/mock@v1.4.3 requires
	rsc.io/quote/v3@v3.1.0: unrecognized import path "rsc.io/quote/v3": reading https://rsc.io/quote/v3?go-get=1: 503 Service Unavailable

One thing I discovered is if I just:

  1. remove the go.mod and go.sum
  2. reinitialize the module via go mod init github.com/aws/amazon-kinesis-firehose-for-fluent-bit
  3. Change the go version in go.mod from 1.14 to 1.12
  4. Re-run go build Then it compiles successfully. But I suspect that’s just because the indirect dependency on rsc.io/sampler and rsc.io/quote were removed
+2
hencrice on Jul 28, 2020

Okay, setting GOPROXY fixed it for me:

$ export GOPROXY="https://proxy.golang.org"

Then when I directly download binaryregexp, it succeeds:

$ go get -u rsc.io/binaryregexp
go: finding github.com/go-openapi/errors v0.17.0
go: finding github.com/go-openapi/jsonreference v0.19.2
go: finding github.com/mitchellh/mapstructure v1.1.2
go: finding golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f
go: finding github.com/globalsign/mgo v0.0.0-20180905125535-1ca0a4f7cbcb
go: finding github.com/asaskevich/govalidator v0.0.0-20180720115003-f9ffefc3facf
go: finding github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329
go: finding github.com/kr/pty v1.1.5
go: finding golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8
go: finding github.com/go-openapi/jsonpointer v0.19.2
go: finding rsc.io/binaryregexp v0.2.0
go: finding github.com/PuerkitoBio/purell v1.1.1
go: finding github.com/go-openapi/loads v0.19.0
go: finding github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8
go: finding golang.org/x/net v0.0.0-20190613194153-d28f0bde5980
go: finding github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63
go: finding github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578
go: finding golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59
go: finding github.com/go-openapi/runtime v0.19.0
go: finding github.com/go-openapi/errors v0.18.0
go: finding github.com/go-openapi/jsonreference v0.18.0
go: finding golang.org/x/sys v0.0.0-20190321052220-f7bb7a8bee54
go: finding golang.org/x/net v0.0.0-20190320064053-1272bf9dcd53
go: finding golang.org/x/crypto v0.0.0-20190320223903-b7391e95e576
go: finding github.com/go-openapi/validate v0.18.0
go: finding github.com/go-openapi/jsonpointer v0.18.0
go: finding github.com/go-openapi/strfmt v0.18.0
go: finding github.com/go-openapi/spec v0.17.0
go: finding golang.org/x/net v0.0.0-20181005035420-146acd28ed58
go: finding github.com/mailru/easyjson v0.0.0-20190312143242-1de009706dbe
go: finding github.com/go-openapi/swag v0.17.0
go: finding github.com/docker/go-units v0.3.3
go: finding github.com/go-openapi/swag v0.18.0
go: finding github.com/go-openapi/jsonreference v0.17.0
go: finding github.com/go-openapi/loads v0.18.0
go: finding github.com/go-openapi/runtime v0.0.0-20180920151709-4f900dc2ade9
go: finding github.com/go-openapi/spec v0.18.0
go: finding github.com/go-openapi/analysis v0.0.0-20180825180245-b006789cd277
go: finding github.com/go-openapi/jsonpointer v0.17.0
go: finding github.com/PuerkitoBio/purell v1.1.0
go: finding github.com/go-openapi/strfmt v0.17.0
go: finding github.com/pborman/uuid v1.2.0
go: finding github.com/go-openapi/analysis v0.17.0
go: finding github.com/go-openapi/loads v0.17.0
go: finding github.com/go-openapi/analysis v0.18.0
go: finding github.com/google/uuid v1.0.0
go: downloading rsc.io/binaryregexp v0.2.0
go: extracting rsc.io/binaryregexp v0.2.0
+2
AntonNguyen on Jul 28, 2020

rsc.io is working again now (see https://github.com/rsc/quote/issues/4#issuecomment-665300101). Making cmd/go tests not depend on external services is tracked in issue #28856. I don’t think there’s anything else left to do here, so I’ll close this.

+1
dmitshur on Jul 28, 2020

@hencrice The GOPRIVATE environment variable in go env output is set to *, which tells the go command to consider all modules as private and not to use the module proxy and checksum database for them.

To be able to fetch rsc.io/quote/v3 and other public modules from the mirror, that environment variable will need to be updated to something narrower. See https://golang.org/cmd/go/#hdr-Module_configuration_for_non_public_modules for more information about it.

+1
dmitshur on Jul 28, 2020

@AntonNguyen Could you show your full go env? I tried setting GOPROXY but it still failed for me. @hencrice

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/circleci/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY="https://proxy.golang.org"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/go/src/github.com/user/repo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build689553606=/tmp/go-build -gno-record-gcc-switches"
+1
AntonNguyen on Jul 28, 2020

Just wanted to chime in and say that it’s breaking for my builds (go 1.12) as well:

go test -coverprofile=coverage.txt -covermode=count ./...
go: finding github.com/golang/mock v1.4.3
go: finding github.com/go-kit/kit v0.9.0
go: finding github.com/prometheus/client_golang v1.3.0
go: finding github.com/golang/protobuf v1.4.2
go: finding github.com/kelseyhightower/envconfig v1.4.0
go: finding github.com/streadway/handy v0.0.0-20190108123426-d5acb3125c2a
go: finding github.com/stretchr/testify v1.6.1
go: finding google.golang.org/grpc v1.30.0
go: finding github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: finding github.com/golang/protobuf v1.3.2
go: finding github.com/go-logfmt/logfmt v0.4.0
go: finding github.com/go-stack/stack v1.8.0
go: rsc.io/quote/v3@v3.1.0: unrecognized import path "rsc.io/quote/v3" (parse https://rsc.io/quote/v3?go-get=1: no go-import meta tags ())
go: finding github.com/json-iterator/go v1.1.8
go: finding github.com/stretchr/testify v1.3.0
go: finding github.com/prometheus/client_model v0.1.0
go: finding github.com/prometheus/procfs v0.0.8
go: finding golang.org/x/tools v0.0.0-20190425150028-36563e24a262
go: finding github.com/kr/pretty v0.1.0
go: finding go.opencensus.io v0.22.1
go: finding golang.org/x/sys v0.0.0-20191220142924-d4481acd189f
go: finding github.com/cespare/xxhash/v2 v2.1.1
go: finding gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
go: finding github.com/beorn7/perks v1.0.1
go: finding github.com/davecgh/go-spew v1.1.0
go: finding golang.org/x/net v0.0.0-20190628185345-da137c7871d7
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/stretchr/objx v0.1.0
go: finding github.com/prometheus/common v0.7.0
go: finding github.com/golang/mock v1.3.1
go: finding github.com/satori/go.uuid v1.2.0
go: finding cloud.google.com/go/datastore v1.1.0
go: error loading module requirements
+1
AntonNguyen on Jul 28, 2020

this is mitigated by using the proxy, which you appear to have disabled with GOPROXY=direct

but @rsc probably needs to update their app engine server

+1
seankhliao on Jul 28, 2020
Read more comments on GitHub
← go: os/user: TestGroupIds fails on windows 10
go: Panic in crypto/tls/common.go →