go: proxy.golang.org: widespread not found: fetch timed out
I pushed an untagged commit to the github.com/scjalliance/astconf module 3 hours ago, but sum.golang.org claims that it can’t find it when requesting its pseudo-version v0.0.0-20221209034541-5c12b5578696. It returns a 404 Not Found result.
Running go list -m for the branch does list the expected commit, but takes a very long time to run:
time go list -m -json github.com/scjalliance/astconf@main
{
"Path": "github.com/scjalliance/astconf",
"Version": "v0.0.0-20221209034541-5c12b5578696",
"Query": "main",
"Time": "2022-12-09T03:45:41Z",
"Origin": {
"VCS": "git",
"URL": "https://github.com/scjalliance/astconf",
"TagSum": "t1:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=",
"Ref": "refs/heads/main",
"Hash": "5c12b5578696a1517acd34d3c60d88b303b0b964"
}
}
real 2m55.660s
user 0m0.115s
sys 0m0.068s
Performing a go get for the branch returns server response: not found: fetch timed out:
go get github.com/scjalliance/astconf@main
go: downloading github.com/scjalliance/astconf v0.0.0-20221209034541-5c12b5578696
go: github.com/scjalliance/astconf@main: github.com/scjalliance/astconf@v0.0.0-20221209034541-5c12b5578696: verifying module: github.com/scjalliance/astconf@v0.0.0-20221209034541-5c12b5578696: reading https://sum.golang.org/lookup/github.com/scjalliance/astconf@v0.0.0-20221209034541-5c12b5578696: 404 Not Found
server response: not found: fetch timed out
I get a not found: fetch timed out response after waiting 58 seconds when I go here:
I don’t know if it matters, but I did perform a forced git push to the astconf repository earlier in the evening, and I renamed the branch from master to main yesterday.
I should also note that pkg.go.dev lists v0.0.0-20221209023833-cad60e1e1338 as the latest version right now, but when viewing that version specifically it also claims that it isn’t the latest version. I used the package request feature of pkg.go.dev to request the v0.0.0-20221209023833-cad60e1e1338 release earlier this evening.
I’ve been watching the GitHub Status Page and it claims all is well.
I am filing this report as directed by the FAQ on sum.golang.org, which was added in response to #42809:
Note that if someone requested the version before the tag was pushed, it may take up to 30 minutes for the mirror’s cache to expire and fresh data about the version to become available. If the version is still not available after 30 minutes, please file an issue.
I first encountered this issue while running the Go 1.19.2 tooling on windows/amd64. I updated to 1.19.4 but the issue persists. The slow response to go list -m was reproduced in a linux environment.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 77
- Comments: 28 (13 by maintainers)
Links to this issue
Commits related to this issue
- Revert "ci: temporary workaround for golang proxy/sumdb bug (#5463)" This reverts commit 44c78a9e7425390cf9d71f79bad5ac5d37efd4b1. It's been fixed: https://github.com/golang/go/issues/57185 Signed-... — committed to srenatus/opa by srenatus 2 years ago
- Revert "ci: temporary workaround for golang proxy/sumdb bug (#5463)" (#5469) This reverts commit 44c78a9e7425390cf9d71f79bad5ac5d37efd4b1. It's been fixed: https://github.com/golang/go/issues/5718... — committed to open-policy-agent/opa by srenatus 2 years ago
- feat: cache proxied checksum database content Given the recent incident with sum.golang.org (see https://github.com/golang/go/issues/57185), we should always cache the contents of the proxied checksu... — committed to goproxy/goproxy by aofei 2 years ago
please use the emoji voting on the issue if you’re seeing the issue, we don’t need a hundred comments of the same
go env -w GOPROXY=direct go env -w GOSUMDB=off
Hi all, I’m starting to look into this. Not sure what is going on yet.
Hi @bcmills, do you have a moment to check this out? Apparently sum.golang.org has some problems now, lots of “not found: fetch timed out”.
This issue just resurfaced for me.
We believe that the service has recovered and fetch/list should be succeeding again. There is still a tail of background refreshes completing, so some fetches may still take longer than normal because they are not cached.
We have rolled out additional mitigations and have not seen any issues in around a week.
We have continued to have a couple of additional sporadic issues in the same vain as the original issue, though shorter duration. We are still working to mitigate.
@wallyqs We received a large amount of distributed traffic overnight for module versions that did not exist. This led to a backlog of versions to process that we’ve only just burned down. The 1 minute timeout is inside of proxy.golang.org.
I’m closing this as the fetching issues should be resolved. Please open another bug if you are experience more issues.
I can confirm that it works for us again.
Same Issue here. It takes up to 60 secs to list a package. And sometimes I get like you a 404
I ran into this as well today; I was able to get things moving again with
export GOSUMDB=offandexport GOPROXY=directfor now.Having the same issue as @VetSoftKeith