kubernetes: 'unknown revision v0.0.0' errors, seemingly due to 'require k8s.io/foo v0.0.0'

For anyone else who hits this issue, after much weeping and gnashing of teeth, this is the little script I wrote to switch kubernetes versions:

#!/bin/sh
set -euo pipefail

VERSION=${1#"v"}
if [ -z "$VERSION" ]; then
    echo "Must specify version!"
    exit 1
fi
MODS=($(
    curl -sS https://raw.githubusercontent.com/kubernetes/kubernetes/v${VERSION}/go.mod |
    sed -n 's|.*k8s.io/\(.*\) => ./staging/src/k8s.io/.*|k8s.io/\1|p'
))
for MOD in "${MODS[@]}"; do
    V=$(
        go mod download -json "${MOD}@kubernetes-${VERSION}" |
        sed -n 's|.*"Version": "\(.*\)".*|\1|p'
    )
    go mod edit "-replace=${MOD}=${MOD}@${V}"
done
go get "k8s.io/kubernetes@v${VERSION}"

Update on 2023-01-10:

As noted later in this issue, use of packages within the k8s.io/kubernetes module as a library/dependency is unsupported and not recommended. The modules intended to be consumed as libraries are published with go.mod files that do not require any replace directives.

That is noted in the k8s.io/kubernetes repo README as well:

To use Kubernetes code as a library in other applications, see the list of published components. Use of the k8s.io/kubernetes module or k8s.io/kubernetes/… packages as libraries is not supported.

If in spite of that, other projects want to use k8s.io/kubernetes as a library, replicating the replace directives from the k8s.io/kubernetes go.mod file to point to appropriate tags of the other k8s.io modules is required.

Original comment follows:

@liggitt Can you give me an example of a require I have to add to my go.mod?

pin all the dependencies to a matching level of k8s.io/kubernetes (these all correspond to the kubernetes-1.15.0 tag):

require k8s.io/kubernetes v1.15.0

replace (
	k8s.io/api => k8s.io/api v0.0.0-20190620084959-7cf5895f2711
	k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.0.0-20190620085554-14e95df34f1f
	k8s.io/apimachinery => k8s.io/apimachinery v0.0.0-20190612205821-1799e75a0719
	k8s.io/apiserver => k8s.io/apiserver v0.0.0-20190620085212-47dc9a115b18
	k8s.io/cli-runtime => k8s.io/cli-runtime v0.0.0-20190620085706-2090e6d8f84c
	k8s.io/client-go => k8s.io/client-go v0.0.0-20190620085101-78d2af792bab
	k8s.io/cloud-provider => k8s.io/cloud-provider v0.0.0-20190620090043-8301c0bda1f0
	k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.0.0-20190620090013-c9a0fc045dc1
	k8s.io/code-generator => k8s.io/code-generator v0.0.0-20190612205613-18da4a14b22b
	k8s.io/component-base => k8s.io/component-base v0.0.0-20190620085130-185d68e6e6ea
	k8s.io/cri-api => k8s.io/cri-api v0.0.0-20190531030430-6117653b35f1
	k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.0.0-20190620090116-299a7b270edc
	k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.0.0-20190620085325-f29e2b4a4f84
	k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.0.0-20190620085942-b7f18460b210
	k8s.io/kube-proxy => k8s.io/kube-proxy v0.0.0-20190620085809-589f994ddf7f
	k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.0.0-20190620085912-4acac5405ec6
	k8s.io/kubelet => k8s.io/kubelet v0.0.0-20190620085838-f1cb295a73c9
	k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.0.0-20190620090156-2138f2c9de18
	k8s.io/metrics => k8s.io/metrics v0.0.0-20190620085625-3b22d835f165
	k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.0.0-20190620085408-1aef9010884e
)

Copied from the referenced golang issue thread:

k8s.io/kubernetes is not primarily intended to be consumed as a module. Only the published subcomponents are (and go get works properly with those).

If you want to consume k8s.io/kubernetes as a module, you’d probably need to add require directives for matching versions of all of the subcomponents, rather than using go get

/close

This might be a Bad Idea™, but appending this to the consumer go.mod file at least compiled the failing example in https://github.com/golang/go/issues/32776#issuecomment-505614864:

require k8s.io/kubernetes 8c3b7d7679ccf368
replace (
        k8s.io/api => k8s.io/kubernetes/staging/src/k8s.io/api 8c3b7d7679ccf368
        k8s.io/apiextensions-apiserver => k8s.io/kubernetes/staging/src/k8s.io/apiextensions-apiserver 8c3b7d7679ccf368
        k8s.io/apimachinery => k8s.io/kubernetes/staging/src/k8s.io/apimachinery 8c3b7d7679ccf368
        k8s.io/apiserver => k8s.io/kubernetes/staging/src/k8s.io/apiserver 8c3b7d7679ccf368
        k8s.io/cli-runtime => k8s.io/kubernetes/staging/src/k8s.io/cli-runtime 8c3b7d7679ccf368
        k8s.io/client-go => k8s.io/kubernetes/staging/src/k8s.io/client-go 8c3b7d7679ccf368
        k8s.io/cloud-provider => k8s.io/kubernetes/staging/src/k8s.io/cloud-provider 8c3b7d7679ccf368
        k8s.io/cluster-bootstrap => k8s.io/kubernetes/staging/src/k8s.io/cluster-bootstrap 8c3b7d7679ccf368
        k8s.io/code-generator => k8s.io/kubernetes/staging/src/k8s.io/code-generator 8c3b7d7679ccf368
        k8s.io/component-base => k8s.io/kubernetes/staging/src/k8s.io/component-base 8c3b7d7679ccf368
        k8s.io/cri-api => k8s.io/kubernetes/staging/src/k8s.io/cri-api 8c3b7d7679ccf368
        k8s.io/csi-translation-lib => k8s.io/kubernetes/staging/src/k8s.io/csi-translation-lib 8c3b7d7679ccf368
        k8s.io/kube-aggregator => k8s.io/kubernetes/staging/src/k8s.io/kube-aggregator 8c3b7d7679ccf368
        k8s.io/kube-controller-manager => k8s.io/kubernetes/staging/src/k8s.io/kube-controller-manager 8c3b7d7679ccf368
        k8s.io/kube-proxy => k8s.io/kubernetes/staging/src/k8s.io/kube-proxy 8c3b7d7679ccf368
        k8s.io/kube-scheduler => k8s.io/kubernetes/staging/src/k8s.io/kube-scheduler 8c3b7d7679ccf368
        k8s.io/kubectl => k8s.io/kubernetes/staging/src/k8s.io/kubectl 8c3b7d7679ccf368
        k8s.io/kubelet => k8s.io/kubernetes/staging/src/k8s.io/kubelet 8c3b7d7679ccf368
        k8s.io/legacy-cloud-providers => k8s.io/kubernetes/staging/src/k8s.io/legacy-cloud-providers 8c3b7d7679ccf368
        k8s.io/metrics => k8s.io/kubernetes/staging/src/k8s.io/metrics 8c3b7d7679ccf368
        k8s.io/node-api => k8s.io/kubernetes/staging/src/k8s.io/node-api 8c3b7d7679ccf368
        k8s.io/sample-apiserver => k8s.io/kubernetes/staging/src/k8s.io/sample-apiserver 8c3b7d7679ccf368
        k8s.io/sample-cli-plugin => k8s.io/kubernetes/staging/src/k8s.io/sample-cli-plugin 8c3b7d7679ccf368
        k8s.io/sample-controller => k8s.io/kubernetes/staging/src/k8s.io/sample-controller 8c3b7d7679ccf368
)

8c3b7d7679ccf368 was master on k8s.io/kubernetes as of a couple days ago.

That was generated by taking the replace directives that point to the relative filesystem path ./staging/... in k8s.io/kubernetes/go.mod, and using those to make new replace directives that instead point to k8s.io/kubernetes/staging/... at revision 8c3b7d7679ccf368:

echo 'require k8s.io/kubernetes 8c3b7d7679ccf368'
echo 'replace ('
curl -sS https://raw.githubusercontent.com/kubernetes/kubernetes/8c3b7d7679ccf368cc4cedb352fac4cd1b82124e/go.mod \
| sed -n -r 's# \./staging/(.*)$# k8s.io/kubernetes/staging/\1 8c3b7d7679ccf368#p'
echo ')'

The results of that can then be added to a consumer’s go.mod file.

@bcmills I completely get that. But people who are “doing it wrong” by depending on k8s.io/kubernetes are not the only ones that are broken. Anyone that even has it as a transitive dependency is broken. I don’t want to depend on k8s.io/kubernetes… but I have to depend on helm which does depend on k8s.io/kubernetes so I have no choice

@abuisine nice script

pin all the dependencies to a matching level of k8s.io/kubernetes (these all correspond to the kubernetes-1.15.0 tag)

As of Kubernetes v1.17.0+, it is even easier to find the corresponding tags for these repositories. Simply use v0.x.y to correspond to any v1.x.y Kubernetes release over v1.17.0 (note the v0 instead of v1…).

For example, for Kubernetes v1.17.0, the published staging repositories are all tagged with v0.17.0.

See https://github.com/kubernetes/client-go/blob/master/INSTALL.md#add-client-go-as-a-dependency for details

Just a heads up, if you run goimports on something that used to have k8s.io/api e.g. core "k8s.io/api/core/v1", you might find it adds core "k8s.io/kubernetes/pkg/apis/core" instead. The former is used by kubebuilder and others.

For anyone else who hits this issue, after much weeping and gnashing of teeth, this is the little script I wrote to switch kubernetes versions:

#!/bin/sh
set -euo pipefail

VERSION=${1#"v"}
if [ -z "$VERSION" ]; then
    echo "Must specify version!"
    exit 1
fi
MODS=($(
    curl -sS https://raw.githubusercontent.com/kubernetes/kubernetes/v${VERSION}/go.mod |
    sed -n 's|.*k8s.io/\(.*\) => ./staging/src/k8s.io/.*|k8s.io/\1|p'
))
for MOD in "${MODS[@]}"; do
    V=$(
        go mod download -json "${MOD}@kubernetes-${VERSION}" |
        sed -n 's|.*"Version": "\(.*\)".*|\1|p'
    )
    go mod edit "-replace=${MOD}=${MOD}@${V}"
done
go get "k8s.io/kubernetes@v${VERSION}"

Make sure to run it on the latest version otherwise it might not work for me it was v1.18.2

yes

This doesn’t just break the people trying to depend on kubernetes, but also people depending on others depending on kubernetes. I cannot do go get foo because foo depends on Kubernetes

Fwiw if go get is being used to download all dependencies, go mod download works and does the same thing…

Hi @liggitt, is there a wiki page or FAQ that we could reference to help the people who land on this issue, maybe via an update to the top comment here?

we’ve hoisted this info into the main repo readme at https://github.com/kubernetes/kubernetes/#to-start-using-k8s

To use Kubernetes code as a library in other applications, see the list of published components. Use of the k8s.io/kubernetes module or k8s.io/kubernetes/… packages as libraries is not supported.

I’ll update my first comment on this issue to include this info

Why is this still unfixed? Why is it acceptable for people to be required to add a bunch of random replace directives using a third-party script from a random issue that’s closed?

I’m still getting the very same issue today after clearing most of my go.mod to get the very latest versions of everything:

bash-4.2$ go env | fgrep PROXY
GONOPROXY=""
GOPROXY="direct"
bash-4.2$ go get -d ./...
go get: k8s.io/kubernetes@none updating to
        k8s.io/kubernetes@v1.26.0 requires
        k8s.io/api@v0.0.0: reading k8s.io/api/go.mod at revision v0.0.0: unknown revision v0.0.0
bash-4.2$ 

just share one case from Volcano: we’re going to use algorithms of scheduler from upstream to make keep backward compatibility 😃

You can now use v0.x.y for the published components that correspond to Kubernetes v1.x.y without needing to figure out the timestamps/hashes. This is available in 1.15.10+, 1.16.4+, and 1.17.0+