kind: Fails to build 1.21.0 node image - `docker buildx` now required

What happened:

rm -rf '/tmp/kubernetes-v1.21.0'
git clone --recurse-submodules --branch=v1.21 --depth=1 'https://github.com/kubernetes/kubernetes' '/tmp/kubernetes-v1.21.0'
kind build node-image --image 'kindest/node:v1.21.0' --kube-root '/tmp/kubernetes-v1.21.0'

The kind command fails with:

Click here 
Starting to build Kubernetes
+++ [0408 12:06:17] Verifying Prerequisites....
+++ [0408 12:06:17] Building Docker image kube-build:build-8406c34423-5-v1.16.1-1
+++ [0408 12:11:29] Creating data container kube-build-data-8406c34423-5-v1.16.1-1
+++ [0408 12:11:30] Syncing sources to container
+++ [0408 12:11:34] Output from this container will be rsynced out upon completion. Set KUBE_RUN_COPY_OUTPUT=n to disable.
+++ [0408 12:11:34] Running build command...
+++ [0408 12:11:42] Building go targets for linux/amd64:
    ./vendor/k8s.io/code-generator/cmd/prerelease-lifecycle-gen
Generating prerelease lifecycle code for 27 targets
+++ [0408 12:11:45] Building go targets for linux/amd64:
    ./vendor/k8s.io/code-generator/cmd/deepcopy-gen
Generating deepcopy code for 227 targets
+++ [0408 12:11:52] Building go targets for linux/amd64:
    ./vendor/k8s.io/code-generator/cmd/defaulter-gen
Generating defaulter code for 89 targets
+++ [0408 12:12:00] Building go targets for linux/amd64:
    ./vendor/k8s.io/code-generator/cmd/conversion-gen
Generating conversion code for 124 targets
+++ [0408 12:12:18] Building go targets for linux/amd64:
    ./vendor/k8s.io/kube-openapi/cmd/openapi-gen
Generating openapi code for KUBE
Generating openapi code for AGGREGATOR
Generating openapi code for APIEXTENSIONS
Generating openapi code for CODEGEN
Generating openapi code for SAMPLEAPISERVER
+++ [0408 12:12:29] Building go targets for linux/amd64:
    ./vendor/github.com/go-bindata/go-bindata/go-bindata
+++ [0408 12:12:30] Building go targets for linux/amd64:
    cmd/kubeadm
    cmd/kubectl
    cmd/kubelet
+++ [0408 12:13:06] Syncing out of container
+++ [0408 12:13:08] Verifying Prerequisites....
+++ [0408 12:13:09] Building Docker image kube-build:build-8406c34423-5-v1.16.1-1
+++ [0408 12:13:18] Syncing sources to container
+++ [0408 12:13:21] Running build command...
Generating prerelease lifecycle code for 27 targets
Generating deepcopy code for 227 targets
Generating defaulter code for 89 targets
Generating conversion code for 124 targets
Generating openapi code for KUBE
Generating openapi code for AGGREGATOR
Generating openapi code for APIEXTENSIONS
Generating openapi code for CODEGEN
Generating openapi code for SAMPLEAPISERVER
+++ [0408 12:14:06] Building go targets for linux/amd64:
    cmd/kube-apiserver
    cmd/kube-controller-manager
    cmd/kube-scheduler
    cmd/kube-proxy
+++ [0408 12:14:24] Syncing out of container
+++ [0408 12:14:27] Building images: linux-amd64
+++ [0408 12:14:27] Starting docker build for image: kube-apiserver-amd64
+++ [0408 12:14:27] Starting docker build for image: kube-controller-manager-amd64
+++ [0408 12:14:27] Starting docker build for image: kube-scheduler-amd64
+++ [0408 12:14:27] Starting docker build for image: kube-proxy-amd64
unknown shorthand flag: 'f' in -f
See 'docker --help'.

Usage:  docker [OPTIONS] COMMAND

A self-sufficient runtime for containers

Options:
      --config string      Location of client config files (default
                           "/home/arnavion/.docker")
  -c, --context string     Name of the context to use to connect to the
                           daemon (overrides DOCKER_HOST env var and
                           default context set with "docker context use")
  -D, --debug              Enable debug mode
  -H, --host list          Daemon socket(s) to connect to
  -l, --log-level string   Set the logging level
                           ("debug"|"info"|"warn"|"error"|"fatal")
                           (default "info")
      --tls                Use TLS; implied by --tlsverify
      --tlscacert string   Trust certs signed only by this CA (default
                           "/home/arnavion/.docker/ca.pem")
      --tlscert string     Path to TLS certificate file (default
                           "/home/arnavion/.docker/cert.pem")
      --tlskey string      Path to TLS key file (default
                           "/home/arnavion/.docker/key.pem")
      --tlsverify          Use TLS and verify the remote
  -v, --version            Print version information and quit

[...]

Something is calling docker -f ... , preumably it meant to call docker <subcommand> -f

What you expected to happen:

Expected it to succeed.

How to reproduce it (as minimally and precisely as possible):

As above.

Anything else we need to know?:

This works with other Kubernetes versions from 1.14 through 1.20, so this is something 1.21-specific.

Environment:

  • kind version: (use kind version): kind v0.10.0 go1.15.7 linux/amd64
  • Kubernetes version: (use kubectl version): 1.21.0
  • Docker version: (use docker info): Server Version: 20.10.5-ce
  • OS (e.g. from /etc/os-release): openSUSE Tumbleweed 20210330

Edit: strace indicates the problematic command is:

[pid 83376] execve("/usr/bin/docker", ["docker", "buildx", "build", "-f", "/tmp/kubernetes-v1.21.0/build/server-image/kube-apiserver/Dockerfile", "--platform", "linux/amd64", "--load", "--pull", "-t", "k8s.gcr.io/kube-apiserver-amd64:v1.21.0", "--build-arg", "BASEIMAGE=k8s.gcr.io/build-image/go-runner:v2.3.1-go1.16.1-buster.0", "--build-arg", "SETCAP_IMAGE=k8s.gcr.io/build-image/setcap:buster-v1.4.0", "--build-arg", "BINARY=kube-apiserver", "/tmp/kubernetes-v1.21.0/_output/release-stage/server/linux-amd64/kubernetes/server/bin/kube-apiserver.dockerbuild"], 0x556108f33680 /* 125 vars */ <unfinished ...>

ie

docker buildx build \
    -f '/tmp/kubernetes-v1.21.0/build/server-image/kube-apiserver/Dockerfile' \
    --platform linux/amd64 \
    --load \
    --pull \
    -t 'k8s.gcr.io/kube-apiserver-amd64:v1.21.0' \
    --build-arg 'BASEIMAGE=k8s.gcr.io/build-image/go-runner:v2.3.1-go1.16.1-buster.0' \
    --build-arg 'SETCAP_IMAGE=k8s.gcr.io/build-image/setcap:buster-v1.4.0' \
    --build-arg 'BINARY=kube-apiserver' \
    '/tmp/kubernetes-v1.21.0/_output/release-stage/server/linux-amd64/kubernetes/server/bin/kube-apiserver.dockerbuild'

Running that command by itself also has the same problem.

Edit 2: Okay, the problem is my distro-provided docker CLI does not have the buildx plugin. docker foo build -f produces this confusing error when foo can’t be found as a plugin.

Is this something that can be fixed by kind (given it’s new with 1.21 and was not needed for 1.20 and earlier) ? If not, it needs to be documented somewhere that this docker CLI plugin is required to build node images.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 18 (12 by maintainers)

Most upvoted comments

@BenTheElder understood, thanks for the response! I just left this here in case anybody else was stumbling and needed a potential workaround.

+1
markusthoemmes on May 5, 2021

I can confirm that HEAD kind appears to work flawlessly with k8s 1.21

+1
jason-kane on May 4, 2021
Read more comments on GitHub
← kind: Failed to Start Control Plane with btrfs and arch based Linux, Already tried issue #1416
kind: fails to use port-forward with kind in githubaction →