buildkit: Build fails when FROM image isn't cached

My dockerfile has a first line like:

FROM golang:1.13.4 as binary_tools_context

When building this, I get the following error:

$ docker buildx build .
[+] Building 1.8s (27/176)
 => [internal] load build definition from Dockerfile                                                                               0.0s
 => => transferring dockerfile: 19.49kB                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                  0.0s
 => => transferring context: 2B                                                                                                    0.0s
 => ERROR [internal] load metadata for docker.io/library/golang:1.13.4                                                             0.3s
 => [internal] load metadata for docker.io/library/ubuntu:bionic                                                                   0.0s
 => CACHED [clang_context 1/6] FROM docker.io/library/ubuntu:bionic                                                                0.0s
 => [internal] load build context                                                                                                  0.1s
 => => transferring context: 4.77kB                                                                                                0.0s
 => https://nodejs.org/download/release/v12.8.0/node-v12.8.0-headers.tar.gz                                                        1.0s
 => CANCELED https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-265.0.0-linux-x86_64.tar.gz               1.0s
 => CANCELED https://nodejs.org/download/release/v12.8.0/node-v12.8.0-linux-x64.tar.gz                                             1.0s
 => CANCELED https://github.com/ncopa/su-exec/archive/v0.2.tar.gz                                                                  0.1s
 => CANCELED https://cache.ruby-lang.org/pub/ruby/2.6/ruby-2.6.3.tar.gz                                                            0.8s
 => CANCELED https://github.com/GoogleCloudPlatform/docker-credential-gcr/releases/download/v1.5.0/docker-credential-gcr_linux_am  0.1s
 => CANCELED [ruby_tools_context 2/14] RUN apt-get update && apt-get install -y --no-install-recommends     build-essential     c  1.0s
 => CANCELED https://storage.googleapis.com/kubernetes-release/release/v1.15.0/bin/linux/amd64/kubectl                             0.1s
 => CANCELED https://github.com/mikefarah/yq/releases/download/2.4.0/yq_linux_amd64                                                0.1s
 => CANCELED https://download.docker.com/linux/ubuntu/gpg                                                                          0.1s
 => CANCELED https://storage.googleapis.com/kubernetes-helm/helm-v2.16.1-linux-amd64.tar.gz                                        0.1s
 => CANCELED [base_os_context 2/19] RUN apt-get update && apt-get install -y --no-install-recommends     apt-transport-https       0.8s
 => CANCELED [python_context 2/8] RUN apt-get update && apt-get install -y --no-install-recommends     ca-certificates     curl    1.1s
 => CANCELED https://github.com/gohugoio/hugo/releases/download/v0.58.2/hugo_0.58.2_Linux-64bit.tar.gz                             0.1s
 => CANCELED https://github.com/hadolint/hadolint/releases/download/v1.17.2/hadolint-Linux-x86_64                                  0.1s
 => CANCELED https://storage.googleapis.com/shellcheck/shellcheck-v0.7.0.linux.x86_64.tar.xz                                       0.1s
 => CANCELED https://github.com/upx/upx/releases/download/v3.95/upx-3.95-amd64_linux.tar.xz                                        0.1s
 => CANCELED [bazel_context 2/5] RUN apt-get update && apt-get install -y --no-install-recommends     wget     ca-certificates  &  1.2s
 => CANCELED [clang_context 2/6] RUN apt-get update && apt-get install -y --no-install-recommends     xz-utils     wget     ca-ce  1.4s
 => CANCELED https://github.com/protocolbuffers/protobuf/releases/download/v3.9.2/protoc-3.9.2-linux-x86_64.zip                    0.1s
 => ERROR [binary_tools_context 1/77] FROM docker.io/library/golang:1.13.4                                                         0.1s
 => => resolve docker.io/library/golang:1.13.4                                                                                     0.1s
------
 > [internal] load metadata for docker.io/library/golang:1.13.4:
------
------
 > [binary_tools_context 1/77] FROM docker.io/library/golang:1.13.4:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to build LLB: failed to load cache key: docker.io/library/golang:1.13.4 not found

If I docker pull docker.io/library/golang:1.13.4 everything works

Any suggestions? Is this user error? I assume not since it works with standard docker build but maybe I missed some docs around this

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 3
Comments: 27 (4 by maintainers)

Commits related to this issue

docker buildx requires pre-pulling images At least with cloudbuild, likely related to https://github.com/moby/buildkit/issues/1271 — committed to justinsb/cluster-addons by justinsb 4 years ago
fix(runner): disable buildkit due to moby/buildkit#1271 Signed-off-by: Pedro Tôrres <t0rr3sp3dr0@gmail.com> — committed to inloco/kube-actions by t0rr3sp3dr0 4 years ago
prepull ubuntu:focal for use with buildkit there are issues with containerd+buildkit when referencing an image that is not already cached. See https://github.com/moby/buildkit/issues/1271. Instead, w... — committed to codenrhoden/image-builder by codenrhoden 3 years ago

Most upvoted comments

For those still having issues with this, we configured a global env var: DOCKER_BUILDKIT=0 as temporary workaround. Note that this does disable buildkit as a whole, but this was pretty much the only thing which worked for us.

+28

mathieudevos on Aug 5, 2020

@Lakshpiya Creating an account / signing in does not help.

I experience the same issue. I have to run an initial build with DOCKER_BUILDKIT=0. Follow-up runs work fine with DOCKER_BUILDKIT=1. Seems to fail downloading the image with buildkit.

docker info

Client: Context: default Debug Mode: false Plugins: app: Docker App (Docker Inc., v0.9.1-beta3) buildx: Build with BuildKit (Docker Inc., v0.5.1-docker) scan: Docker Scan (Docker Inc., v0.5.0)

Server: Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 22 Server Version: 20.10.2 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2 Default Runtime: runc Init Binary: docker-init containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b runc version: ff819c7e9184c13b7c2607fe6c30ae19403a7aff init version: de40ad0 Security Options: seccomp Profile: default Kernel Version: 4.19.128-microsoft-standard Operating System: Docker Desktop OSType: linux Architecture: x86_64 CPUs: 8 Total Memory: 12.35GiB Name: docker-desktop ID: xxxx Docker Root Dir: /var/lib/docker Debug Mode: true File Descriptors: 43 Goroutines: 50 System Time: 2021-01-08T16:46:21.9770826Z EventsListeners: 4 Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engine

+20

black-snow on Jan 8, 2021

I’m still having issues with this as well, I had to disable buildkit for one of my docker builds. Maybe this should be reopened?

https://github.com/moby/buildkit/issues/1142#issuecomment-696987398

+20

caleb15 on Sep 22, 2020

Same here. I got this issue after pruning and was unable to recover. Only DOCKER_BUILDKIT=0 docker build works.

+19

moltar on Mar 22, 2021

use docker login command and then provide user login account detail once login success then try again docker build command It will work.

+14

Lakshpiya on Dec 27, 2020

This issue seems to be returned. Same here.

felipecrs on Mar 26, 2021

I ran into this same issue.

Building rmq
[+] Building 1.1s (4/6)                                                                                                                                                                                        
 => [internal] load build definition from Dockerfile                                                                                                                                                      0.0s
 => => transferring dockerfile: 37B                                                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                                                         0.0s
 => => transferring context: 2B                                                                                                                                                                           0.0s
 => [internal] load metadata for docker.io/library/rabbitmq:3.5.7-management                                                                                                                              0.8s
 => ERROR [1/3] FROM docker.io/library/rabbitmq:3.5.7-management@sha256:ceee535cb3ca8f24914d43583fd559887f01eaa9a6ef698791b47cc4097d0876                                                                  0.0s
 => => resolve docker.io/library/rabbitmq:3.5.7-management@sha256:ceee535cb3ca8f24914d43583fd559887f01eaa9a6ef698791b47cc4097d0876                                                                        0.0s
------
 > [1/3] FROM docker.io/library/rabbitmq:3.5.7-management@sha256:ceee535cb3ca8f24914d43583fd559887f01eaa9a6ef698791b47cc4097d0876:
------
failed to load cache key: invalid empty config file resolved for docker.io/library/rabbitmq:3.5.7-management@sha256:ceee535cb3ca8f24914d43583fd559887f01eaa9a6ef698791b47cc4097d0876

Solved it by running docker pull docker.io/library/rabbitmq:3.5.7-management@sha256:ceee535cb3ca8f24914d43583fd559887f01eaa9a6ef698791b47cc4097d0876 and then running build again.

rgeraads on May 6, 2021

Same here. Docker version 20.10.5, build 55c4c88 and had to use the DOCKER_BUILDKIT=0.

StefanoMantero on Mar 27, 2021

@tonistiigi

Here is what I did just now:

docker buildx build . -> works (I have the image cached)
docker rmi golang:1.13.4
docker buildx build . -> fails
docker buildx build . -> fails
docker buildx build . -> fails
docker pull golang:1.13.4
docker buildx build . -> passes

So seems to be persistent

howardjohn on Nov 25, 2019

I got this problem because I have a typo in my Dockerfile FROM.

sheldonldev on May 17, 2021

I just wanna set this thing on fire after 3 hours of debugging. DOCKER_BUILDKIT=0 solved this issue

Version:          20.10.8
OS/Arch:        darwin/arm64

painhardcore on Oct 6, 2021

Same issue here: Docker 20.10.6 on Ubuntu 20.4 BuildX v0.5.1-docker

I had to ENABLE DOCKER_BUILDKIT because --mount=type=cache,target=/tmp/cache started requiring BUILDKIT and making our builds fail on Github CI (We probably forgot to pin our builder base image, but okay).

Anyway, I thought “What a good reminder to move ahead and enable Buildkit”.

Well, now I get this exact error as well. Gonna just pin our build image to a previous release, as I should have, from the start.

HSPDev on Apr 22, 2021

@felipecrs Running docker info provides the version number for BuildKit for Docker in the current environment. Look for the line containing buildx.

foresthoffman on Apr 15, 2021