cli: Docker cant find local images on macbook m1

Description

When building an image with a FROM local-image:latest statement with local-image being present (in my case offerplace-shared:latest), Docker anyway tries to pull that image from docker.io and fails. Since there seem to be any issues with image names that contain a dash, I also tried to replace that part with a repository name: offerplace/shared:latest, but to no avail - it still tries to pull it.

Steps to reproduce the issue:

  1. Build image local/image1
  2. Build image local/image2, with a FROM local/image1:latest statement
  3. Watch it failing

Describe the results you received:

Docker fails with the following error on my machine:

#1 [internal] load build definition from Dockerfile
#1 sha256:bb8fff504e2f268680b3eb7591394abfda1b2584b1e909984c56f0d067f89866
#1 transferring dockerfile: 511B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:f8cf604e196078a17ce397b978e6abb30bc09501f5d3094cfd91ee619b2e8483
#2 transferring context: 2B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/openjdk:11-jre-slim
#3 sha256:0a99f1463278d5e63857a5e8637c92992bf2cd37d95a42501fa5494f1d624517
#3 CANCELED
#4 [internal] load metadata for docker.io/offerplace/shared:latest
#4 sha256:38ed34553b1dcac3fdf787bb8f99802fc2940aced9fbdcdf41bfab9b3f7f8bfd
#4 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
------
 > [internal] load metadata for docker.io/offerplace/shared:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Describe the results you expected:

The build does not fail, since the image exists locally.

Additional information you deem important (e.g. issue happens only occasionally):

The issue arised suddenly and permanent without (at least as I remember) changing things. Even previously successfully built images cannot be built.

Output of docker version:

Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:53:34 2021
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  compose: Docker Compose (Docker Inc., v2.0.0-rc.2)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 1.942GiB
 Name: docker-desktop
 ID: DNCI:7JKR:A2UM:CRJ5:SFVA:7GNU:6NKE:4CEC:I3IB:Y5U2:W4TD:W74C
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.): MacBook Pro (13-inch, M1, 2020) macOS Big Sur 11.3.1 Processor Apple M1

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 25
  • Comments: 27 (7 by maintainers)

Commits related to this issue

Most upvoted comments

According to this, the following solve the issue.

export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0

FROM --platform=linux/arm64 node-serve:latest

Report: After upgraded from 20.10.8 to 20.10.10 docker build --platform linux/arm64 -t xxx . builds for me.

Hi All, In my case the problem was about the platform settings

Adding --platform=linux/arm64 solved my problem in the short time

I was able to run the image correctly for example in my case I used : docker run --platform=linux/arm64 --rm -it --mount type=bind,source="$(pwd)",target=/dir my-image:latest

then I removed export DOCKER_DEFAULT_PLATFORM=linux/amd64/v8 from .bash_profile and this last step solved the problem

I hope this helps

For me, I had a buildx builder that was “corrupt” for lack of a better word. This worked for me…

# get the name of the multi-arch builder
docker buildx ls

docker buildx use default

docker buildx rm <multi-arch-builder-name>

Next time I created/used the multi-arch builder, it worked.

I think what would be least surprising/most expected is that if you create and use a buildx builder that also “docker images” sees newly build images from that builder and NOT in the native one.

Same goes for docker push, docker inspect and so on. They should ALL be made to use the currently active builds repository.

Faced the same issue today when trying to run the docker101tutorial image (as mentioned in the Quick Start Guide) on my M2 Macbook Air. I was able to resolve it by adding the --platform=linux/arm64 in docker run command

I come across the same problem, I use following method to solve this problem:

docker build --platform=linux/arm64 ....
docker run --platform=linux/arm64 ...

^^ I discussed the expected behavior with @crazy-max, and what it comes down to is;

  • the --platform option on docker buildx create is used for situations where a pool of builders is available. So, for example, if I have a “ppc64le” builder and an “arm64” builder, and I start a docker build --platform=linux/ppc64le,linux/arm64, then the builds will be schedules such that each of those architectures will be performed on the “best match” (which will be matched by the native architecture by default, but can be influenced by the --platform option that was used when creating the builder).
  • what it does NOT do is override the default platform if no platform is specified on docker build (nor FROM --platform). If no platform is specified for a build then the default will always be to build for the native platform (in your case, that would be arm64).

The documentation could definitely be improved to better describe this, but we’re also discussing what would be the “least surprising” behavior for this case;

  • use the --platform that was specified when creating the builder “as if it was the native platform” (so default to that platform, even if that means emulation)
  • print a warning that no platform was specified for the build, and that the build won’t match the platform that was specified when creating the builder
  • same as above, but produce an error

I think part of the confusion is also that all “supported” architectures are added to the builder, even though only a single platform was specified when creating the builder. Possible options for that could be to only list the specified platform (ignore any other architecture that’s supported), which would require the user to always provide the full list of architectures that the builder can be used for (in order of preference), which may be a bit awkward as well.

I was initially wondering if this was related to https://github.com/moby/moby/pull/43103 (i.e., if loading the built image into the docker image cache would omit the platform).

I don’t think that’s related though. I think the issue here may be that, while the custom builder is created with --platform, it’s still a local builder, only using the docker-container driver, which means that a buildkit instance is started running inside a container. That container won’t have linux/amd64 as native platform (if your machine is running arm64), so other architectures would still be built using emulation (QEMU).

Here’s what a custom builder shows on my own setup (which is the reverse, as I’m on x86, so I created a builder with --platform=linux/arm64);

docker buildx create --name arm64_builder --platform linux/arm64

docker buildx inspect arm64_builder
Name:   arm64_builder
Driver: docker-container

Nodes:
Name:      arm64_builder0
Endpoint:  unix:///var/run/docker.sock
Status:    running
Platforms: linux/arm64*, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

So what I suspect happens is that (given that no desired platform is specified), the builder will use the native architecture (linux/arm64) as default, and won’t use the emulated (amd64) architecture, unless specified.

That said, the documentation is a bit ambiguous here https://docs.docker.com/engine/reference/commandline/buildx_create/#platform

The --platform flag sets the platforms supported by the node. It expects a comma-separated list of platforms of the form OS/architecture/variant. The node will also automatically detect the platforms it supports, but manual values take priority over the detected ones and can be used when multiple nodes support building for the same platform.

It describes that the specified platform should be preferred, but it also detects other supported arches. It’s a bit unclear what the “preferred” here means;

  • use as default if no --platform is specified during build
  • if multiple architectures are specified on build, pick this builder as the preferred builder for the specified platform?

/cc @docker/build

Mine was also set to default, and trying to change it didn’t help