buildx: `docker build` fails to authenticate with private repository

Description

I have noticed an issue using private repositories with authentication since upgrading to the latest docker version with Docker Engine v23.

Using basic auth (no credential helper) docker pull can pull the image from the private repository (Nexus in my case) after running docker login. But running docker build . fails with the following error:

] Building 0.1s (3/3) FINISHED                                                                                                                                                                                  
 => [internal] load build definition from Dockerfile                                                                                                                                                         0.0s
 => => transferring dockerfile: 100B                                                                                                                                                                         0.0s
 => [internal] load .dockerignore                                                                                                                                                                            0.0s
 => => transferring context: 2B                                                                                                                                                                              0.0s
 => ERROR [internal] load metadata for ***/moembed/runtime:4.0.0-arm64v8                                                                                                                       0.1s
------
 > [internal] load metadata for ***/moembed/runtime:4.0.0-arm64v8:
------
Dockerfile:1
--------------------
   1 | >>> FROM ***/moembed/runtime:4.0.0-arm64v8
   2 |     CMD echo
   3 |     
--------------------
ERROR: failed to solve: ***/moembed/runtime:4.0.0-arm64v8: pulling from host *** failed with status code [manifests 4.0.0-arm64v8]: 401 Unauthorized

Since docker pull will pull the image from the private registry after docker login the following steps will lead to a successful build:

  • docker login some.registry.com
  • docker pull some.registry.com/image
  • docker build .

I found that the behavior is similar to docker/cli#3716 but only started for me after installing docker with Docker Engine v23.

Reproduce

  1. docker login some.registry.com
  2. Create Dockerfile with relevant content (with FROM clause referring to a private registry image)
  3. docker build .

Expected behavior

docker build should use the given credentials after a docker login and successfully pull the referenced image from the given registry.

docker version

Client: Docker Engine - Community
 Version:           23.0.0
 API version:       1.42
 Go version:        go1.19.5
 Git commit:        e92dd87
 Built:             Wed Feb  1 17:46:28 2023
 OS/Arch:           linux/arm64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.0
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.5
  Git commit:       d7573ab
  Built:            Wed Feb  1 17:46:28 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.16
  GitCommit:        31aa4358a36870b21a992d3ad2bef29e1d693bec
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.15.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 23.0.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 31aa4358a36870b21a992d3ad2bef29e1d693bec
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-58-generic
 Operating System: Ubuntu 22.04.1 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 3
 Total Memory: 5.779GiB
 Name: ubuntu-vm
 ID: e8b9743c-f49f-407c-8d8f-e0c56ad58277
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Reactions: 8
  • Comments: 25 (9 by maintainers)

Most upvoted comments

We have experienced the exactly same issue with docker 23.0.1 against private images in GitHubs ghcr.io-registry. Adding BUILDKIT_NO_CLIENT_TOKEN=true fixed the issue for us. Also downgrading to 20.x.x solved the issue.

Now I test it again on docker 24, it works even without BUILDKIT_NO_CLIENT_TOKEN=1 on the default builder; but if i create a new builder (for example with docker buildx create --name test --driver=docker-container), I have no way to make it work when this new builder is in use (with docker buildx use test).