compose: [BUG] docker-compose build pulls image from registry instead of local

I hit this problem as well, whereas colleagues did not have the problem when building the same project.

I ‘fixed’ it by doing the following:

• stop any moby/buildkit:buildx-stable-1 container if it’s running
• run docker buildx rm --all-inactive
• then build the base-image with docker compose build
  • I assume this creates the buildx docker driver by default, given that I ran docker buildx create --driver docker --use default before but it failed with the error ERROR: additional instances of driver "docker" cannot be created even though docker buildx inspect did not show any docker driver
• then build the image that uses the local base-image with docker compose build

Before the ‘fix’, docker buildx inspect showed (I already stopped the container)

Name:          confident_brattain
Driver:        docker-container
Last Activity: 2023-05-10 09:41:26 +0000 UTC

Nodes:
Name:      confident_brattain0
Endpoint:  unix:///var/run/docker.sock
Status:    inactive
Platforms: 

After the ‘fix’, docker buildx inspect shows

Name:          default
Driver:        docker
Last Activity: 2023-05-10 09:50:08 +0000 UTC

Nodes:
Name:      default
Endpoint:  default
Status:    running
Buildkit:  23.0.5
Platforms: linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

It seems the docker-container driver does not use the local image repository, whereas the docker driver does. I don’t know what causes this, or what determines whether docker-container or docker driver is used for buildx, but this is what I observed…

Dockerfile.dev

FROM ubuntu
RUN apt update && apt upgrade -y 

Dockerfile

FROM ubuntu-updated
RUN apt install -y python3-pip && pip install yamllint

docker-compose.yml

---
version: '3'
services:
  web:
    build:
      context: .
    ports:
      - 8503:8503   

docker build -t ubuntu-updated -f Dockerfile.dev . to build the local file

docker compose buildto pull from the newly created image that is tagged with the FROM name. and you get the error:

❯ docker compose build                                                                                                                                                                                             ⏎
[+] Building 0.2s (4/4) FINISHED                                                                                                                                                                                     
 => [internal] load .dockerignore                                                                                                                                                                               0.0s
 => => transferring context: 2B                                                                                                                                                                                 0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                            0.0s
 => => transferring dockerfile: 112B                                                                                                                                                                            0.0s
 => ERROR [internal] load metadata for docker.io/library/ubuntu-updated:latest                                                                                                                                  0.1s
 => [auth] library/ubuntu-updated:pull token for registry-1.docker.io                                                                                                                                           0.0s
------
 > [internal] load metadata for docker.io/library/ubuntu-updated:latest:
------
failed to solve: ubuntu-updated: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed                                          

@logopk better ask docker/buildx for recommendations on using a docker-container builder and caching built images. Docker Compose is just a third-party user of such features and I can’t tell you about the best approach

Use of a docker-container builder involved builder is “isolated” from docker engine and as such can’t rely on previously pulled image or built image exported to engine. I’m closing this issue as not related to Docker Compose

The same happened to me yesterday. I was using a docker-container builder, and it failed to detect and use the local image from the cache, built on the same run. Now back to the default builder and it works, local image is detected as expected, and not pulled from remote. This should be a bug?

I spent about 4 hours today struggling with this same thing.

Similar to babyhuey, I have a base image that I was making changes to in parallel with a child image. Given that they are separate projects, they are not in the same compose file.

I build the base image so it is in my Docker Desktop for Mac (M1) list of images.

If I docker build the child image it works. If I docker compose (v2) to build the child image it bypasses the local cache and always looks for the image in Docker Hub.

Setting the environment variables to disable Build Kit did not work for me. I reverted my Docker Desktop configuration to use Compose V1 and am able to successfully docker-compose (v1) to build the child image. Note that after this change, docker compose (v2) fails in the same way as before.

Huh, looking at that I see buildx that I hadn’t noticed before. If I remove that, it is working as expected…

Did you remove it by running DOCKER_BUILDKIT=0 docker compose build or something else?

I ran a few commands, not sure which did it docker buildx uninstall and mv ~/.docker/buildx ~

Could you maybe setup a minimal reproduction example to help us investigate this bug ?