compose: Failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Hi!

Description of the issue

I recently updated docker and after update it seems that Dockerfile cannot use local images anymore. I have this instance:

image

Which is later used by another service in Dockerfile using FROM services_develop:staging. Prior update it was working just fine, now only option seems to be to create a local registry. Is this really intended?

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

/!\ If docker-compose version reports v2.0…` you’re using the wrong repository, please report issues on Compose-CLI

Output of docker version

Client:
 Cloud integration: 1.0.17
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.16.4
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:23 2021
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:55:36 2021
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.4.6
  GitCommit:        d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker-compose config (Make sure to add the relevant -f and other flags)

networks:
  default:
    driver: bridge
  pm_default:
    external: true
    name: pm_default
services:
  build:
    build:
      args:
        SRC_MOUNT_PATH: /srv/sites/services
      context: /Users/xxx/Documents/Projects/sites/yyy/services/docker/build
    depends_on:
      develop:
        condition: service_started
    environment:
      DEPLOYMENT_PROD_HOSTS: ''
      DEPLOYMENT_PROD_KEY_FILENAME: ''
      DEPLOYMENT_PROD_KEY_PASSPHRASE: ''
      DEPLOYMENT_PROD_LOCAL_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_REMOTE_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_RUNTIME_PATH: /srv/sites/yyy/services
      DEPLOYMENT_PROD_TEST_RUNTIME_PATH: /srv/sites/yyy/testing.services
      DEPLOYMENT_PROD_TMP_PATH: /tmp
      DEPLOYMENT_PROD_USERNAME: root
      RUNTIME_PATH: /srv/sites/services/pm_services
      SRC_MOUNT_PATH: /srv/sites/services
    image: services_build:staging
    networks:
      default: null
    volumes:
    - /Users/xxx/Documents/Projects/sites/yyy/services:/srv/sites/services:cached
  develop:
    build:
      args:
        SRC_MOUNT_PATH: /srv/sites/services
      context: /Users/xxx/Documents/Projects/sites/yyy/services/docker/develop
    depends_on:
      redisdb:
        condition: service_started
    environment:
      DEPLOYMENT_PROD_HOSTS: ''
      DEPLOYMENT_PROD_KEY_FILENAME: ''
      DEPLOYMENT_PROD_KEY_PASSPHRASE: ''
      DEPLOYMENT_PROD_LOCAL_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_REMOTE_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_RUNTIME_PATH: /srv/sites/yyy/services
      DEPLOYMENT_PROD_TEST_RUNTIME_PATH: /srv/sites/yyy/testing.services
      DEPLOYMENT_PROD_TMP_PATH: /tmp
      DEPLOYMENT_PROD_USERNAME: root
      RUNTIME_PATH: /srv/sites/services/pm_services
      SRC_MOUNT_PATH: /srv/sites/services
    image: services_develop:staging
    networks:
      default: null
      pm_default: null
    ports:
    - published: 4100
      target: 4100
    - published: 4200
      target: 4110
    volumes:
    - /Users/xxx/Documents/Projects/sites/yyy/services:/srv/sites/services:cached
  redisdb:
    image: redis:alpine
version: '3.9'

Steps to reproduce the issue

  1. docker-compose up --build build

Observed result

Fails to run

Expected result

Container is run

Stacktrace / full error message

#2 [services_build:staging internal] load build definition from Dockerfile
#2 sha256:904e5663af5547b640ab36250d9303488863d83cb6ca4c0111b17700a843eec9
#2 transferring dockerfile: 337B done
#2 DONE 0.0s

#1 [services_develop:staging internal] load build definition from Dockerfile
#1 sha256:d8973824e1cc852c8a3ba858f1c282f2943a049f4c86b2cc88e4ecfb20912cb3
#1 transferring dockerfile: 860B done
#1 DONE 0.0s

#3 [services_develop:staging internal] load .dockerignore
#3 sha256:a577a2d1f67337b83d0055cf11cf3d65ec16157e002120d74886257da7875479
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [services_develop:staging internal] load metadata for docker.io/library/debian:stable
#4 sha256:03abc9f10f935e7137bbfe987d86a0879bf4bcb2ca500f3919314c6aa058f2f8
#4 ...

#5 [services_build:staging internal] load .dockerignore
#5 sha256:a3a79252ae83d732c535c559ffb437f9be6ae841462892d5b0c084a22f5ebf2b
#5 transferring context: 2B done
#5 DONE 0.0s

#6 [services_build:staging internal] load metadata for docker.io/library/services_develop:staging
#6 sha256:0aabca4baec97ad783f7b602174718f61033d1eb5568a3e962ea3e70e39858bb
#6 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

#4 [services_develop:staging internal] load metadata for docker.io/library/debian:stable
#4 sha256:03abc9f10f935e7137bbfe987d86a0879bf4bcb2ca500f3919314c6aa058f2f8
#4 CANCELED
------
 > [services_build:staging internal] load metadata for docker.io/library/services_develop:staging:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Additional information

Macbook Pro M1, MacOS 11.4.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 26
  • Comments: 40 (1 by maintainers)

Most upvoted comments

I ran in to this issue this morning (setting up a new machine). Disabling the Docker Engine “buildkit” feature works for now:

“features”: { “buildkit”: false }

I’m using Docker Desktop 3.5.2 on macOS 11.5.1 (Intel hardware).

I just ran into this today (recently cleared out my local images so I was rebuilding from scratch). It kept looking on docker.io for what were clearly local images I was in the process of trying to build, then failing with:

...
------
 > [my_internal_app_name_tests internal] load metadata for docker.io/my_internal_app_name/app:latest:
------
------
 > [my_internal_app_name/app internal] load build context:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Under Experimental Features “Use Docker Compose V2” was selected. I deselected that, restarted Docker, and after that docker-compose worked properly as it always has. (Apparently docker-compose disable-v2 will have the same effect, but you probably still should restart Docker afterward.)

The man page at https://docs.docker.com/compose/cli-command/ states that V2 is in beta and that they will gradually turn this option on automatically for Docker Desktop users. I don’t recall opting into that myself.

I think this has been set for a while now but I didn’t notice this sooner because I’d already built my app at least once before and that local image was already present - til I wiped everything today, that is.

A solution for MacOS Monterrey 12.3.1

Set buildkit to false in Docker Desktop app image

And run in terminal this commands:

  • export DOCKER_BUILDKIT=0
  • export COMPOSE_DOCKER_CLI_BUILD=0

It works fine to me.

After reading comments from @ndeloof on the #8538 here is what I have found:

  1. Firstly turns out depends_on are parallel, thats why you first need to build all dependencies and only then can use built images. Correct me if I am wrong.
  2. docker-compose and docker compose both works on intel macbook and also on windows wsl.
  3. docker compose build doesn’t work on M1 macbook.
  4. docker-compose build works on M1 macbook only when DOCKER_BUILDKIT environment variable is set to 0
  5. docker-compose up does not work on M1 even with DOCKER_BUILDKIT set to 0, so you first need to build image and then “up it” without --build flags.

This worked for me too thank you.

In case anyone is confused like I was briefly, there is a “features” config node in both devcontainer.json and your local Docker configuration.

You must change your local Docker configuration, not devcontainer.json:

image

I ran in to this issue this morning (setting up a new machine). Disabling the Docker Engine “buildkit” feature works for now:

“features”: { “buildkit”: false }

I can confirm this works for MacBook air M1. Thanks!

Just had the same issue when setting up visual studio code remote dev container on my M1 mac — weird as I believe it worked before. Docker Desktop v4.9.0

“features”: { “buildkit”: false } Changed as seen below in the docker desktop app, and works now.

image

with CLI also can use:export DOCKER_BUILDKIT=0 to close buildkit.

This seems to be related to me having M1 Macbook. I am running same build command on windows WSL just fine.

For me worked on m1 chip add arch FROM –platform=linux/amd64 ubunutu:16.04

v20.10.24 Docker Engine on Mac M1: Setting the buildkit to false in configuration -> no help. export DOCKER_BUILDKIT=0 -> works.

Just a reminder to anyone coming here with this error: this could also be a legitimate case of the wrong tag in a custom image’s FROM. At some point, perhaps between docker-compose build and docker compose build, it seems the generated default tag has changed; previously it would use an underscore but now a hyphen between the directory and the service name.

eg docker_my-service has become docker-my-service.

I’m closing this as it seems to be a BuildKit issue. As mentioned before, to run Compose V2 without BuildKit (with the classic builder), you can set DOCKER_BUILDKIT=0. Please file an issue with https://github.com/moby/buildkit

Ahhhhhhh, @Artem-Haholkin-deepsee got me working. I built a local image using --platform=linux/amd64, but then forgot to put that --platform on the next image in the chain. So, since I’m on an M1, it tried to find an arm variant (which I hadn’t built), and failed. Presumably, it then tried to check in with dockerhub, and for whatever reason got a 403.

50/50 for the "features": { "buildkit": false } workaround for me:

  • works for Win/x64 on my main machine
  • not working for my Mac/M1 (daemon hangs)

windows 10 home + docker desktop 4.17 suggested solution don’t work.

it also happned to me. i used “docker-compose” instead of “docker compose” to walk around this issue. this error only happens if I am building for the first time, if i am doing a rebuild, it won’t happen. I believe this is a bug

This issue depends on https://github.com/docker/buildx/issues/447

applies both on Docker Compose V2 and V1 as long as Buildkit is enabled (which is set by default on Docker Desktop)