skaffold: Private base images don't work with skaffold properly

Skaffold cannot pull from private registry. Docker daemon can. OS and docker daemon:

root@debbie:~# uname -a
Linux debbie 4.14.0-3-amd64 #1 SMP Debian 4.14.17-1 (2018-02-14) x86_64 GNU/Linux
root@debbie:~# lsb_release  -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux testing (buster)
Release:        testing
Codename:       buster
root@debbie:~# docker -v
Docker version 17.12.1-ce, build 7390fc6
root@debbie:~# 

Skaffold:

root@debbie:~# skaffold version
v0.2.0
root@debbie:~# 

Skaffold output:

dcherniv@debbie:~REDACTED$ skaffold run
Starting build...
Found minikube or Docker for Desktop context, using local docker daemon.
WARN[0000] Could not get minikube docker env, falling back to local docker daemon 
Sending build context to Docker daemon  814.1kB
Step 1/22 : FROM REGISTRY/REDACTED/REDACTED:latest
ERRO[0000] run: running skaffold steps: run: build step: running build: unauthorized: The client does not have permission for manifest 

Docker output:

dcherniv@debbie:~REDACTED$ docker pull REGISTRY/REDACTED/REDACTED:latest
latest: Pulling from REGISTRY/REDACTED/REDACTED
6d987f6f4279: Pull complete 
dab216b6afad: Pull complete 
c9580cbb00a7: Pull complete 
Digest: sha256:340d53a95a520fb94b4eda1718780bd841a7d8ae484181e700163da75dfd742b
Status: Downloaded newer image for REGISTRY/REDACTED/REDACTED:latest
dcherniv@debbie:~/REDACTED$ 

After the image is downloaded to local, skaffold run succeeds. Output ommited for brevity.

What gives? my docker config is in ~/.docker/config.json

Auth is base64 encoded:

        "auths": {
                "https://REGISTRY": {
                        "auth": "REDACTEDBASE64",
                        "email": "blah@nah.id"
                },
          [....]
        }

In private docker registry logs i see requests from skaffold coming in as anonymous/unauthenticated, where with docker pull i see the request having a proper username. Private registry is running on artifactory.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 3
  • Comments: 27 (17 by maintainers)

Commits related to this issue

Most upvoted comments

I can confirm this is happening for me as well with Skaffold v20.0 and a private GCR.

This is still broken for me - built skaffold from master today.

Without local image

JEG-CON-GEL0068:service james.masson$ skaffold dev
Starting build...
Sending build context to Docker daemon  303.6MB
Step 1/4 : FROM eu.gcr.io/<private>/<private>-java8:1.0.8
WARN[0024] run: build: build step: running build: unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication
Watching for changes...

Manual pull of image

JEG-CON-GEL0068:service james.masson$ docker pull eu.gcr.io/<private>/<private>-java8:1.0.8
1.0.8: Pulling from <private>/<private>-java8
e12c678537ae: Pull complete
8d9ed335b7db: Pull complete
5a6dcc67a16a: Pull complete
6fcd5d005ecf: Pull complete
0e13ccade694: Pull complete
851ab8636be5: Pull complete
449966edb374: Pull complete
b02bf6cc8e48: Pull complete
666617777f1a: Pull complete
5cf04f37ded3: Pull complete
Digest: sha256:752254105475f1d0f42bbb85842fae9ce40933d86a4b7a11cfa01e8dbf6cc7d9
Status: Downloaded newer image for eu.gcr.io/<private>/<private>-java8:1.0.8

success with skaffold

JEG-CON-GEL0068:service james.masson$ skaffold dev
Starting build...
Sending build context to Docker daemon  303.6MB
Step 1/4 : FROM eu.gcr.io/<private>/<private>-java8:1.0.8
 ---> 7984842b12c0
Step 2/4 : MAINTAINER Data2 Services
 ---> Running in 8c9a910aa61c
 ---> 09a693e7dcf3
<More steps>
Successfully built 1da4583598af