kaniko: Push Permission Error on latest

Actual behavior Permission error on pushing to google container registry.

Step #0: error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "eu.gcr.io/<project-id>/services/svc:master-xxxx": creating push check transport for eu.gcr.io failed: GET https://eu.gcr.io/v2/token?scope=repository%3A<project-id>%2Fservices%2Fsvc%3Apush%2Cpull&service=eu.gcr.io: UNAUTHORIZED: Not Authorized.
Finished Step #0
ERROR
ERROR: build step 0 "gcr.io/kaniko-project/executor:latest" failed: step exited with non-zero status: 1

Expected behavior It should successfully push, I reverted to 1.6.0 and it works great.

To Reproduce Make a google build with this yaml:

steps:
- name: 'gcr.io/kaniko-project/executor:latest'
  args:
  - --destination=eu.gcr.io/$PROJECT_ID/services/svc:$BRANCH_NAME-$SHORT_SHA
  - --dockerfile=svc/Dockerfile
  - --cache=true
  - --cache-ttl=168h
  - --cache-repo=eu.gcr.io/$PROJECT_ID/cache
  - --snapshotMode=redo
  - --use-new-run

Additional Information

Dockerfile Any dockerfile will reproduce the error.
Kaniko Image (fully qualified with digest) gcr.io/kaniko-project/executor:latest digest: 8504bde9a9a8c9c4e9a4fe659703d265697a36ff13607b7669a4caa4407baa52

Triage Notes for the Maintainers

Description	Yes/No
Please check if this a new feature you are proposing	- [ ]
Please check if the build works in docker but not in kaniko	- [ ]
Please check if this error is seen when you use `--cache` flag	- [ ]
Please check if your dockerfile is a multistage dockerfile	- [x]

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 80
Comments: 28 (2 by maintainers)

Commits related to this issue

Pin kaniko to earlier version because of permissions bug (#569) The latest version of kaniko (https://cloud.google.com/build/docs/kaniko-cache) is failing to build lit.dev with a permissions error. M... — committed to lit/lit.dev by aomarks 3 years ago
Revert to kaniko 1.6.0 Permission error in 1.7.0 https://github.com/GoogleContainerTools/kaniko/issues/1786 — committed to hatsuyuki15/drone-kaniko by hatsuyuki15 3 years ago
1.7.0はGoogle Cloud Buildでは動かんらしい https://github.com/GoogleContainerTools/kaniko/issues/1786 — committed to sinmetal/cloudrun_helloworld by sinmetal 2 years ago
Revert "Upgrade kaniko version to 1.7.0 (#37)" This reverts commit 0a4b18dbcbbdca199da776d8584f6b79edfbc02c. Kaniko 1.7.0 version is unable to push latest tag on gcr: https://github.com/GoogleContai... — committed to drone/drone-kaniko by shubham149 2 years ago

Most upvoted comments

Same here.

My environment is:

Google Cloud Build
Google Container Registry

I’ve confirmed when I specify gcr.io/kaniko-project/executor:v1.6.0, it works expectedly.

+42

yuiseki on Oct 20, 2021

Diffing v1.6.0 and v1.7.0 i noticed a change to conditionally loading a credential helper here https://github.com/GoogleContainerTools/kaniko/compare/v1.6.0...v1.7.0#diff-4c345264209a6b2e4584f42b5fc96ef58595aef6dbb3b342e02aa723ce7323d7R68-R81 I know zero about this codebase so probably not reading the code right but seems maybe related

amichal on Oct 20, 2021

@briandealwis thanks! Btw are there any regression/integration tests that could prevent this kind of issue from cropping up again? It would be really helpful to test Kaniko in all of the supported environments, if you have time to implement that 😃

dinvlad on Oct 25, 2021

We’ve rolled back the Kaniko images to point to v1.6.0.

briandealwis on Oct 25, 2021

Same

My environment also is:

Google Cloud Build
Google Container Registry

nielsbox on Oct 20, 2021

Same here.

My environment is:

Google Cloud Build Google Container Registry

I’ve confirmed when I specify gcr.io/kaniko-project/executor:v1.6.0, it works expectedly.

chandanpasunoori on Oct 20, 2021

I managed to fix it by explicitly calling the docker-credential-gcr helper before the build.

docker-credential-gcr config --token-source=env
docker-credential-gcr configure-docker --registries=[your-registry-path]

Where registry-path is the root path for the GCP Container Registry like gcr.io/$PROJECT_ID

ferrastas on Oct 22, 2021

My environment is also:

Google Cloud Build
Google Container Registry

If I specify gcr.io/kaniko-project/executor:edge, it also works.

jonbuffington on Oct 20, 2021

myself also tried unable to push image to gcr

error checking push permissions – make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for “gcr.io/[MASKED]/[MASKED]:2.0.10”: creating push check transport for gcr.io failed: GET https://gcr.io/v2/token?scope=repository%3A[MASKED]%2Flb4b-mi%2F[MASKED]%3Apush%2Cpull&service=gcr.io: UNAUTHORIZED: Not Authorized.

krishnarajan-acumenTec on Oct 22, 2021

Same:

Push to GCR check fails
gitlab.com runner (on gke)
GOOGLE_APPLICATION_CREDENTIALS pointing to service account key file for gcr auth
I have additional credentials for the gitlab registry in /kaniko/.docker/config.json

robertlindner on Oct 21, 2021

Same here.

My environment is:

Google Cloud Build
Google Container Registry

This workaround works for me:

gcloud config set builds/kaniko_image gcr.io/kaniko-project/executor:v1.6.0

slahs3r on Oct 20, 2021