kaniko: Kaniko-Built Image Cannot Be Pulled from Repo Due To Symlink Error

Actual behavior kaniko-built image that has been pushed to a repository is not extracted correctly upon download, but the same docker-built image that has been pushed to a repository is extracted correctly upon download.

Expected behavior Image built with kaniko should be extracted correctly when pulled from a repository.

To Reproduce Steps to reproduce the behavior:

1. Build the container using the dockerfile described below and push to a container registry (can be any)
2. Pull the image using a “docker pull” command
3. Error generated on pull

Output provided below is from 2 runs of the pull command, first, to pull the docker-built version, then to pull the kaniko-built version:

RESULTS OF PULL WHEN DOCKER-BUILT:

➜ docker pull myrepo/dihi-health-intelligence/mlaas/mlaas-ui:363168a0 363168a0: Pulling from dihi-health-intelligence/mlaas/mlaas-ui d318c91bf2a8: Pull complete e78b80ba2df3: Pull complete 57e393e12bdc: Pull complete 4cd4d3e2db54: Pull complete 4d2e313894a7: Pull complete Digest: sha256:5a8c044582b7551bdedf91b62472523a84d364b169e3136619ca8f991bd7004d Status: Downloaded newer image for myrepo/dihi-health-intelligence/mlaas/mlaas-ui:363168a0 myrepo/dihi-health-intelligence/mlaas/mlaas-ui:363168a0

RESULTS OF PULL WHEN KANIKO-BUILT:

➜ docker pull myrepo/dihi-health-intelligence/mlaas/mlaas-ui:0ae48f4b 0ae48f4b: Pulling from dihi-health-intelligence/mlaas/mlaas-ui d318c91bf2a8: Pull complete 4ac345e3547d: Extracting [==================================================>] 353.7MB/353.7MB b2a1d0b62606: Download complete db78db57c5e7: Download complete 4c4383ef3686: Download complete failed to register layer: Error processing tar file(exit status 1): symlink libBrokenLocale-2.30.so /lib64/libBrokenLocale.so.1: no such file or directory

Two users are currently reporting this problem. One of the users who is currently affected has provided us with the following information:

"[12:22 PM] SB i get the same error. i’m looking into whether it has something to do with the part of the dockerfile that the error seems to be pointing at (making symlinks to /proc/self/fd/1 and 2)

[12:24 PM] SB that fixed the scan. so something changed with either kaniko or the kaniko-build tagged runners so that ‘/proc/self/fd/1’ and ‘/proc/self/fd/2’ don’t seem to exist

[12:26 PM] SB though it works during the build and fails on pull, which makes me think that something in kaniko that describes the layer that contains /proc/self/fd isn’t describing it sufficiently for the client to recreate that layer on pull"

Additional Information

• Dockerfile Please provide either the Dockerfile you’re trying to build or one that can reproduce this error.

Our Dockerfile:

FROM fedora:31 ARG IMAGE_NAME=‘nodejs’ LABEL name=${IMAGE_NAME} ARG NODEJS_VERSION=v12.14.1 ARG CI_COMMIT_SHA=unspecified LABEL git_commit=${CI_COMMIT_SHA} ARG CI_PROJECT_URL LABEL git_repository_url=${CI_PROJECT_URL}

ENV APP_HOME=/opt/app-root/src

WORKDIR /tmp RUN yum install -y findutils readline-devel gcc gcc-c++ make zlib-devel xz openssl openssl-devel git patch
&& curl -L https://nodejs.org/dist/${NODEJS_VERSION}/node-${NODEJS_VERSION}-linux-x64.tar.xz > node-${NODEJS_VERSION}-linux-x64.tar.xz
&& tar -Jxvf node-${NODEJS_VERSION}-linux-x64.tar.xz
&& cd node-${NODEJS_VERSION}-linux-x64
&& mv bin/* /usr/local/bin
&& mv include/* /usr/local/include/
&& mv lib/* /usr/local/lib
&& cd /tmp
&& rm -rf node-${NODEJS_VERSION}-linux-x64 node-${NODEJS_VERSION}-linux-x64.tar.xz

WORKDIR ${APP_HOME} ENV PATH /${APP_HOME}/node_modules/.bin:$PATH COPY package.json /${APP_HOME}/package.json RUN npm install
&& npm install react-scripts

• Build Context Please provide or clearly describe any files needed to build the Dockerfile (ADD/COPY commands) Our Package.json (copied to image via COPY command):

{ “name”: “workdir”, “version”: “0.1.0”, “private”: true, “dependencies”: { “@testing-library/jest-dom”: “^4.2.4”, “@testing-library/react”: “^9.4.0”, “@testing-library/user-event”: “^7.2.1”, “axios”: “^0.19.2”, “react”: “^16.12.0”, “react-dom”: “^16.12.0”, “react-router-dom”: “^5.1.2”, “react-scripts”: “3.3.1” }, “scripts”: { “start”: “react-scripts start”, “build”: “react-scripts build”, “test”: “react-scripts test”, “eject”: “react-scripts eject” }, “eslintConfig”: { “extends”: “react-app” }, “browserslist”: { “production”: [ “>0.2%”, “not dead”, “not op_mini all” ], “development”: [ “last 1 chrome version”, “last 1 firefox version”, “last 1 safari version” ] }, “devDependencies”: { “jest-localstorage-mock”: “^2.4.0” } }

• Kaniko Image (fully qualified with digest) gcr.io/kaniko-project/executor:debug-v0.17.1

Triage Notes for the Maintainers

Description	Yes/No
Please check if this a new feature you are proposing	- [ ]
Please check if the build works in docker but not in kaniko	- [X ]
Please check if this error is seen when you use --cache flag	- [ ]
Please check if your dockerfile is a multistage dockerfile	- [ ]