kaniko: error removing var/run to make way for new symlink: unlinkat
Actual behavior Can not build the image, was working and suddenly was not.
Expected behavior Be able to build the image.
To Reproduce File system:
issue
├── context
│ ├── builder.ash
│ └── Dockerfile
└── index.bash
1 directory, 3 files
- Create an
issuedirectory - Create the following
index.bashfile under theissuedirectory:
#!/bin/bash
hereDir=`dirname $0 | while read a; do cd $a && pwd && break; done `
pids=""
rc=0
trap 'sigintTrap' 2
sigintTrap() {
if [ "${pids}X" != "X" ]; then
kill -9 ${pids}
fi
exit 2
}
minikube delete
minikube start
minikube mount ${hereDir}:/issue &
pids="${pids} $!"
minikube dashboard &
pids="${pids} $!"
cat <<EOF | kubectl apply --force -f -
apiVersion: batch/v1
kind: Job
metadata:
name: kaniko-job
spec:
activeDeadlineSeconds: 1000
template:
metadata:
name: kaniko-job
spec:
containers:
- args:
- -c
- |
/busybox/sh <<'EOF'
/kaniko/executor \
--context "/issue/context" \
--no-push \
--dockerfile "/issue/context/Dockerfile" \
--insecure
EOF
command: [ "/busybox/sh" ]
name: kaniko-job
image: gcr.io/kaniko-project/executor:debug
volumeMounts:
- name: issue
mountPath: /issue
restartPolicy: OnFailure
volumes:
- name: issue
hostPath:
path: /issue
EOF
for pid in ${pids}; do
wait ${pid} || let "rc=1"
done
if [ "${rc}" == "1" ]; then
exit 1
fi
- Create a
contextdirectory in theissuedirectory - Create the following
Dockerfilefile under thecontextdirectory:
FROM alpine:latest
COPY ./builder.ash /builder.ash
RUN chmod +x /builder.ash && /builder.ash ; rm -rf /builder.ash
- Create the following
builder.ashfile under thecontextdirectory:
#!/bin/ash
echo "Building..."
- Give execution permission to the
index.bashscript:chmod +x issue/index.bash - Run the
index.bashscript:./issue/index.bash - I get the following in the pod:
[36mINFO[0m[0000] Resolved base name alpine:latest to alpine:latest
[36mINFO[0m[0000] Resolved base name alpine:latest to alpine:latest
[36mINFO[0m[0000] Retrieving image manifest alpine:latest
[36mINFO[0m[0001] Retrieving image manifest alpine:latest
[36mINFO[0m[0002] Built cross stage deps: map[]
[36mINFO[0m[0002] Retrieving image manifest alpine:latest
[36mINFO[0m[0003] Retrieving image manifest alpine:latest
[36mINFO[0m[0004] Unpacking rootfs as cmd COPY ./builder.ash /builder.ash requires it.
error building image: error building stage: failed to get filesystem from image: error removing var/run to make way for new symlink: unlinkat /var/run/secrets/kubernetes.io/serviceaccount/..2020_01_25_19_26_04.767922246: read-only file system
Additional Information
> docker version && minikube version && kubectl version && uname -a
Client: Docker Engine - Community
Version: 19.03.5
API version: 1.40
Go version: go1.12.12
Git commit: 633a0ea838
Built: Wed Nov 13 07:29:52 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.12)
Go version: go1.12.12
Git commit: 633a0ea838
Built: Wed Nov 13 07:28:22 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
GitCommit: fec3683
minikube version: v1.6.2
commit: 54f28ac5d3a815d1196cd5d57d707439ee4bb392
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.2", GitCommit:"59603c6e503c87169aea6106f57b9f242f64df89", GitTreeState:"clean", BuildDate:"2020-01-18T23:30:10Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2019-12-07T21:12:17Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}
Linux mewtwo-ubuntu 5.3.0-26-generic #28~18.04.1-Ubuntu SMP Wed Dec 18 16:40:14 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 10
- Comments: 24 (7 by maintainers)
solution worked for me change version of kaniko: gcr.io/kaniko-project/executor:debug-v0.15.0
Looks like a duplicate of #1001
Bug still present in debug-v0.17.0 - reverting to earlier version fix the build but not ideal.
‘’’ error building image: error building stage: failed to get filesystem from image: error removing var/run to make way for new symlink: unlinkat /var/run/secrets/kubernetes.io/serviceaccount/…2020_02_04_10_04_34.525459306: read-only file system ‘’’
Strong recommendation to everyone; use an explicitly tagged release of the kaniko image. Don’t use
latestin your critical systems. ~This change has not been released and is only affecting people usinglatest~ I was incorrect,latestis only built on release. This issue is only affecting people who are using thedebugtag which is built on every commit to master. But still, use an explicit version!@tejal29 I wanted to confirm that v0.17.1 doesn’t have the issue, and if i actively specify
--whitelist-var-run=falseI can get the same error again - confirming the flag’s actively returning to the other behavior.Thank you for your work on this! ❤️ 🎉!
I have the same issue (
0.17.0):with
--whitelist-var-run=false0.16.0works btw.@tejal29 What is the solution to this problem? #1001 seems to be related to CloudBuild (which I have no knowledge about). This issue here seems to be related to Kubernetes. AFAIK whitelisting
/var/runexposes the service account token in the image, which is a security thread. Not whitelisting breaks the build.closing this as dupe of #1001