kaniko: Error: failed to get filesystem from image: removing whiteout operation not permitted

Actual behavior

When using kaniko via Jenkins in a k8s cluster to build an image which contain whiteout file, got this error.

Resolved base name nginxinc/nginx-unprivileged:stable-alpine to nginxinc/nginx-unprivileged:stable-alpine 
Using dockerignore file: /home/jenkins/agent/workspace/mptabilite-front_feature_jenkins/.dockerignore 
Resolved base name nginxinc/nginx-unprivileged:stable-alpine to nginxinc/nginx-unprivileged:stable-alpine 
Retrieving image manifest nginxinc/nginx-unprivileged:stable-alpine 
Retrieving image manifest nginxinc/nginx-unprivileged:stable-alpine 
Built cross stage deps: map[]                
Retrieving image manifest nginxinc/nginx-unprivileged:stable-alpine 
Retrieving image manifest nginxinc/nginx-unprivileged:stable-alpine 
Unpacking rootfs as cmd COPY /dist /usr/share/nginx/html requires it. 
error building image: error building stage: failed to get filesystem from image: removing whiteout etc/nginx/.wh..wh..opq: fstatat /etc/nginx/.wh..opq: operation not permitted

Expected behavior Expect no error Note: Reproducing the bug in a local docker install give no error I have tried with version 0.9.0 which works and 1.16.0 which don’t work but no other version beetween.

To Reproduce Steps to reproduce the behavior:

  1. Launch a kaniko image on a k8s cluster using /busybox/cat as the entrypoint
  2. Get a shell into the image (/busybox/sh)
  3. Create a Dockerfile containing only one line FROM nginxinc/nginx-unprivileged:stable-alpine
  4. Launch the /kaniko/executor process

Additional Information

  • Dockerfile
FROM nginxinc/nginx-unprivileged:stable-alpine
  • Build Context None

  • Kaniko Image (fully qualified with digest) debug-1.17.1

Triage Notes for the Maintainers

Description Yes/No
Please check if this a new feature you are proposing
  • - [ ]
Please check if the build works in docker but not in kaniko
  • - [X]
Please check if this error is seen when you use --cache flag
  • - [ ]
Please check if your dockerfile is a multistage dockerfile
  • - [ ]

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 18 (6 by maintainers)

Most upvoted comments

For anyone having this issue (removing whiteout … operation not permitted ). It appears to be an issue with older version of docker running the aufs storage driver (default in older debian installs). Upgrading to docker 18 or higher and using the storage driver overlay2 fixed this issue for me. To check what storage driver you are using do a docker info.

We encounter the same problem with using php:7.4.4-fpm-buster as base image and doing anything that triggers unpacking rootfs. Here for example apt-get update

confirmed for php:7.4.6-fpm-buster too