kaniko: Failure under gVisor: failed to get filesystem from image: chtimes /bin/bash: invalid argument

Actual behavior

A build that succeeds under v0.16.0 fails under v0.17.0 with the error:

error building image: error building stage: failed to get filesystem from image: chtimes /bin/bash: invalid argument

Expected behavior

I expect the build to complete successfully.

To Reproduce Steps to reproduce the behavior:

  1. Create a package.json and package-lock.json using a command like

    npm install --save @atomist/automation-client
    
  2. Run the kaniko build with the following arguments

    --context=dir:///where/you/mount/the/dir --destination=where/you/can:push --dockerfile=Dockerfile --force
    

Additional Information

  • Dockerfile
FROM atomist/sdm-base:0.4.1

COPY package.json package-lock.json ./
RUN npm ci \
    && npm cache clean --force

COPY . ./

USER atomist:atomist
  • Build Context Please provide or clearly describe any files needed to build the Dockerfile (ADD/COPY commands)

Any package.json and package-lock.json should suffice.

  • Kaniko Image (fully qualified with digest) gcr.io/kaniko-project/executor:v0.17.0 sha256:c65c64d157bb6b1f15278e8ee28b02184e83e39340ddc25d346f18396c24da1d

Triage Notes for the Maintainers

I am seeing the failure when the build is being run in Kubernetes running on Container-optimized OS with the containerd runtime. The error occurs both when caching is being used and when it is not.

Here are the debug logs without caching:

DEBU[0000] Getting source context from dir:///atm/home
DEBU[0000] Build context located at /atm/home
DEBU[0000] Copying file /atm/home/Dockerfile to /kaniko/Dockerfile
WARN[0000] kaniko is being run outside of a container. This can have dangerous effects on your system
WARN[0000] Error detecting container runtime. Using default keychain: container runtime could not be found
DEBU[0000] Skip resolving path /kaniko/Dockerfile
DEBU[0000] Skip resolving path /atm/home
DEBU[0000] Skip resolving path /cache
DEBU[0000] Skip resolving path
DEBU[0000] Skip resolving path
DEBU[0000] Skip resolving path
INFO[0000] Resolved base name atomist/sdm-base:0.4.1 to atomist/sdm-base:0.4.1
INFO[0000] Using dockerignore file: /atm/home/.dockerignore
INFO[0000] Resolved base name atomist/sdm-base:0.4.1 to atomist/sdm-base:0.4.1
INFO[0000] Retrieving image manifest atomist/sdm-base:0.4.1
DEBU[0000] No file found for cache key sha256:555a4f6fb9b0fb9d180e2e9a0bfcd04c44128cb76eca26516ccb7ba7dd304b5c stat /cache/sha256:555a4f6fb9b0fb9d180e2e9a0bfcd04c44128cb76eca26516ccb7ba7dd304b5c: no such file or directory
DEBU[0000] Image atomist/sdm-base:0.4.1 not found in cache
INFO[0000] Retrieving image manifest atomist/sdm-base:0.4.1
INFO[0000] Built cross stage deps: map[]
INFO[0000] Retrieving image manifest atomist/sdm-base:0.4.1
DEBU[0001] No file found for cache key sha256:555a4f6fb9b0fb9d180e2e9a0bfcd04c44128cb76eca26516ccb7ba7dd304b5c stat /cache/sha256:555a4f6fb9b0fb9d180e2e9a0bfcd04c44128cb76eca26516ccb7ba7dd304b5c: no such file or directory
DEBU[0001] Image atomist/sdm-base:0.4.1 not found in cache
INFO[0001] Retrieving image manifest atomist/sdm-base:0.4.1
INFO[0001] Unpacking rootfs as cmd COPY package.json package-lock.json ./ requires it.
DEBU[0001] Mounted directories: [{/kaniko false} {/etc/mtab false} {/tmp/apt-key-gpghome true} {/dev false} {/dev/pts false} {/proc false} {/sys false} {/dev/termination-log false} {/atm/home false} {/atm/input false} {/atm/output false} {/etc/hosts false} {/etc/hostname false} {/etc/resolv.conf false} {/kaniko/.docker/config.json false} {/var/run/secrets/kubernetes.io/serviceaccount false} {/tmp false}]
error building image: error building stage: failed to get filesystem from image: chtimes /bin/bash: invalid argument
Description Yes/No
Please check if this a new feature you are proposing
  • - [ ]
Please check if the build works in docker but not in kaniko
  • - [x]
Please check if this error is seen when you use --cache flag
  • - [x]
Please check if your dockerfile is a multistage dockerfile
  • - [ ]

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 1
  • Comments: 27 (13 by maintainers)

Commits related to this issue

Most upvoted comments

@cvgw here are the results you requested:

runc runsc runsc-nightly runsc-head
kaniko 0.16.0 success success success success
kaniko 0.17.1 success fail fail fail
kaniko 16e60cd success success success success
kaniko debug-16e60cd success success success success

runsc is now release-20200219.0, runsc-nightly is release-20200211.0-39-g8dae8a10f01b, and runsc-head is release-20200219.0-46-ga92087f0f8fe.

The new version looks good, thanks again!

So it sounds like the difference is that the gVisor implementation of the system call underlying os.Chtimes does not handle negative values, while the same Linux system call must.