kaniko: COPY resolves symlinks it shouldn't try to resolve

I got this minimal Dockerfile to fail on v0.18.0 and v0.19.0 when building with target two using --target two (building target one works fine though, so it seems the other target is somehow interfering). v0.17.1 is working fine so I suspect it must be related to this issue. Hope it is of any help for testing. The Dockerfile:

FROM alpine:3.11 AS base
RUN mkdir -p /dir/source /dir/target
RUN echo test > /dir/target/file.txt
WORKDIR /dir/source
RUN ln -s ../target/file.txt file.txt

FROM scratch as one
COPY --from=base /dir/ /dir/

FROM scratch as two
COPY --from=base /dir/ /dir/

Resulting log from base stage:

INFO[0001] Retrieving image manifest alpine:3.11        
INFO[0002] Built cross stage deps: map[0:[/dir/ /dir/]] 
INFO[0002] Retrieving image manifest alpine:3.11        
INFO[0003] Retrieving image manifest alpine:3.11        
INFO[0004] Unpacking rootfs as cmd RUN mkdir -p /dir/source /dir/target requires it. 
INFO[0004] Taking snapshot of full filesystem...        
INFO[0004] Resolving paths                              
INFO[0004] RUN mkdir -p /dir/source /dir/target         
INFO[0004] cmd: /bin/sh                                 
INFO[0004] args: [-c mkdir -p /dir/source /dir/target]  
INFO[0004] Taking snapshot of full filesystem...        
INFO[0005] Resolving paths                              
INFO[0005] RUN echo test > /dir/target/file.txt         
INFO[0005] cmd: /bin/sh                                 
INFO[0005] args: [-c echo test > /dir/target/file.txt]  
INFO[0005] Taking snapshot of full filesystem...        
INFO[0005] Resolving paths                              
INFO[0005] WORKDIR /dir/source                          
INFO[0005] cmd: workdir                                 
INFO[0005] Changed working directory to /dir/source     
INFO[0005] RUN ln -s ../target/file.txt file.txt        
INFO[0005] cmd: /bin/sh                                 
INFO[0005] args: [-c ln -s ../target/file.txt file.txt] 
INFO[0005] Taking snapshot of full filesystem...        
INFO[0005] Resolving paths                              
INFO[0005] Saving file /dir/ for later use              
INFO[0005] Saving file /dir/ for later use              
error building image: symlink ../target/file.txt /kaniko/0/dir/source/file.txt: file exists

@bramus Tried your issue on latest v0.19.0 build and it does not work. i will add a fix in next release.

docker run -it --entrypoint /busybox/sh -v /usr/local/google/home/tejaldesai/.config/gcloud:/root/.config/gcloud -v /usr/local/google/home/tejaldesai/workspace/kaniko/integration:/workspace gcr.io/kaniko-project/executor:debug
/ # /kaniko/executor -f dockerfiles/Dockerfile1 --context=dir://workspace --destination=gcr.io/tejal-test/test-symlink
...

Also having symlink issues with an image containing a PHP app, where its dependency manager (Composer) will create symlinks to provided binaries.

When building using Docker itself, the files are – as expected – symlinked into vendor/bin/:

bramus in ~/repos/vbridge/pubsub-process-cctv on develop*
$ docker run -it --entrypoint /bin/bash "pubsub-process-cctv"
root@7e98449610d1:/# ls -Al /app/vendor/bin/
total 0
lrwxrwxrwx 1 www-data www-data 43 Mar 16 12:48 google-cloud-batch -> ../google/cloud/Core/bin/google-cloud-batch
lrwxrwxrwx 1 www-data www-data 50 Mar 16 12:48 google-cloud-debugger -> ../google/cloud/Debugger/bin/google-cloud-debugger
lrwxrwxrwx 1 www-data www-data 26 Mar 16 12:48 phpunit -> ../phpunit/phpunit/phpunit
lrwxrwxrwx 1 www-data www-data 30 Mar 16 12:57 pubsub-pull -> ../plonk/plonk/bin/pubsub-pull
root@7e98449610d1:/# 

When building using kaniko, the files in the vendor/bin/ are no symlinks but actual copies:

bramus in ~/repos/vbridge/pubsub-process-cctv on develop*
$ docker run -it --entrypoint /bin/bash "eu.gcr.io/REDACTED/pubsub-process-cctv:develop"
root@afbe6246ea6d:/# ls -Al /app/vendor/bin/
total 16
-rwxr-xr-x 1 www-data www-data 1729 Mar 16 13:59 google-cloud-batch
-rwxr-xr-x 1 www-data www-data 2125 Mar 16 13:59 google-cloud-debugger
-rwxr-xr-x 1 www-data www-data 1443 Mar 16 13:59 phpunit
-rwxr-xr-x 1 www-data www-data 2030 Mar 16 14:51 pubsub-pull
root@afbe6246ea6d:/#

For reference, the Composer dependencies were built in a separate image, and then COPY’d into the main image (e.g. multi-stage build):

FROM composer:1.9.1 as composer
COPY composer.json composer.lock /app/
RUN composer install \
    --ignore-platform-reqs \
    --no-interaction \
    --no-plugins \
    --no-scripts \
    --prefer-dist \
    --optimize-autoloader \
    -vvv

…

FROM php:7.3-cli
COPY --chown=www-data:www-data --from=composer /app/vendor/ /app/vendor/
…

EDIT: Is this this issue, or is it #437 ?

INFO[0024] Taking snapshot of files…
error building image: error building stage: failed to take snapshot: unable to add file /usr/src/services/backend/backend to layered map: error creating hash for /usr/src/services/backend/backend: lstat /usr/src/services/backend/backend: no such file or directory ERROR

Thanks @WestonThayer The error you are seeing is due to #1112 and not due to symlinks.

Thank you. I can confirm this issue is fixed on debug-edge.

Are you unable to reproduce the two repro I gave in the original post?

yes i am. The reason why we made that change was to mimic what docker does and picked up the regression. I will work on fixing it and fixing the tests.