kaniko: Kaniko build fails on apt-get reporting "unable to fetch libraries"
Actual behavior Kaniko build fails on apt-get reporting “unable to fetch libraries” while the same build succeed under ‘docker build’.
A changing the docker file to rebuild the package cache per this documentation (https://sites.google.com/site/longphanwiki/basic-skills/ubuntu-linux) will solve the problem, however, these directives are not needed when running the build via docker build.
Expected behavior The container is built successfully.
To Reproduce Steps to reproduce the behavior:
- Perform a kaniko build of the docker file listed below
- Perform a docker build of this same file
- Compare output
Additional Information
- Dockerfile
The dockerfile that’s failing under kaniko is defined below:
FROM botpress/server:v12_0_0
RUN chmod -R gou+rw /botpress
WORKDIR /botpress
RUN apt-get update \
&& apt-get -y install curl gnupg \
&& curl -sL https://deb.nodesource.com/setup_11.x | bash \
&& apt-get -y install nodejs \
&& apt-get -y install rsync \
&& apt-get upgrade -y
RUN cd /botpress && npm install && npm install node-sp-auth --save-dev && npm install sp-request --save-dev && npm install activedirectory --save-dev
EXPOSE 3000-55000
CMD ["./bp"]
- Build Context N/A
- Kaniko Image (fully qualified with digest) gcr.io/kaniko-project/executor:debug
Notes: If you add the following directives before the apt-get install, kaniko will build the container:
RUN apt-get clean \
&& cd /var/lib/apt \
&& mv lists lists.old \
&& mkdir -p lists/partial \
&& apt-get update \
&& apt-get upgrade -y
Question:
Is this behavior anticipated? If not, is this a bug in kaniko?
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 4
- Comments: 35 (15 by maintainers)
@andraxin that kind of comment is uncalled for. Refer to the code of conduct for more info
https://github.com/GoogleContainerTools/kaniko/blob/master/code-of-conduct.md
got the same issue
Hi @tejal29
I’m not sure in what way one isolated example that happens to work is supposed to help in solving the original problem, but since the issue is tagged as
awaiting evidence, I thought I’d provide more evidence and slightly more insight into what’s actually broken and why the mostly irrelevant example provided happens to still work.The evidence shows that
kanikoresets all timestamps in the original image to the current date and time, resulting inapt updatenot fetching any updates, which has a reasonable chance of producing the kind of errors reported above (and seen below). Recent image builds that happen to have those exact sameInReleasefiles in the image that are hosted on the servers are unaffected; and so are install commands that do not reference packages whose version has not changed between theInReleasefiles embedded in the image and those on the server…Here’s a simple
Dockerfilethat should allow anyone to reproduce the issue:Passing this to
kanikoresults in the following output culminating in a version mismatch error:For comparison, Docker preserves the original timestamps, which is obviously the sane thing to do and allows
apt-get updateand the entire build to succeed as expected. For brevity, I post only the relevant directory listing here:@cvgw Ok, I got it. I had to nuke the executor:debug version on my machine and pull a new one and that solved the issue. I think we’re good and we can put this issue to rest.
I think this has something to do with mounting the context at
/tmp(I have no idea why)IMAGE=gcr.io/kaniko-project/executor:793This fails
This succeeds
@vguaglione can you try mounting your context at
/workspaceinstead of/tmp?Dockerfile used
Thanks @vguaglione I was able to repro with that example.
I also tested it with Pr #981 and that seems to fix it
I can repo it using v0.15.0. Same behavior observed using v0.16.0 also:
Dockerfile:
FROM botpress/server:v12_0_0 RUN chmod -R gou+rw /botpress WORKDIR /botpress
RUN apt-get update
&& apt-get -y install curl gnupg
&& curl -sL https://deb.nodesource.com/setup_11.x | bash
&& apt-get -y install nodejs
&& apt-get -y install rsync
&& apt-get upgrade -y
RUN cd /botpress && npm install && npm install node-sp-auth --save-dev && npm install sp-request --save-dev && npm install activedirectory --save-dev
EXPOSE 3000-55000 CMD [“./bp”]
Kaniko build command:
docker run -v $(pwd):/tmp gcr.io/kaniko-project/executor:v0.15.0 --dockerfile=/tmp/Dockerfile --context=/tmp --no-push
Kaniko Output:
INFO[0000] Resolved base name botpress/server:v12_0_0 to botpress/server:v12_0_0 INFO[0000] Resolved base name botpress/server:v12_0_0 to botpress/server:v12_0_0 INFO[0000] Retrieving image manifest botpress/server:v12_0_0 INFO[0000] Image botpress/server:v12_0_0 not found in cache INFO[0000] Retrieving image manifest botpress/server:v12_0_0 INFO[0000] Built cross stage deps: map[]
INFO[0000] Retrieving image manifest botpress/server:v12_0_0 INFO[0000] Image botpress/server:v12_0_0 not found in cache INFO[0000] Retrieving image manifest botpress/server:v12_0_0 INFO[0001] Unpacking rootfs as cmd RUN chmod -R gou+rw /botpress requires it. INFO[0048] Taking snapshot of full filesystem…
INFO[0054] RUN chmod -R gou+rw /botpress
INFO[0054] cmd: /bin/sh
INFO[0054] args: [-c chmod -R gou+rw /botpress]
INFO[0055] Taking snapshot of full filesystem…
INFO[0076] WORKDIR /botpress
INFO[0076] cmd: workdir
INFO[0076] Changed working directory to /botpress
INFO[0076] RUN apt-get update && apt-get -y install curl gnupg && curl -sL https://deb.nodesource.com/setup_11.x | bash && apt-get -y install nodejs && apt-get -y install rsync && apt-get upgrade -y INFO[0076] cmd: /bin/sh
INFO[0076] args: [-c apt-get update && apt-get -y install curl gnupg && curl -sL https://deb.nodesource.com/setup_11.x | bash && apt-get -y install nodejs && apt-get -y install rsync && apt-get upgrade -y] Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease Reading package lists… Reading package lists… Building dependency tree… Reading state information… The following additional packages will be installed: dirmngr gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm gpgv krb5-locales libasn1-8-heimdal libassuan0 libcurl4 libgssapi-krb5-2 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libk5crypto3 libkeyutils1 libkrb5-26-heimdal libkrb5-3 libkrb5support0 libksba8 libldap-2.4-2 libldap-common libnghttp2-14 libnpth0 libreadline7 libroken18-heimdal librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libsqlite3-0 libwind0-heimdal pinentry-curses readline-common Suggested packages: dbus-user-session libpam-systemd pinentry-gnome3 tor parcimonie xloadimage scdaemon krb5-doc krb5-user libsasl2-modules-gssapi-mit | libsasl2-modules-gssapi-heimdal libsasl2-modules-ldap libsasl2-modules-otp libsasl2-modules-sql pinentry-doc readline-doc The following NEW packages will be installed: curl dirmngr gnupg gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm krb5-locales libasn1-8-heimdal libassuan0 libcurl4 libgssapi-krb5-2 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libk5crypto3 libkeyutils1 libkrb5-26-heimdal libkrb5-3 libkrb5support0 libksba8 libldap-2.4-2 libldap-common libnghttp2-14 libnpth0 libreadline7 libroken18-heimdal librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libsqlite3-0 libwind0-heimdal pinentry-curses readline-common The following packages will be upgraded: gpgv 1 upgraded, 41 newly installed, 0 to remove and 38 not upgraded. Need to get 5124 kB of archives. After this operation, 15.2 MB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gpgv amd64 2.2.4-1ubuntu1.2 [198 kB] Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 readline-common all 7.0-3 [52.9 kB] Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 libreadline7 amd64 7.0-3 [124 kB] Ign:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsqlite3-0 amd64 3.22.0-1ubuntu0.1 Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 krb5-locales all 1.16-2ubuntu0.1 [13.5 kB] Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libkrb5support0 amd64 1.16-2ubuntu0.1 [30.9 kB] Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libk5crypto3 amd64 1.16-2ubuntu0.1 [85.6 kB] Err:4 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libsqlite3-0 amd64 3.22.0-1ubuntu0.1 404 Not Found [IP: 91.189.88.174 80] Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 libkeyutils1 amd64 1.5.9-9.2ubuntu2 [8720 B] Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libkrb5-3 amd64 1.16-2ubuntu0.1 [279 kB] Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libgssapi-krb5-2 amd64 1.16-2ubuntu0.1 [122 kB] Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libroken18-heimdal amd64 7.5.0+dfsg-1 [41.3 kB] Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 libasn1-8-heimdal amd64 7.5.0+dfsg-1 [175 kB] Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 libheimbase1-heimdal amd64 7.5.0+dfsg-1 [29.3 kB] Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 libhcrypto4-heimdal amd64 7.5.0+dfsg-1 [85.9 kB] Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libwind0-heimdal amd64 7.5.0+dfsg-1 [47.8 kB] Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 libhx509-5-heimdal amd64 7.5.0+dfsg-1 [107 kB] Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 libkrb5-26-heimdal amd64 7.5.0+dfsg-1 [206 kB] Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 libheimntlm0-heimdal amd64 7.5.0+dfsg-1 [14.8 kB] Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 libgssapi3-heimdal amd64 7.5.0+dfsg-1 [96.5 kB] Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsasl2-modules-db amd64 2.1.27~101-g0780600+dfsg-3ubuntu2 [14.8 kB] Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsasl2-2 amd64 2.1.27~101-g0780600+dfsg-3ubuntu2 [49.2 kB] Err:22 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libldap-common all 2.4.45+dfsg-1ubuntu1.2 404 Not Found [IP: 91.189.88.174 80] Err:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libldap-2.4-2 amd64 2.4.45+dfsg-1ubuntu1.2 404 Not Found [IP: 91.189.88.174 80] Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 libnghttp2-14 amd64 1.30.0-1ubuntu1 [77.8 kB] Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 librtmp1 amd64 2.4+20151223.gitfa8646d.1-1 [54.2 kB] Ign:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libcurl4 amd64 7.58.0-2ubuntu3.7 Ign:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 curl amd64 7.58.0-2ubuntu3.7 Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 libassuan0 amd64 2.5.1-2 [35.0 kB] Get:29 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gpgconf amd64 2.2.4-1ubuntu1.2 [123 kB] Err:26 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 libcurl4 amd64 7.58.0-2ubuntu3.7 404 Not Found [IP: 91.189.88.174 80] Err:27 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 curl amd64 7.58.0-2ubuntu3.7 404 Not Found [IP: 91.189.88.174 80] Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 libksba8 amd64 1.3.5-2 [92.6 kB] Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 libnpth0 amd64 1.5-3 [7668 B] Get:32 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 dirmngr amd64 2.2.4-1ubuntu1.2 [316 kB] Get:33 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gnupg-l10n all 2.2.4-1ubuntu1.2 [49.6 kB] Get:34 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gnupg-utils amd64 2.2.4-1ubuntu1.2 [127 kB] Get:35 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gpg amd64 2.2.4-1ubuntu1.2 [467 kB] Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 pinentry-curses amd64 1.1.0-1 [35.8 kB] Get:37 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gpg-agent amd64 2.2.4-1ubuntu1.2 [227 kB] Get:38 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gpg-wks-client amd64 2.2.4-1ubuntu1.2 [91.9 kB] Get:39 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gpg-wks-server amd64 2.2.4-1ubuntu1.2 [84.9 kB] Get:40 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gpgsm amd64 2.2.4-1ubuntu1.2 [215 kB] Get:41 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gnupg amd64 2.2.4-1ubuntu1.2 [249 kB] Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsasl2-modules amd64 2.1.27~101-g0780600+dfsg-3ubuntu2 [48.7 kB] Fetched 4083 kB in 2s (1784 kB/s) E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/s/sqlite3/libsqlite3-0_3.22.0-1ubuntu0.1_amd64.deb 404 Not Found [IP: 91.189.88.174 80] E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/o/openldap/libldap-common_2.4.45+dfsg-1ubuntu1.2_all.deb 404 Not Found [IP: 91.189.88.174 80] E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/o/openldap/libldap-2.4-2_2.4.45+dfsg-1ubuntu1.2_amd64.deb 404 Not Found [IP: 91.189.88.174 80] E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/c/curl/libcurl4_7.58.0-2ubuntu3.7_amd64.deb 404 Not Found [IP: 91.189.88.174 80] E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/c/curl/curl_7.58.0-2ubuntu3.7_amd64.deb 404 Not Found [IP: 91.189.88.174 80] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? error building image: error building stage: failed to execute command: waiting for process to exit: exit status 100
@hugokerstens Sorry we missed this in triage.