buildx: Error building python 3.6 slim
I have this Dockerfile:
FROM python:3.6-slim
RUN apt-get update && apt-get install -y --no-install-recommends make
When I run this command:
docker buildx build . --pull --platform linux/arm/v5
I get this output:
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 27.3s (6/6) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 129B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/python:3.6-slim 0.6s
=> [auth] library/python:pull token for registry-1.docker.io 0.0s
=> CACHED [1/2] FROM docker.io/library/python:3.6-slim@sha256:200bc4213e83751cc4a5496e72573ce8173713fc73266a1fd3ec62bd378a5890 0.0s
=> => resolve docker.io/library/python:3.6-slim@sha256:200bc4213e83751cc4a5496e72573ce8173713fc73266a1fd3ec62bd378a5890 0.0s
=> ERROR [2/2] RUN apt-get update && apt-get install -y --no-install-recommends make 26.6s
------
> [2/2] RUN apt-get update && apt-get install -y --no-install-recommends make:
#6 0.582 Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
#6 0.583 Get:2 http://deb.debian.org/debian buster InRelease [121 kB]
#6 0.679 Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
#6 2.670 Get:4 http://security.debian.org/debian-security buster/updates/main armel Packages [232 kB]
#6 4.570 Get:5 http://deb.debian.org/debian buster/main armel Packages [7627 kB]
#6 6.556 Get:6 http://deb.debian.org/debian buster-updates/main armel Packages [7860 B]
#6 8.912 Fetched 8106 kB in 8s (962 kB/s)
#6 8.912 Reading package lists...
#6 16.39 Reading package lists...
#6 23.71 Building dependency tree...
#6 24.28 Reading state information...
#6 24.65 Suggested packages:
#6 24.65 make-doc
#6 25.19 The following NEW packages will be installed:
#6 25.19 make
#6 25.39 0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
#6 25.39 Need to get 327 kB of archives.
#6 25.39 After this operation, 1300 kB of additional disk space will be used.
#6 25.39 Get:1 http://deb.debian.org/debian buster/main armel make armel 4.2.1-1.2 [327 kB]
#6 26.17 debconf: delaying package configuration, since apt-utils is not installed
#6 26.29 Fetched 327 kB in 0s (2000 kB/s)
#6 26.37 Error while loading /usr/sbin/dpkg-split: No such file or directory
#6 26.38 Error while loading /usr/sbin/dpkg-deb: No such file or directory
#6 26.38 dpkg: error processing archive /var/cache/apt/archives/make_4.2.1-1.2_armel.deb (--unpack):
#6 26.38 dpkg-deb --control subprocess returned error exit status 1
#6 26.40 Errors were encountered while processing:
#6 26.40 /var/cache/apt/archives/make_4.2.1-1.2_armel.deb
#6 26.50 E: Sub-process /usr/bin/dpkg returned an error code (1)
------
Dockerfile:3
--------------------
1 | FROM python:3.6-slim
2 |
3 | >>> RUN apt-get update && apt-get install -y --no-install-recommends make
4 |
--------------------
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c apt-get update && apt-get install -y --no-install-recommends make]: exit code: 100
Some observations:
It seems like it’s looking for /usr/sbin/dpkg-split and /usr/sbin/dpkg-deb, even though those are located in the /usr/bin folder.
Platforms linux/arm/v7 and linux/arm/v8 both fail, however linux/amd64 does not fail.
Changing my Dockerfile to use FROM python:3.6 (instead of slim) builds correctly for all cases (armv5, armv7, arm64v8, amd64), and manually starting a container (after doing the multiarch qemu setup) with docker run -it --rm arm32v5/python:3.6-slim bash allows me to install make just fine.
I wasn’t sure if this was a buildx issue or just a python-slim issue, but I was also able to do this as my Dockerfile:
FROM arm32v5/python:3.6-slim
RUN apt-get update && apt-get install -y --no-install-recommends make
and do a regular docker build:
docker build . --pull
and it builds correctly. So my guess is it has something to do with buildx? Unless arm32v5/python:3.6-slim is different than python:3.6-slim that buildx would be pulling.
This might be related to #493, not entirely sure.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 16
- Comments: 35
Commits related to this issue
- prometheus_speedtest: Add workaround for "lsb_release -a" missing Docker error. Found this workaround on https://github.com/docker/buildx/issues/495#issuecomment-754688157. — committed to jeanralphaviles/prometheus_speedtest by jeanralphaviles 3 years ago
- trying solution in https://github.com/docker/buildx/issues/495 — committed to averissimo/unbound-docker by averissimo 3 years ago
- Try fixes from https://github.com/docker/buildx/issues/495#issuecomment-778896842 — committed to rfay/ddev-images by rfay 3 years ago
- Change base image to debian:bullseye-slim, add tree, add php8.0-uploadprogress (#57) * Change base image to debian:bullseye-slim * Experiment with removing the old gotta-have-it line * Use a re... — committed to drud/ddev-images by rfay 3 years ago
- https://github.com/docker/buildx/issues/495#issuecomment-918925854 — committed to dgtlmoon/changedetection.io by dgtlmoon 3 years ago
- GH actions arm issue https://github.com/docker/buildx/issues/495 — committed to tgoelles/Python_docker by tgoelles 3 years ago
- Drop arm/v5 support as it cannot handle by buildx. See: https://github.com/docker/buildx/issues/495 . Signed-off-by: Masaki Muranaka <monaka@monami-ya.com> — committed to pizzafactory-contorno/piatto by monaka 3 years ago
- Test met extra symlinks https://github.com/docker/buildx/issues/495#issuecomment-772267281 — committed to wschoot/github-pages by wschoot 2 years ago
- Work around ARM buildx of debian-slim issues Got the idea for the symlinks from https://github.com/docker/buildx/issues/495 — committed to QuinnJensen/nlan-docker by deleted user 2 years ago
- try again — committed to mynaparrot/plugNmeet-server by jibon57 2 years ago
- Update deploy-docker.yml Try to fix docker buildx by using suggestion from https://github.com/docker/buildx/issues/495#issuecomment-761562905 — committed to dfuchss/synapse-arm by dfuchss 2 years ago
- fix QEMU issue https://github.com/docker/buildx/issues/495#issuecomment-918925854 — committed to 417-72KI/Docker-Swift-Mint by 417-72KI 2 years ago
- Add soft links for some reason... https://github.com/docker/buildx/issues/495#issuecomment-772267281 — committed to willat8/docker-shepherd by willat8 3 years ago
- run binfmt first https://github.com/docker/buildx/issues/495#issuecomment-918925854 — committed to dictcp/fexemu-on-docker by dictcp 2 years ago
What actually solved my problem after two painful days was creating the following custom builder:
Now I can successfully run:
on my
linux/amd64machine.While trying I saw that there is an update to docker 20.10.2. Don’t know if this did the trick, but now it works.
However in the process I also did a
docker run --rm --privileged multiarch/qemu-user-static --reset -p yesFound the same problem when I tried to build arm64 image and included
apt-get install iputils-pingin my Dockerfile (my base image is openjdk:11-jre-slim, however).The solution proposed by pcko1 didn’t work for me but the solution proposed by junka worked. Just adding several soft links before
apt-get:It’s even more confusing when I found I can
apt-getdirectly inside container running on raspberrypi.Just in case you are using Github Actions and the were trying the workarounds mentioned earlier, there may be a much more obvious solution for you: use the docker/setup-qemu-action in a step before setting up Docker Buildx:
This at least solved it for me – without any further workarounds like symlinking from /usr/bin to /usr/sbin.
Bump on this! Worked for me. I would just recommend not using
latest.@pcko1 your suggestion resolved my issue also, cheers mate! 😃
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes docker buildx create --name multiarch --driver docker-container --use docker buildx inspect --bootstrap
@rxue92 I just had the same problem. You should have a look at
/proc/sys/fs/binfmt_misc/and see if there are conflicting entries for the same architecture (arm64 in your case). I found a duplicate and remove the non-qemu one like this:In my case, it was an entry created by the OS for cross-compiling, so I had to remove it in order to get the docker builds working.
I am getting the exact same problem when I
buildwith thisdockerfile:which results in:
For the record, I am cross-building on
ubuntu 20.04 / amd64and for a raspberry pi 4 target machine (ubuntu server 20.10 / arm64) usingbuildxand the following command:My custom builder looks like this:
Any ideas?
I investigated a similar issue (same error message) and came to the following conclusion:
2.11.1I get theseapt-getcrashes. When compiling the same qemu version from the upstream sources (without Debian/Ubuntu patches), it works fine.docker run --rm --privileged multiarch/qemu-user-static --reset -p yeshelp here, is because they force the kernel to load working qemu binaries contained within the Docker image instead of taking the system’s broken qemu. Beware that you are effectively loading third-party binaries as root, which are nowhere to be found on the host’s disk after the container stops (they live in-memory).Also experiencing this on a daemonless build. What can I do in my case?
In my case running:
docker run --privileged --rm tonistiigi/binfmt --install allonce again to update the previous installation fixed the issues.
don’t know the root cause, add these in dockerfile would help
soft link path could be different
@JesseBowling
In my experimentations, you run into issues if you create a builder. Things work fine if you use the default builder and don’t multi platform build (not that you can with the default builder anyway). Is this your experience as well?
But why was is it working before the system update?
I wouldn’t see this Issue as closed or fixed.
Just an update after reading though all the comments here and trying out some solutions:
I am primarily using GitHub Actions to do my building with Buildx, hopefully what I was able to reproduce on my own machine closely matches what GitHub Actions was doing. I realized that the list of Buildx platforms supported in GitHub Actions did not include
linux/arm/v5. Once I removed that from my build workflow, the build was successful!It’s weird, because having
linux/arm/v5didn’t break things before, but perhaps some change happened 2 months ago and now it does break. Probably for the best though, as I’m going to assume whatever was being built might not have been guaranteed to be arm/v5 compatible? Also interestingly,linux/arm/v5works just fine forpython:3.6(not slim) with the example Dockerfile I provided (I tested it today). So I’m not sure what’s going on there.At any rate, it seems that if I use the platforms that Buildx reports are supported, all should be good (assuming the base image also was built with it). At the time of writing, GitHub Actions
docker/setup-buildx-action@v1supports linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, and linux/arm/v6.@JesseBowling I concur arm64 builds work for me as well.
I’m running builds on a ubuntu server. On that server, your examples have no problems for me with or without the PATH edit. Only when I create a docker builder in order to build for more than one platform do I run into these issues (even if I use the created builder to only build for one platform).
I tried pinning qemu docker image to 5.0.0-5 which was released 4 months ago, but that made no difference.
@fidesachates It seems to be specific to armhf (arm/v7 and arm/v8) builds. ARM64 and AMD64 build fine.
I’m not sure if I understand your question enough to give you a more solid answer than that. I can say that I’ve been primarily using this via Github Actions (i.e., https://github.com/CommunityHoneyNetwork/rdphoney/actions/runs/418797571) and it first broke 22 days ago.
The snippets I provided in the original issue came from trying to recreate the issue as simply as possible on my Macbook, which may well have had a non-default builder.