buildx: Value too large for defined data type

if you don’t use --platform=$BUILDPLATFORM it will use the target platform. By specifying --platform=$BUILDPLATFORM you force it to build with your computer’s platform and you won’t have the bug from qemu.

@MarcelCoding, you are right. If you follow ehuss’ comment you end up at the QEMU bugtracker which goes into more detail for those interested.

Unfortunately the fix seems very far away, if there even will be a fix. Over in the cargo repo we got around the issue by mounting a tmpfs on the ~/.cargo folder, and this is appears to not trigger the bug while compiling cryptography anymore. Perhaps this is something that would work here as well.

@jdrouet Thanks for the example Dockerfile - worked like a charm!

Not really related to buildx, but does anyone know how to pass the BUILDPLATFORM to the Dockerfile when building the Dockerfile with docker-compose? Up to now, I’ve always used docker-compose build when I wanted to quickly build a docker image on my machine and only used buildx for building images for multiple architectures.

But with the --platform=$BUILDPLATFORM workaround from above, docker-compose build always fails with: failed to parse platform : "" is an invalid component of "": platform specifier component must match "^[A-Za-z0-9_-]+$": invalid argument.

I already tried: docker-compose build --build-arg "BUILDPLATFORM=linux/amd64" but unfortunately that doesn’t work either - same error.

It’s not a big deal for me, as I can use buildx also for my local builds, but I was just wondering if anyone has an idea?

edit: I figured it out. COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose build works for me (just make sure that your docker-compose version is not too old)

You can take a look here. I do a fetch using the build platform arch and then copy it in the target platform.

Hello.

I am now reaching another level since this error arises when building a Python program, which depends on cryptography, a library which in turn builds with Rust… See : pyca/cryptography#5771

Let me explain the Python way : on some architectures (here linux/arm/v7) there is no “Python Wheel” package for this dependency so the pip install command builds it from source… using Rust… triggering again a Value too large… error !

The problem is that I don’t think I can alter the way this dependency builds when triggered by a pip install (the cargo build command is out of hand), and anyway I’m getting tired to implement patches making my Dockerfiles hard to maintain…

Any news ? Is there any link to an original issue on qemu or anything ?

I’ve disabled rust in the cryptography build for now but this trick will be removed from their next release…

I was able to use armv6 and aarch64 just fine, its just arm7 causing this weird issue. Looks like manually building is the way to go for now

I’ve tried using Java and JLink and it occurred the same error: log.txt

Caused by: java.nio.file.FileSystemException: /opt/java/openjdk/jmods: Value too large for defined data type

[linux/arm/v7] qemu-v6.1.0-20

https://github.com/MarcelCoding/luna/blob/0b567aa6e1e7889bfe8ad3295dbf9dac1a855cf3/docker/Dockerfile.github-actions#L17-L23 https://github.com/MarcelCoding/luna/blob/0b567aa6e1e7889bfe8ad3295dbf9dac1a855cf3/.github/workflows/ci.yaml#L214-L227

@crazy-max it looks like the issue is still present, I’m building a docker image for multiple platforms, and it fails when I include linux/arm/v7 as can be seen here: https://github.com/texthtml/midi-synthetizer-autoconnect/runs/3399891145?check_suite_focus=true

I’m using the docker/setup-qemu-action@v1 which is using the Qemu binaries from https://github.com/tonistiigi/binfmt

if you don’t use --platform=$BUILDPLATFORM it will use the target platform. By specifying --platform=$BUILDPLATFORM you force it to build with your computer’s platform and you won’t have the bug from qemu.

Right, I managed to build it thanks to your example ! I’m not a Rust developer : I’m trying to compile someone else’s Rust project ; so I didn’t figured out that it was a Rust-specific solution.

As I understand it now (if useful to anyone) :

1. since the error happens when running cargo fetch, this step needs to be executed inside the working container (cargo vendor will do it)
2. then copy the generated/downloaded files (vendor, .cargo) into the target image
3. finally finish the build inside the target image in offline mode (cargo build ... --offline), using the previously cached files

I have “vendor” files that look like architecture-specific (e.g. winapi-i686-pc-windows-gnu) so I will see if it works at runtime when I can test a container. I wonder then if Rust would simply allow us to fully cross-compile for all targets, then just copy the generated files into the final image?

My process is a little bit different since I first clone the code of an existing project from github, so I don’t need to do cargo init. The final Dockerfile looks like this and it builds fine : https://gist.github.com/nicobo/06530acc4fd82497878d45a93348ae3c

Thanks @jdrouet !