rustup: rustup-init hangs in armv7 docker container running on an arm64 Linux with `reqwest` backend
Problem
Running curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh in armv7 docker container running on an arm64 Linux hangs.
Steps
Run a docker container with --platform linux/arm/v7 on an arm64 Linux, install curl and run the sh.rustup.sh script
$ docker run --rm -it --platform linux/arm/v7 ubuntu:22.04 bash
root@668fa2822782:/# apt update && apt install curl -y
root@668fa2822782:/# curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --verbose
info: downloading installer
info: profile set to 'default'
info: default host triple is aarch64-unknown-linux-gnu
verbose: installing toolchain 'stable-aarch64-unknown-linux-gnu'
verbose: toolchain directory: '/root/.rustup/toolchains/stable-aarch64-unknown-linux-gnu'
info: syncing channel updates for 'stable-aarch64-unknown-linux-gnu'
verbose: creating temp file: /root/.rustup/tmp/dqlfqns1_reba5ms_file
verbose: downloading file from: 'https://static.rust-lang.org/dist/channel-rust-stable.toml.sha256'
verbose: downloading with reqwest
hangs forever at the verbose: downloading with reqwest step, use RUSTUP_USE_CURL=1 works fine
root@668fa2822782:/# RUSTUP_USE_CURL=1 curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | RUSTUP_USE_CURL=1 sh -s -- -y --verbose
info: downloading installer
info: profile set to 'default'
info: default host triple is aarch64-unknown-linux-gnu
verbose: installing toolchain 'stable-aarch64-unknown-linux-gnu'
verbose: toolchain directory: '/root/.rustup/toolchains/stable-aarch64-unknown-linux-gnu'
info: syncing channel updates for 'stable-aarch64-unknown-linux-gnu'
verbose: creating temp file: /root/.rustup/tmp/0lfud7winasgeiob_file
verbose: downloading file from: 'https://static.rust-lang.org/dist/channel-rust-stable.toml.sha256'
verbose: downloading with curl
verbose: deleted temp file: /root/.rustup/tmp/0lfud7winasgeiob_file
verbose: no update hash at: '/root/.rustup/update-hashes/stable-aarch64-unknown-linux-gnu'
verbose: creating temp file: /root/.rustup/tmp/zs3_t8gv_pyprmk6_file.toml
verbose: downloading file from: 'https://static.rust-lang.org/dist/channel-rust-stable.toml'
verbose: downloading with curl
verbose: checksum passed
verbose: creating temp file: /root/.rustup/tmp/zwsok2uoflekbroj_file
verbose: downloading file from: 'https://static.rust-lang.org/dist/channel-rust-stable.toml.asc'
verbose: downloading with curl
verbose: deleted temp file: /root/.rustup/tmp/zwsok2uoflekbroj_file
verbose: Good signature from on https://static.rust-lang.org/dist/channel-rust-stable.toml from:
verbose: from builtin Rust release key
verbose: RSAEncryptSign/85AB96E6-FA1BE5FE - Rust Language (Tag and Release Signing Key) <rust-key@rust-lang.org>
verbose: Fingerprint: 108F 6620 5EAE B0AA A8DD 5E1C 85AB 96E6 FA1B E5FE
verbose: deleted temp file: /root/.rustup/tmp/zs3_t8gv_pyprmk6_file.toml
info: latest update on 2022-11-03, rust version 1.65.0 (897e37553 2022-11-02)
info: downloading component 'cargo'
verbose: creating Download Directory directory: '/root/.rustup/downloads'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2022-11-03/cargo-1.65.0-aarch64-unknown-linux-gnu.tar.xz'
verbose: downloading with curl
verbose: checksum passed
info: downloading component 'clippy'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2022-11-03/clippy-1.65.0-aarch64-unknown-linux-gnu.tar.xz'
verbose: downloading with curl
verbose: checksum passed
info: downloading component 'rust-docs'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2022-11-03/rust-docs-1.65.0-aarch64-unknown-linux-gnu.tar.xz'
verbose: downloading with curl
verbose: checksum passed
info: downloading component 'rust-std'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2022-11-03/rust-std-1.65.0-aarch64-unknown-linux-gnu.tar.xz'
verbose: downloading with curl
verbose: checksum passed
info: downloading component 'rustc'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2022-11-03/rustc-1.65.0-aarch64-unknown-linux-gnu.tar.xz'
verbose: downloading with curl
55.9 MiB / 79.4 MiB ( 70 %) 0 B/s in 1s ETA: Unknown^C
Possible Solution(s)
No response
Notes
No response
Rustup version
rustup-init 1.25.1 (bb60b1e89 2022-07-12)
Installed toolchains
N/A
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 40 (23 by maintainers)
I found the underlying issue: https://github.com/rust-lang/rust/issues/60605 armv6l LLVM uses the CP15 barrier instruction, which are deprecated since ARMv7. Linux by default is compiled to emulate and trace these calls, to emit a warning:
Above shows that reqwest is not the only thing which emits this call, but rustup-init does as well, later on when installing the downloaded component, leading to the same hang. So
RUSTUP_USE_CURL=1and similar help only with one step, to hang with the next, as also reflected by OP logs.We even faced this a year earlier in a different context: doing Rust builds on Raspberry Pi 4 with 64-bit kernel on a 32-bit Raspberry Pi OS. It did not appear anymore/was not needed as we do not ship/promote the Raspbian/armv6l userland images for 64-bit capable Raspberry Pi models anymore, but have Debian/armv7l userland images, if really 32-bit is required for some reason. So I forgot about it.
It also explains why I face the issue on armv6l userland only, as the armv7l LLVM seems to not make use of this instruction anymore, as it is deprecated for this architecture already. As this was different for OP, it might have changed within the last years.
Since aarch64 CPUs usually still support the deprecated CP15 barrier instruction, Linux can be configured to make use of the hardware instruction instead of emulating it, which bypasses all the tracing and warning and solves the issue:
Or permanently:
The default value is
1, which means to emulate the instruction, causing the issue.Alternatively the kernel can be compiled without the support to emulate this instruction:
Let’s hope this gets some more attention at LLVM, since future ARM versions won’t support
abi.cp15_barrier=2(hardware instruction) anymore.I tried to disable cURL and use the default reqwest backend instead with the 1.27 beta, and it also didn’t change anything compared to before.
Here is the output:
Here a script to replicate via Docker Raspbian container:
Alternatively to test with a specific
rustup-initbuild and usestrace:--default-host arm-unknown-linux-gnueabihfis irrelevant here, as the toolchain architecture does not matter: it fails atrustup-init. I just added it to avoid possible confusion, asrustup-initselects by defaultarmv7-unknown-linux-gnueabihffor the toolchain in this case, even ifuname -mreturnsarmv6l, whilerustup-init.shcorrectly selects thearm-unknown-linux-gnueabihfbuild ofrustup-init. Some inconsistency, but a different topic.When not using the
arm-unknown-linux-gnueabihfbuild ofrustup-init, but thearmv7-unknown-linux-gnueabihfone instead (skipping theunameswapper or downloading it explicitly), for me it works, other than OP here. Maybe something has changed in the meantime.Testing the new
rustup-initdev version:@djc Yes, if I have correctly understood @MichaIng’s report:
In the container they first issue:
… note the default target was detected as
armv7. I then suggested (in https://github.com/rust-lang/rustup/issues/4189#issuecomment-2664443987) trying out our beta build, but got negative feedback:This is why I’m asking for a minimal reproduction to make sure we are on the same page.
For good measure I also tried the same Dockerfile but with Ubuntu 18.04, 20.04 and 22.04 and I see the exact same behaviour across all of them.
That certainly looks like docker is providing the host’s cpuinfo through. This is the issue we had on reqwest - https://github.com/seanmonstar/reqwest/issues/642 which eventually boiled down to https://github.com/lxc/lxcfs/issues/553 which is what eventually fixed it in that case - I wonder if Docker needs equivalent help.