rustup: rustup fails with symlink RUSTUP_HOME/toolchains

Problem

% rustup update
error: I/O Error: Too many levels of symbolic links (os error 40)

BTW: This message isn’t very helpful.

I’m having a symlink at ~/.rustup/toolchains pointing to ~/.cache/rustup/toolchains

Steps

  1. mkdir ~/rustup-test && cd ~/.rustup && cp -r downloads tmp update-hashes settings.toml ~/rustup-test
  2. ln -s ~/.rustup/toolchains ~/rustup-test/
  3. RUSTUP_HOME=$HOME/rustup-test rustup update

Possible Solution(s)

No response

Notes

This happens since the update from

   nightly-x86_64-unknown-linux-gnu updated - rustc 1.78.0-nightly (3b1717c05 2024-03-10) (from rustc 1.78.0-nightly (2d24fe591 2024-03-09))
   stable-x86_64-unknown-linux-gnu unchanged - rustc 1.76.0 (07dca489a 2024-02-04)

to

    nightly-x86_64-unknown-linux-gnu updated - rustc 1.78.0-nightly (3cbb93223 2024-03-13) (from rustc 1.78.0-nig
htly (3b1717c05 2024-03-10))
   stable-x86_64-unknown-linux-gnu unchanged - rustc 1.76.0 (07dca489a 2024-02-04)
% RUSTUP_HOME=$HOME/rustup-test strace -Zk -e t=openat rustup update
10:27:49.009219 openat(AT_FDCWD</home/joerg/git/rustup>, "/home/joerg/kein_Backup/rustup-test/toolchains", O_RDONLY|O_NOFOLLOW|O_CLOEXEC) = -1 ELOOP (Too many levels of symbolic links) <0.000013>
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__open64+0x51) [0xf7861]
 > /home/joerg/.cache/cargo/bin/rustup(std::sys::unix::fs::File::open_c+0xe8) [0x511148]
 > /home/joerg/.cache/cargo/bin/rustup(std::fs::OpenOptions::_open+0x12a) [0x5391fa]
 > /home/joerg/.cache/cargo/bin/rustup(rustup::toolchain::toolchain::Toolchain::exists+0x172) [0x3fb192]
 > /home/joerg/.cache/cargo/bin/rustup(rustup::toolchain::toolchain::Toolchain::new+0x34) [0x3fac74]
 > /home/joerg/.cache/cargo/bin/rustup(rustup::toolchain::distributable::DistributableToolchain::new+0x3f) [0x3d92af]
 > /home/joerg/.cache/cargo/bin/rustup(<core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::next+0xf80) [0x4e8550]
 > /home/joerg/.cache/cargo/bin/rustup(rustup::config::Cfg::list_channels+0x9a) [0x4e692a]
 > /home/joerg/.cache/cargo/bin/rustup(rustup::cli::rustup_mode::update+0x3b6) [0x4a38c6]
 > /home/joerg/.cache/cargo/bin/rustup(rustup::cli::rustup_mode::main+0x1b2d) [0x47e09d]
 > /home/joerg/.cache/cargo/bin/rustup(rustup_init::main+0x1262) [0xeea32]
 > /home/joerg/.cache/cargo/bin/rustup(std::sys_common::backtrace::__rust_begin_short_backtrace+0x3) [0xec8c3]
 > /home/joerg/.cache/cargo/bin/rustup(main+0x414) [0xef3c4]
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_init_first+0x8a) [0x276ca]
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x27785]
 > /home/joerg/.cache/cargo/bin/rustup(_start+0x29) [0xec7f7]
error: I/O Error: Too many levels of symbolic links (os error 40)

Rustup version

rustup 1.27.0 (bbb9276d2 2024-03-08)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.79.0-nightly (2f090c30d 2024-03-23)`

Installed toolchains

Default host: x86_64-unknown-linux-gnu
rustup home:  /home/joerg/.rustup

installed toolchains
--------------------

stable-x86_64-unknown-linux-gnu
nightly-x86_64-unknown-linux-gnu (default)

active toolchain
----------------

nightly-x86_64-unknown-linux-gnu (default)
rustc 1.79.0-nightly (2f090c30d 2024-03-23)

About this issue

  • Original URL
  • State: closed
  • Created 3 months ago
  • Comments: 16 (11 by maintainers)

Commits related to this issue

Most upvoted comments

It works:

% RUSTUP_UPDATE_ROOT='https://dev-static.rust-lang.org/rustup' rustup update
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: latest update on 2024-04-28, rust version 1.79.0-nightly (aed2187d5 2024-04-27)
…
info: checking for self-update
info: downloading self-update

  stable-x86_64-unknown-linux-gnu unchanged - rustc 1.77.2 (25ef9e3d8 2024-04-09)
   nightly-x86_64-unknown-linux-gnu updated - rustc 1.79.0-nightly (aed2187d5 2024-04-27) (from rustc 1.79.0-nightly (ef8b9dcf2 2024-04-24))

info: cleaning up downloads & tmp directories

% mv .rustup/toolchains .cache/rustup/
% ln -s ../.cache/rustup/toolchains .rustup
% cargo --version        
cargo 1.79.0-nightly (b60a15551 2024-04-26)

Is there a way I can downgrade my rustup to a non-broken version and prevent it from re-updating itself?

Yes. To downgrade, download an older version of rustup-init from the dist server https://static.rust-lang.org/rustup and run it to install it.

the env variable RUSTUP_UPDATE_ROOT will override that, so if you set it to an empty web host in your environment, rustup will no longer be able to self-update.

Or, you can pass --no-self-update to rustup when you invoke an update / install. If you’re using an IDE that invokes rustup, consult its documentation.

Indeed thanks for the bug.

@rami3l I think what we need here for .rustup/toolchains and .rustup/downloads, but not for directories in general, are the following semantics:

  • when making the directory, if it exists already and is a symlink to a dir, or is a dir, use it, otherwise error
  • when deleting the directory, if it is a symlink, unlink it, or if a directory, recursively delete with link-following-safety, otherwise error.

I think remove_dir_all + the fs_at crates should permit solving this quite nicely.

re remove_dir_all behaviour, see https://docs.rs/remove_dir_all/0.8.2/remove_dir_all/ the last two paragraphs before ‘features’. tl;dr symlinks are not directories, and if you’re dealing with a symlink, just unlink, don’t expect remove_dir_all to delete it.

re: the semantics on windows, the implementation details are different, but roughly identical semantics are achievable - the open with FILE_FLAG_BACKUP_SEMANTICS is required to read the link out of a link, compared to linux where a different syscall is used to read links.

Happy to review a patch.

@jo-so Okay, I’ll keep both issues open but those two issues do seem related. Unfortunately since this is caused by self-referential symlinks (created presumably in another rustup run other than the current one), your strace isn’t very helpful.

I’ll find some time to investigate into both issues.

It’s not about self-referential symlinks. As you can see in the strace the flag O_NOFOLLOW gets passed to the openat, which causes not even one symlink is crossed.

With a self compiled version you get

% RUSTUP_FORCE_ARG0=rustup RUSTUP_HOME=/home/joerg/kein_Backup/rustup-test strace -Zk -e t=openat target/debug/rustup-init update
11:59:57.099680 openat(AT_FDCWD</home/joerg/git/rustup>, "/home/joerg/kein_Backup/rustup-test/toolchains", O_RDONLY|O_NOFOLLOW|O_CLOEXEC) = -1 ELOOP (Too many levels of symbolic links) <0.000032>
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__open64+0x51) [0xf7861]
 > /home/joerg/git/rustup/target/debug/rustup-init(std::sys::pal::unix::fs::File::open_c+0xfc) [0x10d489c]
 > /home/joerg/git/rustup/target/debug/rustup-init(std::fs::OpenOptions::_open+0x8b) [0x10c73fb]
 > /home/joerg/git/rustup/target/debug/rustup-init(std::fs::OpenOptions::open+0x73) [0x4f4223]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::utils::raw::open_dir+0x64) [0x33d3c4]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::toolchain::toolchain::Toolchain::exists+0x296) [0x1dbe16]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::toolchain::toolchain::Toolchain::new+0xa8) [0x1db848]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::toolchain::distributable::DistributableToolchain::new+0x95) [0x25bfe5]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::config::Cfg::list_channels::{{closure}}+0x80) [0x2f8ba0]
 > /home/joerg/git/rustup/target/debug/rustup-init(core::iter::adapters::map::map_try_fold::{{closure}}+0x69) [0x3637e9]
 > /home/joerg/git/rustup/target/debug/rustup-init(core::iter::adapters::filter::filter_try_fold::{{closure}}+0x121) [0x1a2f21]
 > /home/joerg/git/rustup/target/debug/rustup-init(core::iter::adapters::filter_map::filter_map_try_fold::{{closure}}+0x14c) [0x2e7bac]
 > /home/joerg/git/rustup/target/debug/rustup-init(core::iter::traits::iterator::Iterator::try_fold+0x10f) [0x34e11f]
 > /home/joerg/git/rustup/target/debug/rustup-init(<core::iter::adapters::filter_map::FilterMap<I,F> as core::iter::traits::iterator::Iterator>::try_fold+0x4f) [0x2e692f]
 > /home/joerg/git/rustup/target/debug/rustup-init(<core::iter::adapters::filter::Filter<I,P> as core::iter::traits::iterator::Iterator>::try_fold+0x47) [0x1a1ac7]
 > /home/joerg/git/rustup/target/debug/rustup-init(<core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold+0x40) [0x361230]
 > /home/joerg/git/rustup/target/debug/rustup-init(<core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::try_fold+0x4f) [0x30611f]
 > /home/joerg/git/rustup/target/debug/rustup-init(<core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::next+0x24) [0x305e84]
 > /home/joerg/git/rustup/target/debug/rustup-init(<alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter+0x53) [0x1f22b3]
 > /home/joerg/git/rustup/target/debug/rustup-init(alloc::vec::in_place_collect::<impl alloc::vec::spec_from_iter::SpecFromIter<T,I> for alloc::vec::Vec<T>>::from_iter+0x1e) [0x1fcc1e]
 > /home/joerg/git/rustup/target/debug/rustup-init(<alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter+0x29) [0x2000d9]
 > /home/joerg/git/rustup/target/debug/rustup-init(<core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter::{{closure}}+0x22) [0x1b8302]
 > /home/joerg/git/rustup/target/debug/rustup-init(core::iter::adapters::try_process+0x74) [0x306a44]
 > /home/joerg/git/rustup/target/debug/rustup-init(<core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter+0x27) [0x1b8287]
 > /home/joerg/git/rustup/target/debug/rustup-init(core::iter::traits::iterator::Iterator::collect+0xe) [0x36283e]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::config::Cfg::list_channels+0x106) [0x19f216]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::config::Cfg::update_all_channels+0x3f) [0x19f2df]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::cli::common::update_all_channels+0x54) [0x18d5e4]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::cli::rustup_mode::update+0x43f) [0x2343ef]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::cli::rustup_mode::main+0xd62) [0x226412]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup_init::run_rustup_inner+0x38b) [0x1813bb]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup_init::run_rustup+0x1a2) [0x180ef2]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup_init::maybe_trace_rustup+0xd) [0x180d3d]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup_init::main::{{closure}}+0xe) [0x180a4e]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::currentprocess::with::{{closure}}+0x26c) [0x1822ec]
 > /home/joerg/git/rustup/target/debug/rustup-init(std::thread::local::LocalKey<T>::try_with+0xdc) [0x181cdc]
 > /home/joerg/git/rustup/target/debug/rustup-init(std::thread::local::LocalKey<T>::with+0x31) [0x181bd1]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup::currentprocess::with+0x85) [0x182065]
 > /home/joerg/git/rustup/target/debug/rustup-init(rustup_init::main+0x42) [0x180d22]
 > /home/joerg/git/rustup/target/debug/rustup-init(core::ops::function::FnOnce::call_once+0xb) [0x18250b]
 > /home/joerg/git/rustup/target/debug/rustup-init(std::sys_common::backtrace::__rust_begin_short_backtrace+0xe) [0x183bae]
 > /home/joerg/git/rustup/target/debug/rustup-init(std::rt::lang_start::{{closure}}+0x11) [0x183c21]
 > /home/joerg/git/rustup/target/debug/rustup-init(std::rt::lang_start_internal+0x441) [0x10c2301]
 > /home/joerg/git/rustup/target/debug/rustup-init(std::rt::lang_start+0x3a) [0x183bfa]
 > /home/joerg/git/rustup/target/debug/rustup-init(main+0x1e) [0x1816ae]
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_init_first+0x8a) [0x276ca]
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x27785]
 > /home/joerg/git/rustup/target/debug/rustup-init(_start+0x21) [0x1806d1]
error: I/O Error: Too many levels of symbolic links (os error 40)
11:59:57.159481 +++ exited with 1 +++

Which points to https://github.com/rust-lang/rustup/blob/0c501d55c54031a992395078d535b6d2574dc693/src/utils/raw.rs#L45-L54 that adds the NOFOLLOW flag

contrary to Toolchain::exists() https://github.com/rust-lang/rustup/blob/0c501d55c54031a992395078d535b6d2574dc693/src/toolchain/toolchain.rs#L70-L74

Here’s a git-bisect run. A good version was 1.25.0, and the culprit is 2c9297d51a077d9a364e98ad8ebaee2301cd0c0d.

# bad: [1c612cf65439901200f89c0123dbdf9dcae5e539] fix(ci): fix file paths in CI-generated `*.sha256` files on *nix
# good: [976d72a1920a0266936320afc63b5e651888359f] (changelog): Fix typo
git bisect start 'origin/master' '1.25.0'
# skip: [44c7d94079d19dacdf63fd90cf16297df6293843] Merge pull request #3338 from hi-rustin/rustin-patch-clippy-tip
git bisect skip 44c7d94079d19dacdf63fd90cf16297df6293843
# good: [060b8f9e3f5a8a810b49108aa57cee2e784ee104] Merge pull request #3054 from MoSal/master
git bisect good 060b8f9e3f5a8a810b49108aa57cee2e784ee104
# bad: [d63b6e5f7608369934a4774611af9a83104d3a78] Address `#[warn(clippy::useless_conversion)]`
git bisect bad d63b6e5f7608369934a4774611af9a83104d3a78
# good: [de9a9f08c8b33f58541aa3df0ee80c5244bd5354] Merge pull request #3252 from hi-rustin/rustin-patch-actions-run-test
git bisect good de9a9f08c8b33f58541aa3df0ee80c5244bd5354
# good: [41b631c778d4806f2739e81fc7f4d7b765b8c8b9] Merge pull request #3333 from workingjubilee/use-mutex-const-new
git bisect good 41b631c778d4806f2739e81fc7f4d7b765b8c8b9
# bad: [8a704172114133095086c11b649afed6a64b3d95] CI support for loongarch64-unknown-linux-gnu
git bisect bad 8a704172114133095086c11b649afed6a64b3d95
# good: [395e49270394680dd03c74f69dcfedc6c3283ab5] Suggest right toolchain when running clippy
git bisect good 395e49270394680dd03c74f69dcfedc6c3283ab5
# bad: [2c9297d51a077d9a364e98ad8ebaee2301cd0c0d] Rework Toolchain model and drop relative file path overrides
git bisect bad 2c9297d51a077d9a364e98ad8ebaee2301cd0c0d
# good: [46812bf38a66c51558d4036c630d6640ead45492] Merge pull request #3335 from xfix/toml-0.7.3
git bisect good 46812bf38a66c51558d4036c630d6640ead45492
# good: [8662c33cc983a8fc0690ebb536cca91e027af094] Merge pull request #3343 from cuishuang/master
git bisect good 8662c33cc983a8fc0690ebb536cca91e027af094
# good: [4b29b79af7fe09cc7911528257241be5fbba25c6] Merge pull request #3287 from rbtcollins/tracing
git bisect good 4b29b79af7fe09cc7911528257241be5fbba25c6
# first bad commit: [2c9297d51a077d9a364e98ad8ebaee2301cd0c0d] Rework Toolchain model and drop relative file path overrides