rustup: fresh install fails on Windows: "error: could not rename component file from '... \rust-docs\share/doc/rust/html' ..."

In addition, you could try setting RUSTUP_IO_THREADS=disabled in the environment and see if it behaves with that.

I have the same problem. Corporate laptop with McAfee that can not be turn off. I need this release in order to start learning rust. Please hurry 😃

Yay, I can confirm that nuking .rustup folder and re-installing using the latest build from CI worked.

Unfortunately it does not work for me, I am still getting the following

info: installing component 'rust-docs'
 11.3 MiB /  11.3 MiB (100 %) 252.8 KiB/s in  2m 21s ETA:  0s
info: retrying renaming 'C:\Users\---\.rustup\tmp\fry3x4mosy837ftc_dir\rust-docs\share/doc/rust/html' to 'C:\Users\---\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'

I have a corporate with McAfee enabled, and this is preposterous that docs are blocked from being moved (hey, if it was a virus, it would already have infected my computer).

@esbgo97, most corporate laptops do not allow turning off the Antivirus.

It is a big pain, I agree. McAfee is among the worst offenders 😦

If I could disable my anti-virus temporarily I’m sure it would fix this issue. However, at work I am unable to disable the anti-virus program. This makes it impossible for me to use rustup update.

Is it possible to make it so rustup won’t give up after the very first attempt at moving? Or give us an option to, instead of moving the files, try copying to destination and deleting from source instead?

RUSTUP_IO_THREADS=disabled worked for me as well.

I think it may be due to the fact that McAfee is single threaded.

We’ve been improving the performance of rustup’s IO because it used to be execrable on Windows and on high-latency non-Windows platforms. We might simply be too fast for McAfee these days, and previously we were slow enough that you never saw it misbehaving. I guess we could look at offering a --slow-io option or somesuch if you think that might help.

To me it says “retry renaming” for a while and then it fails on rustup 1.19.0 (2af131cf9 2019-09-08), Windows 10.

info: rolling back changes
error: could not rename component file from 'C:\Users\gborruni\.rustup\tmp\akm490eg7dpm6pbd_dir\rust-docs\share/doc/rust/html' to 'C:\Users\gborruni\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: caused by: permission denied

Corporate McAfee enabled, cannot disable.

Issue is not restricted to fresh install. I did a rustup update before nuking .rustup and got the same error.

The latest version of rustup (out yesterday) has this change in.

At the very least I’d suggest it’s worth adding a panic or log message to that code in utils.rs so we get a clear message (rather than futile retries!) if the dest directory does exist - at least then we’ll know. I’m struggling to find another reason it would behave like this. And that code is clearly assuming that PermissionDenied errors are worth retrying which is only a valid assumption if the destination doesn’t already exist, so this would be a safe check to add.

I did an experiment creating a trivial Rust program that just runs fs::rename (i.e. not using rustup) and I was able to successfully rename that html directory (including moments after unzipping it, so freshly written) without any problem from McAfee. So there must be another factor making it worse when rustup does it, not sure what that is.

We’ve been assuming that the PermissionDenied error is caused by the anti-virus problem every time it retries… but actually the anti-virus problem maybe only caused the very first PermissionDenied.

The utils.rs code for the rename function tries to rename src to dest, and if it fails tries again to rename src to dest after a backoff time. I think this logic is faulty (at least on Windows) if the rename “partly” fails (i.e. some files are moved but others are not). In that case, every time it does a retry the dest will already exist and on Windows guess what error you get when renaming to a directory dest that already exists - a code:5 PermissionDenied! 😃 In which case no matter how many times it retries it’s never gonna work.

So maybe what we need to make this robust is some retry logic that can cope with the dest already existing?

The description in https://en.cppreference.com/w/cpp/filesystem/rename suggests that on POSIX is the new directory already exists it will be deleted first but a) a trivial invocation of fs::rename shows that isn’t what happens on Windows and b) even if it was, that would be dangerous for the way rustup is using rename in the case where the src directory was partly moved to dest since we dont’ want to delete away the chunk of documentation that was successfully moved and end up with half the files missing.

I just hit the same thing running “rustup-init” to get Rust installed for the first time - so looks like it’s still a problem with latest Rust and latest McAfee. Had to do a “minimal” install to avoid the failure (the disable threads env var didn’t do it for me).

Maybe it’s worth increasing the retry backoff as suggested by @rbtcollins ?

You could try a much older rustup, e.g. 1.17.0 from https://static.rust-lang.org/rustup/archive/1.17.0/x86_64-pc-windows-msvc/rustup-init.exe though you will need to ensure you pass --no-self-update otherwise it’ll end up upgrading itself

Badly behaved scan-on-write virus scanners are becoming a great pain to me. I wonder if there’s any way we can detect and mitigate this.

I’m closing this as the symptom is well resolved. I’m going to file a new bug about possible future improvements.

Just tried it again now. The installation seems to work great. Despite that it temporarily hangs up / long loops on retrying renaming 'C:\Users\username\.rustup\tmp\utq1gp9blz83pn2s_dire\rust-docs\share/doc/rust/html' to 'C:\Users\username\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'

for several dozen retries…it does succeed in installation now.

Thank you so much, you guys are fantastic!

Ok, I went away and did some tests. I don’t have a McAfee so I made my own. You can find my code here.

It seems to work but there could of course be other factors at play that I’m missing (or my AV simulator is broken in some way). I’ve made a rustup fork (rename_dir branch) for real world testing if anybody is willing. The only changes are to utils.rs. The idea is to catch a moment where all files have been closed but none are yet open. This may be a rare occurrence but, with the vagaries of both threads and file I/O, the stars should hopefully align eventually. It could go without the yield_now at all but I’d worry about the chance of starving a process that needs to make forward progress to release a handle. Though admittedly this is very unlikely with a multi-core system and a scheduler that can move threads between cores.

To be clear, I’m not saying this will definitely work or is the best idea. But I am interested in file handling in Windows and would at least like to rule out this possibility.

My thought is to perform directory moves by doing the following:

1. Walk the src directory
2. Use hardlinks (if possible) to “copy” each file to dst.
3. Mark the src files for deletion but ignore any errors (if those errors are causing an issue).

Perform any necessary cleanup right before exit, informing the user if this fails. I think this isn’t too difficult if the src directory was a subfolder of the tmp folder?

Just to give you a datapoint for how long to retry - I had a file read (50kb text file) this morning block for just over 65 seconds because of McAffee - it was consuming 100% of one core through the whole operation. We could easily be looking at a similar amount of time for the rust install.

@s-andrews you may have something different going on; if you can start with the console output you see that would be helpful.

I’ve put the console output below. The procmon log file can be downloaded from http://ftp1.babraham.ac.uk/ftpusr76/rust_install.zip for the next month.

There was a pause at the end for around 20 seconds and McAffee was completely swamping the CPU through the whole operation.

Incidentally there was a (presumably) similar failure when I tried to uninstall to do this test.

C:\Users\andrewss\Downloads>rustup-init.exe

Welcome to Rust!

This will download and install the official compiler for the Rust
programming language, and its package manager, Cargo.

It will add the cargo, rustc, rustup and other commands to
Cargo's bin directory, located at:

  C:\Users\andrewss\.cargo\bin

This can be modified with the CARGO_HOME environment variable.

Rustup metadata and toolchains will be installed into the Rustup
home directory, located at:

  C:\Users\andrewss\.rustup

This can be modified with the RUSTUP_HOME environment variable.

This path will then be added to your PATH environment variable by
modifying the HKEY_CURRENT_USER/Environment/PATH registry key.

You can uninstall at any time with rustup self uninstall and
these changes will be reverted.

Current installation options:


   default host triple: x86_64-pc-windows-msvc
     default toolchain: stable
               profile: default
  modify PATH variable: yes

1) Proceed with installation (default)
2) Customize installation
3) Cancel installation
>1

info: profile set to 'default'
info: default host triple is x86_64-pc-windows-msvc
info: syncing channel updates for 'stable-x86_64-pc-windows-msvc'
info: latest update on 2020-05-07, rust version 1.43.1 (8d69840ab 2020-05-04)
info: downloading component 'cargo'
  3.2 MiB /   3.2 MiB (100 %)   1.8 MiB/s in  2s ETA:  0s
info: downloading component 'clippy'
info: downloading component 'rust-docs'
 12.1 MiB /  12.1 MiB (100 %) 979.2 KiB/s in 10s ETA:  0s
info: downloading component 'rust-std'
 14.8 MiB /  14.8 MiB (100 %)   1.7 MiB/s in  9s ETA:  0s
info: downloading component 'rustc'
 36.3 MiB /  36.3 MiB (100 %)   1.8 MiB/s in 22s ETA:  0s
info: downloading component 'rustfmt'
  2.0 MiB /   2.0 MiB (100 %)   1.2 MiB/s in  2s ETA:  0s
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'rust-docs'
 12.1 MiB /  12.1 MiB (100 %) 449.6 KiB/s in 26s ETA:  0s
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: rolling back changes
error: could not rename component file from 'C:\Users\andrewss\.rustup\tmp\fvkhz7zkmy_n6zxw_dir\rust-docs\share/doc/rust/html' to 'C:\Users\andrewss\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
error: caused by: permission denied

Press the Enter key to continue.

Hi

Tried to install rust today same issue This is my work computer and I cannot disable the McAfee anti-virus

Is there any other possible workarounds

I think a good solution is to provide a way to copy instead of move those files. Either by including it in the backoff step or by giving users an option.

This has been asked before and the answer seems to be just to retry the operation in a loop. https://stackoverflow.com/questions/964920/is-it-possible-to-reasonably-workaround-an-antivirus-scanning-the-working-direct

RUSTUP_IO_THREADS=disabled did the trick for me, many thanks! Guess it makes it slower but hey! 😃

UPDATE: the trick works for rustup install but not rustup update. The workaround is to rustup uninstall followed by rustup install rather than rustup update for a given toolchain.

It might be possible to get around this restriction by doing copies instead of moves, and later deleting the source. This would temporarily use more disk space but I think it’s better than not being able to use rust.

@norru If they would be prepared to add rustup to the low-risk program group then it might allow rustup to install toolchains more effectively. The content would then be scanned on first use, rather than on-install. This shouldn’t increase the risk to your computer but would allow Rust to install. If corporate IT is not prepared to do this, then I’m not sure there’s a lot I can do to help since I have no idea how to work around McAfee breaking filesystem semantics left-and-right.

I fear McAfee is being sufficiently awful that there’s little we can do right now. Can your company IT add a folder exception for rust toolchains?

If it says “retry renaming” instead of failing then the fix is working.

I have the same issue here with the Avira anti-virus. Turning off the “Real-Time Protection” before running rustup-init.exe makes it work.

Thank you for generating this report. It does look to me like a probable anti-virus problem.

You mentioned in 1723 that you have procmon traces - have a look in that at the time of the error, are there any other processes with handles open anywhere in that subdirectory? I’m happy to look through if you can upload the PML (as a zip/tar) to s3 or seed it as a torrent or some such…