rclone: Invalid characters in the Local File System cause rclone copy and sync to re-download existing files under the new name

What is the problem you are having with rclone?

In Google Drive, you can store folders under any name. This implies that local file systems (Windows, in this case) may not be able to perfectly copy file names, since they contain invalid file names. After rclone replaces these invalid characters, it does not seem to compare files under the new (fixed) path under the other configuration destination, causing rclone to re-download all files into the destination.

For example, if I have a folder named “TEST:TEST” in Google Drive, and run copy/sync, rclone will download it to the Windows FS as “TEST_TEST”, and then copy all files inside (remote) “TEST:TEST” into (local) “TEST_TEST”. Running the same copy/sync command, however, then causes rclone to re-download all files in "TEST:"TEST to “TEST_TEST” again, as it does not seem to check the correct folder name’s contents.

What is your rclone version (output from rclone version)

rclone v1.43.1, go version: go1.11

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 Pro, 64-bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive to Windows FS

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy (remote:) (local_path)

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

PS C:\Users<USERNAME>\Documents\rclone> .\rclone.exe -vv copy <GOOGLE_DRIVE_CONFIG> <LOCAL_PATH> 2018/10/08 14:05:18 DEBUG : rclone: Version “v1.43.1” starting with parameters [“C:\Users\<USERNAME>\Documents\rclone\rclone.exe” “-vv” “copy” “<GOOGLE_DRIVE_CONFIG>” “<LOCAL_PATH”] 2018/10/08 14:05:18 DEBUG : Using config file from “C:\Users\<USERNAME>\.config\rclone\rclone.conf” 2018/10/08 14:05:24 INFO : Local file system at \?<LOCAL_PATH>: Waiting for checks to finish 2018/10/08 14:05:24 INFO : Local file system at \?<LOCAL_PATH>: Waiting for transfers to finish 2018/10/08 14:05:24 NOTICE: Local file system at \?<LOCAL_PATH>: Replacing invalid characters in “\<LOCAL_PATH>\TEST:TEST\<FILE NAME>” to “\\?\E:\<LOCAL_PATH>\TEST_TEST\<FILE NAME>” 2018/10/08 14:06:23 INFO : Transferred: 740.996M / 2.140 GBytes, 34%, 11.344 MBytes/s, ETA 2m7s Errors: 0 Checks: 7 / 7, 100% Transferred: 0 / 1, 0% Elapsed time: 1m5.3s Transferring:

• …<FILE NAME>: 33% /2.140G, 12.537M/s, 1m55s

2018/10/08 14:07:23 INFO : Transferred: 1.465G / 2.140 GBytes, 68%, 11.969 MBytes/s, ETA 57s Errors: 0 Checks: 7 / 7, 100% Transferred: 0 / 1, 0% Elapsed time: 2m5.3s Transferring:

• …<FILE NAME>: 68% /2.140G, 12.283M/s, 56s

2018/10/08 14:08:19 INFO : <FILE NAME>: Copied (new) 2018/10/08 14:08:19 INFO : Transferred: 2.140G / 2.140 GBytes, 100%, 12.093 MBytes/s, ETA 0s Errors: 0 Checks: 7 / 7, 100% Transferred: 1 / 1, 100% Elapsed time: 3m1.2s

2018/10/08 14:08:19 DEBUG : 4 go routines active 2018/10/08 14:08:19 DEBUG : rclone: Version “v1.43.1” finishing with parameters [“C:\Users\<USERNAME>\Documents\rclone\rclone.exe” “-vv” “copy” “GOOGLE_DRIVE:” “LOCAL_PATH”]