rclone: Rclone sync fails - request wrongly url encoded?
Continuation of: https://forum.rclone.org/t/rclone-sync-fails-request-wrongly-url-encoded/23152
What is the problem you are having with rclone?
rclone sync fails with the error Couldn’t delete: Container Not Found Other commands work correctly, even rclone deletefile when deleting the file that failed during rclone sync
I have an analysis of what seems to be going wrong, the rclone sync command seems to be url encoding the request that fails. When I manually execute the request using curl, the url encoded variant returns a 404 - Not found.
If I manually decode the request then the response is as you would expect.
Request that fails:
HTTP REQUEST (req 0xc0024db300) 2021/03/25 16:17:13 DEBUG : GET /production%255Fencrypted%255Fsegments?format=json&limit=1000&prefix=mariadbservers%2Fweekly%2Fpiwik%255F2021%252D02%252D10.gz.bin%2F1612992682.03079176%2F9221554274 HTTP/1.1 Same request decoded, succeeds.
From Nick Craig-Wood: What it looks like is that rclone failed to URL decode the Manifest in the previous response
X-Object-Manifest: production%5Fencrypted%5Fsegments/mariadbservers/weekly/piwik%5F2021%2D02%2D10.gz.bin/1612992682.03079176/9221554274
And URL encoding that again produces the rather odd
GET /production%255Fencrypted%255Fsegments
What is your rclone version (output from rclone version)
I tried today’s latest beta.
Main version we are using though:
rclone v1.54.0
- os/arch: linux/amd64
- go version: go1.15.7
Which OS you are using and how many bits (e.g. Windows 7, 64 bit)
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.7 LTS
Release: 16.04
Codename: xenial
Which cloud storage system are you using? (e.g. Google Drive)
A combination of Openstack objectstore (Swift) and the encryption backend.
[backup]
auth = XXX
key = XXXX
tenant = XXX
type = swift
user = XXX
[backup_encrypted]
directory_name_encryption = false
filename_encryption = off
password = XXXXX
password2 = XXXX
remote = backup:/acceptance_encrypted/
type = crypt
The command you were trying to run (e.g. rclone copy /tmp remote:tmp)
rclone sync /mnt/backup/mysql backup_encrypted:mariadbservers --transfers=20 --skip-links --stats 0
A log from the command with the -vv flag (e.g. output from rclone -vv copy /tmp remote:tmp)
Logs for version master build 1.55: https://pastebin.com/pM11bBwS The relevant section in the log is the same for the latest beta.
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 15 (7 by maintainers)
@ncw I think I found the issue in the swift library: https://github.com/ncw/swift/pull/168