rclone: Hubic - Unsolicited response received on idle HTTP: 408 Request Time-out
For reference: https://forum.rclone.org/t/hubic-unsolicited-response-received-on-idle-http-408-request-time-out/3453
Description: Since 2-3 weeks, when using the below command with Hubic: rclone -v --checksum --log-file /var/tmp/hubic-backup.log sync /backup/ hubic:default/backup/ I receive the below errors on all commands being run at Hubic via rclone: Unsolicited response received on idle HTTP channel starting with "HTTP/1.0 408 Request Time-out\r\nCache-Control: no-cache\r\nConnection: close\r\nContent-Type: text/html\r\n\r\n408 Request Time-out\nYour browser didn’t send a complete request in time.\n\n I already refreshed the token - it didn’t help.
Version: rclone v1.36
OS: centos-release-7-3.1611.el7.centos.x86_64
Storage: HUBIC
Command being run:
rclone -v --checksum --log-file /var/tmp/hubic-backup.log sync /backup/ hubic:default/backup/
Extract of the log:
Unsolicited response received on idle HTTP channel starting with "HTTP/1.0 408 Request Time-out\r\nCache-Control: no-cache\r\nConnection: close\r\nContent-Type: text/html\r\n\r\n408 Request Time-out\nYour browser didn’t send a complete request in time.\n\n
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 7
- Comments: 40 (10 by maintainers)
--contimeout 1m0schanging this has no effect (e.g.10m0s)--timeout 5m0schanging this has no effect (e.g.50m0s)--stats 1m0schanging this has no effect (e.g.10m0s)--checkers 4all values yields408response except--checkers 1, which always succeed.--checkers 2occasionally succeed.--tpslimit 23 --tpslimit-burst 123 transactions per second is magic number for me, <=23 always succeed, >23 always fails. The burst needs to be kept low, <=4.I was testing the parameters with
ncdu, i.e.rclone --tpslimit 23 ncdu hubic:default.I can repro uploading a large tree with 200k little files to Rackspace Cloud Files as well. It seems as though rclone opens http connections but then does nothing with them for an extended period while it is trolling the directory tree locally.
HI, I’m using rclone to copy files from Digital Ocean spaces to Amazon S3 busckets and I’m getting the same error 408 Request Time-out.
Does that error mean that some file failed to be copied? can I lose data?
Thanks @vovcacik @UltimateByte for help with command line args
@jay1337 if you try the latest beta you should find it handles this a bit better.
See
--conntimeoutflag3 OVH datacenters were down today because of a huge power delivery issue, they are under loads of issues to get all services back online, so probably not the best time to diagnose this ^^
Thanks @vovcacik The following arguments allowed the download of my Hubic backup to continue quite OK:
--tpslimit 20 --tpslimit-burst 2Before that, it would fail after just a few seconds.Note that some 408 errors occurred anyways, so it might be a good idea to put the limit at 18 and the burst at 1, just to be safe. (my download is too advanced now, so I cannot test until it’s over, in around 4.912 days according to my calculations)
Full command syntax & real world example to download a backup:
rclone --tpslimit 20 --tpslimit-burst 2 copy cloud:HubiC-DeskBackup_Work /srv/dev-disk-by-label-Work/