drive: Rate Limit errors trigger full update
I use drive in this daily script to backup a directory tree on Linux mint 17.2 64 bit.
!/bin/sh -x
exec > /home/xxxx/edev/drive/Gdrivedate +%Y_%_%d_%H%M.log
exec 2>&1
at 1300 -f ~/edev/drive/Gdrive.sh
cd ~/Gdata drive version date drive push -depth -1 -no-prompt --exclude-ops delete --verbose ~/Gdata date drive list -depth -1 --sort name -version exit
No files have changed for several days so the last few log files looked like this…
warning: commands will be executed using /bin/sh job 163 at Thu Nov 12 13:00:00 2015 drive version: 0.3.2 Commit Hash: <CURRENT_COMMIT> Go Version: <GO_VERSION> OS: <OS_INFO> BuildTime: Wed Nov 11 13:00:00 AEST 2015 Resolving… Everything is up-to-date. Wed Nov 11 13:00:54 AEST 2015 /0Archive v47280 /Documents v49511 /Pictures v43220 [snip long list of files]
Today drive attempted to upload the entire dir tree…
warning: commands will be executed using /bin/sh job 164 at Fri Nov 13 13:00:00 2015 drive version: 0.3.2 Commit Hash: <CURRENT_COMMIT> Go Version: <GO_VERSION> OS: <OS_INFO> BuildTime: Thu Nov 12 13:00:00 AEST 2015 Resolving… googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
0 / 1146126235 0.00 %
512 / 1146126235 0.00 % 497h51m2s
131584 / 1146126235 0.01 % 2h25m15s
164352 / 1146126235 0.01 % 2h19m32s
263168 / 1146126235 0.02 % 1h41m39s
[snip]
4726664 / 1146126235 0.41 % 2h52m19s
4759432 / 1146126235 0.42 % 2h51m56sremoteMod/remoteMkdirAll: /Pictures got googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
push: /Pictures/2009-05-05–15.03.58 err: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
4792200 / 1146126235 0.42 % 2h51m33s [snip several more rate errors]
Have the rate errors disrupted or aborted the resolving so the entire tree will be uploaded ?
regards CB
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Reactions: 1
- Comments: 44 (12 by maintainers)
Commits related to this issue
- propagate and handle remote file resolution errors Fixes https://github.com/odeke-em/drive/issues/480. Fixes https://github.com/odeke-em/drive/issues/668. Fixes https://github.com/odeke-em/drive/issu... — committed to odeke-em/drive by odeke-em 8 years ago
- propagate and handle remote file resolution errors Fixes https://github.com/odeke-em/drive/issues/480. Fixes https://github.com/odeke-em/drive/issues/668. Fixes https://github.com/odeke-em/drive/issu... — committed to odeke-em/drive by odeke-em 8 years ago
I also get a “got googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded”
I have tried increasing the chunk size to 64MiB to see if it is data rate issue or a “number of API requests” issue. The number of 403 instances increases when the chunk size increases. So it seems to be a data rate issue rather than a number of requests issue.
Running Master branch as on 17/12/2018.
Hey y’all, please get the latest. Thank you very much for all the patience and the bug reports. Finally fixed this problem that was caused by not properly relaying remote resolution errors back to the callers. The fix is in https://github.com/odeke-em/drive/pull/741. Please get the latest code and even in these rate limit/resolutions problems drive will now be able to relay encountered errors and properly abort instead of performing the wrong behavior.