rclone: dropbox (batch) upload encounters internal_error/, eventually stalls for hours
The associated forum post URL from https://forum.rclone.org
What is the problem you are having with rclone?
Periodic “batch didn’t complete” errors, and “internal_error/”, eventually stalls for hours.
$ rclone --compress-level 6 copy --max-age 24h --immutable data dropbox:abcd-db/2021/ab/data --transfers=16 --fast-list -P
2021-07-28 00:27:30 ERROR : Dropbox root 'abcd-db/2021/ab/data': sync batch commit: failed to commit batch length 3: wait for batch failed after 120 tries: batch didn't complete
2021-07-28 00:57:38 ERROR : Dropbox root 'abcd-db/2021/ab/data': sync batch commit: failed to commit batch length 9: wait for batch failed after 120 tries: batch didn't complete
2021-07-28 04:24:40 ERROR : Dropbox root 'abcd-db/2021/ab/data': sync batch commit: failed to commit batch length 2: wait for batch failed after 120 tries: internal_error/
2021-07-28 04:37:48 ERROR : Dropbox root 'abcd-db/2021/ab/data': sync batch commit: failed to commit batch length 1: wait for batch failed after 120 tries: internal_error/
2021-07-28 09:09:56 ERROR : Dropbox root 'abcd-db/2021/ab/data': sync batch commit: failed to commit batch length 1: wait for batch failed after 120 tries: batch didn't complete
Transferred: 47.007Gi / 94.672 GiByte, 50%, 0 Byte/s, ETA 0s
Transferred: 9874 / 19898, 50%
Elapsed time: 13h12m36.2s
Transferring:
* 16/1603f1e23740fdc0409…b92b03273b94a4d7994f13:100% /4.351Mi, 0/s, 0s
* 16/16047c392cef6414c62…d865c52cd2d52fde0b5e65:100% /4.636Mi, 0/s, 0s
* 16/1605dc8faed7a97b6bd…3438aa9271816e1648eb48:100% /4.117Mi, 0/s, 0s
* 15/1567bd4c08f70c36bbc…f3f5db36379d3232bd628c:100% /5.355Mi, 0/s, 0s
* 15/156ed0c5721219d689e…816a40ec6221eb3afdbcd2:100% /4.329Mi, 0/s, 0s
* 15/156effb85b203ff1779…80d73e24fe8f4e39b4ab1a:100% /4.113Mi, 0/s, 0s
* 15/156f5780f0bf3d57449…9017b188391d5d62daa2b6:100% /4.320Mi, 0/s, 0s
* 15/156fdf0842375137277…559906589bf2eeb15de4f2:100% /4.643Mi, 0/s, 0s
* 15/15709b7f128dafbba0b…cd03e3ba53c4a09c0d891f:100% /4.260Mi, 0/s, 0s
* 15/1570e79fbae580647e0…b65602d7daa0327437b7bc:100% /4.752Mi, 0/s, 0s
* 15/15725bb2b702a4300aa…11e3eb291f8dfa35fbbe3e:100% /4.692Mi, 0/s, 0s
* 15/157260c8b208be72180…e8f15f2a0a19d7d4f89f46:100% /4.638Mi, 0/s, 0s
* 26/26f4361399afb974e2b…5bd21da68d5650530b7262:100% /6.773Mi, 0/s, 0s
* 26/26f46f7f4c6a12a2059…2030baa9c2e93bbe8969ca:100% /4.063Mi, 0/s, 0s
* 1f/1f62c5168ac628a2cdb…858604af9efbb8c8a36f39:100% /4.039Mi, 0/s, 0s
* 2f/2f3a2118cd0763edf31…e0364bab861cd16e74f33f:100% /4.758Mi, 0/s, 0s
What is your rclone version (output from rclone version
)
rclone v1.56.0
- os/version: debian 9.13 (64 bit)
- os/kernel: 4.19.0-0.bpo.13-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.16.5
- go/linking: static
- go/tags: none
Which OS you are using and how many bits (e.g. Windows 7, 64 bit)
Which cloud storage system are you using? (e.g. Google Drive)
Dropbox
The command you were trying to run (e.g. rclone copy /tmp remote:tmp
)
rclone --compress-level 6 copy --max-age 24h --immutable data dropbox:abcd-db/2021/ab/data --transfers=16 --fast-list -P
A log from the command with the -vv
flag (e.g. output from rclone -vv copy /tmp remote:tmp
)
Rerunning with -vv
now. Will update when problems are logged
How to use GitHub
- Please use the 👍 reaction to show that you are affected by the same issue.
- Please don’t comment if you have no relevant information to add. It’s just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 2
- Comments: 28 (13 by maintainers)
Commits related to this issue
- dropbox: add --dropbox-batch-commit-timeout to control batch timeout This also adds an Debug message log showing how long each batch took See: #5491 — committed to rclone/rclone by ncw 3 years ago
- dropbox: add --dropbox-batch-commit-timeout to control batch timeout This also adds an Debug message log showing how long each batch took See: #5491 — committed to rclone/rclone by ncw 3 years ago
I’ll give it a try and report back