rclone: drive: large server side copies failing with 500 error

Thank you for reading my question.

I an using “rclone copy” to transfer shared folders to my google drive. But something went wrong.

code here

rclone copy -v mydrive:A mydrive:A
rclone sync mydrive:A mydrive:A

sync error code here

2019/03/22 19:39:16 ERROR : S H E-2gether 4ever Encore live concert in Taipei 2014 1080i BluRay REMUX AVC DTS -HD MA 5.1 -HDS/BDMV/STREAM/00000.m2ts: Failed to copy: googleapi: Error 500: Internal Error, internalError
2019/03/22 19:39:16 ERROR : Google drive root 'Media/Music': not deleting files as there were IO errors
2019/03/22 19:39:16 ERROR : Google drive root 'Media/Music': not deleting directories as there were IO errors
2019/03/22 19:39:16 ERROR : Attempt 1/3 failed with 1 errors and: googleapi: Error 500: Internal Error, internalError
2019/03/22 21:51:25 ERROR : S H E-2gether 4ever Encore live concert in Taipei 2014 1080i BluRay REMUX AVC DTS -HD MA 5.1 -HDS/BDMV/STREAM/00000.m2ts: Failed to copy: googleapi: Error 500: Internal Error, internalError
2019/03/22 21:51:25 ERROR : Google drive root 'Media/Music': not deleting files as there were IO errors
2019/03/22 21:51:25 ERROR : Google drive root 'Media/Music': not deleting directories as there were IO errors
2019/03/22 21:51:25 ERROR : Attempt 2/3 failed with 1 errors and: googleapi: Error 500: Internal Error, internalError
2019/03/23 00:09:55 ERROR : S H E-2gether 4ever Encore live concert in Taipei 2014 1080i BluRay REMUX AVC DTS -HD MA 5.1 -HDS/BDMV/STREAM/00000.m2ts: Failed to copy: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
2019/03/23 00:09:55 ERROR : Google drive root 'Media/Music': not deleting files as there were IO errors
2019/03/23 00:09:55 ERROR : Google drive root 'Media/Music': not deleting directories as there were IO errors
2019/03/23 00:09:55 ERROR : Attempt 3/3 failed with 1 errors and: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
2019/03/23 00:09:55 Failed to sync: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded

copy error code here

2019/03/23 16:44:06 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:              9219 / 9219, 100%
Transferred:            0 / 1, 0%
Elapsed time:    6h6m1.4s
Transferring:
 * 极客时间-已经完结的课程/11 大咖说/In…16期 续彬/InfoQ大咖说-续彬.m4v: transferring

2019/03/23 16:45:06 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:              9219 / 9219, 100%
Transferred:            0 / 1, 0%
Elapsed time:    6h7m1.4s
Transferring:
 * 极客时间-已经完结的课程/11 大咖说/In…16期 续彬/InfoQ大咖说-续彬.m4v: transferring

This code block has been repeated all the time, but the file cannot be copied over.

Firstly, I am using a deep os with a memory size of 0.6g, rclone v1.46. And I added 2g of swap space.

Second, I have replaced default api by my own. And the upper limit of 750G is far from reaching.

Finally, I try to replace “rclone copy” with “rclone sync”,but the problem still exists.

I would like to know the reason for this problem and how to fix this problem.

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 40 (25 by maintainers)

Commits related to this issue

drive: fix server side copy of big files - fixes #3070 Having a MimeType in the Copy parameters causes certain big file copies to error with "500" errors. @darthShadow discovered that removing the m... — committed to rclone/rclone by ncw 5 years ago
drive: fix server side copy of big files Before this change rclone was sending a MimeType in the requests for server side Move and Copy. The conjecture is that if you attempt to set the MimeType to ... — committed to rclone/rclone by ncw 5 years ago

Most upvoted comments

@IIeTp Server Side Copies are disabled by default in 1.48. Are you using the --drive-server-side-across-configs flag to enable it?

darthShadow on Jul 4, 2019

Super debugging @darthShadow well done 😃

I tried the example above and it worked perfectly.

I made a beta of rclone with this change in for everyone to try

https://beta.rclone.org/branch/v1.48.0-062-gde20031e-fix-3070-drive-copy-beta/ (uploaded in 15-30 mins)

ncw on Jul 4, 2019

I tried out a couple of variants of the above request using different parameters via curl and the API explorer available at https://developers.google.com/drive/api/v3/reference/files/copy.

Looks like the issue is with the mimeType. As soon as I don’t specify a mimeType in the request it starts working.

I have been able to replicate similar behaviour on my own files which failed to copy server-side. Their original mimeType was video/x-matroska which timed out after ~90s with the same error but if I tried copying the same files with the mimeType set to application/octet-stream or even no mimeType, it works.

darthShadow on Jul 4, 2019