cli-microsoft365: 'spo file add' fails to upload files larger than 250 MB

When trying to upload a file of ~300 MB size using spo file add I get the following error message:

{
    "odata.error": {
        "code": "-1, Microsoft.SharePoint.Client.InvalidClientQueryException",
        "message": {
            "lang": "en-US",
            "value": "The request message is too big. The server does not allow messages larger than 262144000 bytes."
        }
    }
}

It appears that files of 250 MB or more can’t be uploaded in the body of a single request, as done currently.

In old REST API documentation there is mention of endpoints StartUpload, ContinueUpload, FinishUpload and CancelUpload which allow for chunked file uploads.

Would it be possible to switch to a chunked upload approach for large files? Or are there other methods to accomplish this using the CLI?

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 18 (18 by maintainers)

Commits related to this issue

Extended 'spo file add' with chunked uploads solving #1052 — committed to VelinGeorgiev/office365-cli by muecs 5 years ago
Extended 'spo file add' with chunked uploads solving #1052 — committed to VelinGeorgiev/office365-cli by muecs 5 years ago
Extended 'spo file add' with chunked uploads solving #1052 — committed to VelinGeorgiev/office365-cli by muecs 5 years ago
Extended 'spo file add' with chunked uploads solving #1052 — committed to VelinGeorgiev/office365-cli by muecs 5 years ago
Extended 'spo file add' with chunked uploads solving #1052 — committed to VelinGeorgiev/office365-cli by muecs 5 years ago
Extended 'spo file add' with chunked uploads solving #1052 — committed to VelinGeorgiev/office365-cli by muecs 5 years ago
Extended 'spo file add' with chunked uploads solving #1052 — committed to VelinGeorgiev/office365-cli by muecs 5 years ago
Extended 'spo file add' with chunked uploads solving #1052 — committed to pnp/cli-microsoft365 by muecs 5 years ago

Most upvoted comments

@VelinGeorgiev, sorry for the radio silence, I’ve only just now gotten around to working on this.

Just to give you an update, I’ve got chunked file uploads implemented (3acc20b0ee8b36111cd88dd840df271415a99711) and been running a couple manual tests successfully. I still need to look into updating the unit tests before I can open a PR.

Do you have an ETA on a v2 release yet?

muecs on Aug 20, 2019

Hey guys, I am working on this to finish it.

VelinGeorgiev on Jan 21, 2020

@garrytrinder Of course; my work so far is on this branch: https://github.com/muecs/office365-cli/commits/file-add-chunking

muecs on Nov 19, 2019

@garrytrinder Not at present, although I have been using chunked uploads in production for over two months now. To open a PR only tests are still missing which I need to make time for.

muecs on Nov 11, 2019

I don’t mind, it’s a trivial change. The only downside I can see is that we’d always be using at least 2 API calls rather than just one, even for the tiniest file. But since I’ve implemented the StartUpload call to have no payload (rather than the first chunk), at least there is immediate feedback whether an upload is actually permitted/possible.

muecs on Aug 22, 2019

That is great @muecs, thank you ! ETA on v2 release is September. @waldekmastykarz when in September? First Sunday of Sept ? 😄

VelinGeorgiev on Aug 20, 2019