payload: "Large" file uploads results in: MongoServerError: Transaction with { txnNumber: 31 } has been aborted.
Link to reproduction
No response
Describe the Bug
I’ve successfully created, configured, and tested a media collection that has no problem uploading images, resizing them, and forwarding them to my S3 bucket using the @payload/plugin-cloud-storage package. There’s no issue with how the collection functions, unless I try to upload a video with a filesize of ~180MB.
The nova function is just a thin sugar-wrapper on my fetch request.
The request hangs for about 3 minutes and then I see the following in the Payload console logs:
And the very helpful error message in the response from the request:
This is obviously some sort of MongoDB timeout, but I’m confused as to why a MongoDB timeout would have any impact on a file upload at all. Is there a MongoDB transaction that starts at the onset of the file upload, and then it doesn’t finish until the file upload is done?
Nonetheless, I tried to increase the upload.limits.filesize value in my payload.config.ts file, but the issue persists.
I’m looking to use Payload to handle uploads of up to 5GB. Is this possible with Payload?
To Reproduce
Create a media collection and attempt to upload a large file via the REST API interface.
Payload Version
2.8.1
Adapters and Plugins
No response
About this issue
- Original URL
- State: open
- Created 4 months ago
- Reactions: 1
- Comments: 23 (3 by maintainers)
Could it be the transactionLifetimeLimitSeconds?
The transaction will initialize at the start of the request, but it won’t commit until after the file completes. I think this is the underlying issue and something we need to consider.
The upload code is written this way, because if you were trying to upload a file and it fails, we wouldn’t want it to have a collection document. It sounds like we need to rethink this approach and introduce an intermediate db commit with a delete cleanup for failing uploads.
@BrianJM I just did some more testing with a 21Mb file and an 80Mb file.
The 21Mb file uploaded fine and produced no errors. Payload performed as expected.
The 80Mb file produced the same errors as the original 180Mb file. The file itself landed fine in my S3 Bucket, but the MongoDB error was thrown, and there’s no correlating doc in my
mediacollection.@BrianJM Disabled transactions and still getting the same error:
Interestingly enough, the file actually does upload successfully to my S3 bucket:
However, I’m still getting the mongo error and there is no doc for the uploaded file in the DB.