vapor: Precondition failure when write request body into file asyncronously.

Describe the bug

I’m implementing upload functionality with HTTPBodyStreamStrategy == .stream. Inside request handler i’m writing data to file in chunks ayncronously. When large file is uploaded precondition failure is executed.

Here is my implementation:

import Vapor
import NIO

func routes(_ app: Application) throws {
    app.on(.POST, "upload", body: .stream, use: { req async throws in
        var offset: Int64 = 0
        let uploads = req.application.directory.workingDirectory.appending("uploads")
        let outPath = "\(uploads)/file"
        try? FileManager.default.createDirectory(atPath: uploads, withIntermediateDirectories: false)
        try? FileManager.default.removeItem(atPath: outPath)
        let fileHandle = try NIOFileHandle(path: outPath, mode: .write, flags: .allowFileCreation())

        for try await buffer in req.body {
            try await req.application.fileio.write(fileHandle: fileHandle,
                                                   toOffset: offset,
                                                   buffer: buffer,
                                                   eventLoop: req.eventLoop).get()
            offset += Int64(buffer.readableBytes)
        }

        try fileHandle.close()
        return "ok"
    })
}

To Reproduce

  1. create large file (600M): mkfile -n 600m file
  2. curl -X POST 'http://localhost:8080/upload' --data-binary '@file'
  3. see error error

Also i noticed that error appear after repeating curl request several times on smaller files, like 100M.

Expected behavior

No error should happen.
If i use futures implementation, everything works fine.

Environment

  • Vapor Framework version: 4.74.1
  • Vapor Toolbox version: 18.6.0
  • OS version: macOS 13.2 (22D49)

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 2
  • Comments: 25 (9 by maintainers)

Most upvoted comments

Just an FYI we’re working on this in #3232

Also - this is a showstopper bug. There should be a note in the documentation not to use this feature, as this will crash production servers - until a resolution is found.

I’m getting the same crash with 4.102.1, any chance of getting this fixed asap as my whole project depends on Vapor not crashing?

@hsharghi we’ve just released https://github.com/vapor/vapor/releases/tag/4.92.3 which has the fix in it

This should now be fixed, feel free to reopen if not!

I am able to reproduce it without any fileio:

app.on(.POST, "upload", body: .stream, use: { request async throws in
    let data = request.body.collect(upTo: Int.max)
}

This alone is enough to hit the .noSignalReceived and crash the server on 4.102.1.

~I wonder if Vapor running behind nginxproxy is causing some issues here because I can observe a different behavior in Xcode vs prod. In Xcode the body.collect() is relatively fast (~200ms for 100MB file), but on prod this takes ~1min with the same file and actually crashes before it finishes.~

UPDATE: Turns out this nginxproxy doesn’t play a role. The mentioned 1min was not spent in .collect() but in swift-backtrace --unwind after a crash.

Just encountered this in a very large project. We’re gonna have to rip out async code to fix as we can’t have our server crashing in production. Does anyone have any sense of what’s causing this?

@0xTim The issue still exists

Vapor/RequestBody+Concurrency.swift:34: Fatal error

preconditionFailure()              NIO-SGLTN-0-#5 (10): Fatal error