async-http-client: `HTTPClient.execute(...) async throws` API violates Structured Concurrency

Structured Concurrency mandates that upon the return of a function any background work it may have started has completed.

With HTTPClient.execute(...) async throws that’s clearly not the case as it returns a half-finished HTTPClientResponse where the response.body yet has to arrive.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 1
  • Comments: 17 (9 by maintainers)

Most upvoted comments

Wait, wait, who would want to adopt ~Escapable for connections? That doesn’t make any sense to me. That makes them useless.

non-escaping (probably) makes sense for stuff like iterators. Stuff that’s quick to create/discard, usually wrapping something else.

IMO anything that we currently provide in a with style scope makes sense to adopt ~Escapable to get exactly those guarantees enforced by the compiler that we currently only enforce by API shape. ~Escapable will still allow to pass the types into structured child tasks just not outside of the current task. Of course this is mostly hypothetical at this point.

But doing lateral communication in order to save a few ARCs feels backwards. You’d be trading something super fast (ARC, maybe 2ns per retain/release pair) for something awkard, complex and much slower that’s also borderline at odds with the structured concurrency model (lateral comms).

I agree that we can transfer ownership for the period of a request for performance reasons; however, lateral communication will always exist in any system. Different tasks have to communicate with each other in larger applications. Doesn’t mean we have to use for a connection pool but I still expect it to be a common pattern between tasks and not violate structured concurrency.

I’m not sure I agree here. In structured concurrency multiple ways can lead to Rome! The question is how do we structure the concurrency:

Not quite. Structured Concurrency doesn’t mean you get to pick the structure, the structure of your code (the { and }s).

Quotes from

Wikipedia on Structured Concurrency

The core concept is the encapsulation of concurrent threads of execution (here encompassing kernel and userland threads and processes) by way of control flow constructs that have clear entry and exit points and that ensure all spawned threads have completed before exit.

SE-0304:

[…] child-tasks and respected the primary rule of structured concurrency: that a child task cannot live longer than the parent task (or scope) in which it was created.

So the one thing that mustn’t happen is that you pass your scope’s } and still have stuff running. Very clearly, that’s exactly what execute is currently doing.

In AHC the execute function aligns from request head to response head. If you cancel the execute the request is cancelled. If you drop the response body the request is cancelled and the body won’t be loaded. The same is true if you drop the AsyncIterator eventually. So I think AHC is totally structured. It might be structured differently than what you expect but it is structured.

It may have a structure but it’s not the structure that Structured Programming and Structured Concurrency mandate.

All the bad stuff that’s supposed to be impossible with Structured Concurrency can happen with execute.

For example:

  • You can accumulate background work without creating child tasks
  • You can create a function that still has background work running when you return (without explicitly going unstructured with Task { ... } yourself)
  • You can create uncancellable work

For example if you do

var baddieGlobalResponseBag: NIOLockedValueBox<[HTTPClientResponse]> = .init([])

func myFunction() async throws {
    let response = try await HTTPClient.shared.execute(...)
    baddieGlobalResponseBag.withLockedValue { bag in
        bag.append(response)
    }
    return // This return not wait for the work, it becomes uncancellable too
}

And yes, my program is stupid and wrong. But the thing about structured concurrency is that this shouldn’t matter. The compelling thing is that I’m meant to get a ‘guarantee’ that when I returned I will have cleaned up (or still waiting for it), HTTPClient.execute breaks this.

If you will, you can liken this to structured programming. If you write

while true {
   print("hello")
   return
}
print("bye")

then you know that it’ll print hello once and then bye. You don’t need to read some docs to see if maybe the loop continues after the function returned.

And this is despite the fact that the loop condition (true) never goes false and despite the fact that I never break the loop. The loop still stops. Structured Concurrency is meant to work the same way.

The loop continuing after you hit the enclosing } is just not possible because in Structured Programming the control flow follows the structure of your code (the { and }s).

Yes, I brought this up as well. The async APIs are not structured concurrency “conform”. We spawn an unstructured Task inside them and that task is “driving” the request. This leads to exactly this problem. We need to provide a HTTPClient.execute(request: body:) scoped method which we can have overloads for that just accumulate the whole body into a single buffer (with size limits for security)