trpc: bug: Stream Link does not abort
Provide environment information
System:
OS: Windows 11 10.0.22631
CPU: (32) x64 AMD Ryzen 9 7950X 16-Core Processor
Memory: 46.84 GB / 63.65 GB
Binaries:
Node: 20.13.1 - C:\P\apps\nodejs\node.EXE
Yarn: 1.22.19 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
npm: 10.5.2 - C:\P\apps\nodejs\npm.CMD
Browsers:
Edge: Chromium (126.0.2592.81)
Internet Explorer: 11.0.22621.3527
npmPackages:
@trpc/client: next => 11.0.0-rc.436+fbb4a2fb4
@trpc/server: next => 11.0.0-rc.436+fbb4a2fb4
typescript: ^5.5.0 => 5.5.3
Describe the bug
The AbortController signal seems ignored by Stream Links with async generators.
We’d expect an interruption on the server side, and on the client side, but packets continue to be produced by the async generator, flow to the client, and the client keeps awaiting and receiving them.
I don’t know what the correct behavior would be, but right now the signal seems to be completely ignored on both ends.
Link to reproduction
https://github.com/enricoros/trpc-repro-stream-unabort
To reproduce
In summary:
- The client creates a Stream Link, and queries with a
{signal: ac.signal}, like this (minimal):
const trpc = createTRPCClient<AppRouter>({
links: [unstable_httpBatchStreamLink({url: 'http://localhost:3000',})],
});
const ac = new AbortController();
const iterable = await trpc.examples.iterable.query(undefined, { signal: ac.signal });
- The server implements a procedure with an async generator, like this (minimal):
iterable: publicProcedure.query(async function* () {
for (let i = 0; i < 8; i++) {
await new Promise((resolve) => setTimeout(resolve, 500));
yield i;
}
}),
- We would expect both the server and the client to be aborted, but execution continues on both.
This is the output:
Additional information
No response
👨👧👦 Contributing
- 🙋♂️ Yes, I’d be down to file a PR fixing this bug!
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 16 (15 by maintainers)
Thank you @enricoros, this whole interaction with you has been 10/10
@KATT thanks, this is huge news! When my startup gets funding I’ll pay it forward and sponsor the project.
So far, I’m incredibly happy with Streaming via tRPC, especially because it induces correctness upstream via generators/async generators. The new API is so clean, and have infinite room for scalability, which I’m still early in exploring. Cannot wait to remove all my old stream transformation code and manual endpoints. Streaming is such a great tRPC use case because of the flexibility of return types; it’s not one .output(), but many object types generated on the fly with no cost.
I wish I had the time to make my tRPC AI library a bit more independent from the rest of the App, because it’s a more flexible alternative to the Vercel AI sdk (multi-modal, multi-part, dynamic upstream streaming, 14 vendors, etc.)
I hated how I did it so I ended up spending 2-3 hours refactoring after 😬
Thanks, I understand it’s not easy - this is very advanced. For my usage I don’t need the batching part, I don’t know if there could be a simplification where abort() terminated all the connections of the batch.
The other issue is that after throwing on the server side, the connection seems to stay open (while it closes without throwing) – maybe it’s related to batching too.
I’ve added a bounty using polar. I understand it’s not much, but it’s basically my GitHub income for the month (and beats the issues with no bounty! 😃.
Update: Testing ⌛complete.
Works perfectly. I confirm that the signal passed from the context
request.signalworks really well.As an example, I started 4 OpenAI Generations and interrupted them right at the beginning, and the servers show the following stats:
This shows that the signal is really used from the request, passed to the fetch which terminates the upstream connection, and the user saves tokens - input (context) tokens >> output (generated) tokens.
As of now, the client and server work great. I don’t see any other improvement (I didn’t check if
error.name = 'AbortError', but it can be worked around too).Thanks, and glad to honor the pledge.