next.js: Server-Sent Events don't work in Next API routes
Bug report
Describe the bug
When using Next’s API routes, chunks that are written with res.write aren’t sent until after res.end() is called.
To Reproduce
Steps to reproduce the behavior, please provide code snippets or a repository:
- Create the following API route in a Next app:
export default async (req, res) => {
let intervalID = null
res.setHeader('Content-Type', 'text/event-stream')
res.write('data: CONNECTION ESTABLISHED\n')
const end = () => {
if (intervalID) {
clearTimeout(intervalID)
}
}
req.on('aborted', end)
req.on('close', end)
const sendData = () => {
const timestamp = (new Date).toISOString()
res.write(`data: ${timestamp}\n`)
}
intervalID = setInterval(sendData, 1000)
}
- Connect to the route with a tool that supports Server-Sent Events (i.e. Postwoman).
Expected behavior
The route sends a new event to the connection every second.
Actual behavior
The route doesn’t send any data to the connection unless a call to res.end() is added to the route.
System information
- OS: macOS
- Version of Next.js: 9.1.5
Additional context
When using other HTTP frameworks (Express, Koa, http, etc) this method works as expected. It’s explicitly supported by Node’s http.incomingMessage and http.ServerResponse classes which, from what I understand, Next uses as a base for the req and res that are passed into Next API routes.
I’d hazard a guess that #5855 was caused by the same issue, but considered unrelated because the issue was obscured by the express-sse library.
There are also two Spectrum topics about this (here and here) that haven’t garnered much attention yet.
Supporting Websockets and SSE in Next API routes may be related, but fixing support for SSE should be a lower barrier than adding support Websockets. All of the inner workings are there, we just need to get the plumbing repaired.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 3
- Comments: 30 (5 by maintainers)
Commits related to this issue
- Added "no-transform" to "Cache-Control" header Some proxies buffers SSE events to compress the stream. The 'no-transform' directive forbidden this bad behavior. More info at: https://github.com/fa... — committed to nahuel/express-sse by nahuel 6 years ago
- Added "no-transform" to Cache-Control header Some proxies buffers SSE events to compress the stream, delaying the browser event reception. The `no-transform` `Cache-Control` directive forbids this ba... — committed to nahuel/fastify-sse-v2 by nahuel 6 years ago
- Added "no-transform" to Cache-Control header Some proxies buffers SSE events to compress the stream, delaying the browser event reception. The `no-transform` `Cache-Control` directive forbids this ba... — committed to mpetrunic/fastify-sse-v2 by nahuel 6 years ago
Thanks for all the effort on this ticket, folx! I wanted to pop in and say I think we can mark it as resolved. Here’s a quick TL;DR:
Code from above comment, if you want to try
Hi, server-sent events from an API endpoint appear to be working correctly in Next.js itself. Compression shouldn’t be affecting the stream as long as you set
res.setHeader('Cache-Control', 'no-cache, no-transform')specifically theno-transformbit, related compression code.Note: they will not work in a serverless environment since those environments are typically buffered and don’t allow streaming the response from the lambda. Related AWS Lambda docs here. If you want to create a pub-sub system, services like pusher.com are better suited for this and compliment deploying your applications on ZEIT or other serverless environments very well
Here’s a gif of it working locally without any custom
next.config.jsor a custom-server@trezy It seems the issue is that the middleware adds a gzip encoding which the browser has negotiated using the header:
If you add
then it seems to work:
@msand The main reason I’m trying to avoid using a custom server is that I’m deploying to Now. Using a custom server would break all of the wonderful serverless functionality I get there.
Your second point is fair. What I’m trying to do is create an SSE stream for data that would otherwise be handled with basic polling. The server is already dealing with constant reconnections in that case, so an SSE stream actually results in fewer reconnections.
I suppose I could set up a small webserver in the same repo that just uses a separate Now builder. That would allow the processes to remain separate, though it’d still cause all of the SSE connections to abort and reconnect when there are any changes to the project.
Even with those points, I can see plenty of scenarios in which it makes sense to be able to run an SSE endpoint from one of Next’s API routes. Additionally, in the docs it’s specifically stated that…
Since it’s specifically stated that
resis an instance ofhttp.ServerResponse, I’d expect it to behave exactly the wayhttp.ServerResponsebehaves in any other circumstance. Either the documentation should change to reflect the quirks of the implementation or, preferably,res.writeshould be fixed to behave the way it does in any other circumstance.I have switched to using a custom express server. That’s the only way I could get it to work. I guess that’s cool since I can do more with express.
Before deciding to integrate express, I had tried the things mentioned above, none worked.
Turned off gzip compression by setting the option in next.config.js. The behavior remained the same. I inspected the headers on the client (using postman) and confirmed the gzip encoding was removed, but that didn’t seem to fix the problem.
Calling res.flush had no effect either. Instead I get a warning in the console that flush is deprecated and to use flushHeaders instead. But that’s not what I want.
This is a rather strange bug… 😔
On Thursday, 9 January 2020, Mikael Sand notifications@github.com wrote:
Thanks! This was causing issues when deployed in OpenShift. I’m returning this header with “none” from my Spring WebFlux SSE Service and is working as expected now!
I have been trying to get SSE work in nextjs, but could not get working. With custom server and native node httpServer req,res it works, but with Nextjs ‘res’, no messages are sent to the client. I started using Nextjs to get the advantages of server-side rendering, SLS and having the server and client code together. Using express beats the purpose Any pointers on how this could work? This is a blocking problem for me